<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
    <title>@bnjbvr - opensource</title>
    <subtitle>Technical blog and random musings.</subtitle>
    <link rel="self" type="application/atom+xml" href="https://bouvier.cc/tags/opensource/atom.xml"/>
    <link rel="alternate" type="text/html" href="https://bouvier.cc"/>
    <generator uri="https://www.getzola.org/">Zola</generator>
    <updated>2025-05-07T00:00:00+00:00</updated>
    <id>https://bouvier.cc/tags/opensource/atom.xml</id>
    <entry xml:lang="en">
        <title>Une lettre d&#x27;amour pour le Zen Browser</title>
        <published>2025-05-07T00:00:00+00:00</published>
        <updated>2025-05-07T00:00:00+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/zen-browser-love-letter/"/>
        <id>https://bouvier.cc/tech/zen-browser-love-letter/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/zen-browser-love-letter/">&lt;p&gt;Comment je suis passé de Firefox comme navigateur Web principal au Zen Browser.&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;&lt;h2 id=&quot;grandeur-et-decadence-des-pandas-roux&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#grandeur-et-decadence-des-pandas-roux&quot; aria-label=&quot;Anchor link for: grandeur-et-decadence-des-pandas-roux&quot;&gt;🔗&lt;&#x2F;a&gt;grandeur et décadence des pandas roux&lt;&#x2F;h2&gt;
&lt;p&gt;&lt;em&gt;Longtemps, j’ai utilisé le navigateur Web Firefox de bonne heure&lt;&#x2F;em&gt;. Depuis mon adolescence, environ. À l’époque, parce qu’il révolutionnait les performances dans les navigateurs, face à Internet Explorer 6, permettant de naviguer à toute vitesse, avant que Google Chrome n’arrive et ne chamboule tout. Plus tard, je suis parti pour Chrome, finalement revenu pour Firefox, pour ses promesses quant à la protection de la vie privée des utilisateur·ices, son absence d’intérêts commerciaux à la direction de Mozilla. Encore plus tard, j’ai même eu la &lt;em&gt;chance&lt;&#x2F;em&gt; de travailler pour Mozilla, d’apprécier la puissance intellectuelle &lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-1-1&quot;&gt;&lt;a href=&quot;#fn-1&quot;&gt;1&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; des devs de Firefox et des autres produits de Mozilla, de pouvoir discuter avec les divers PDG, voire l’une des personnes à l’origine de l’initiative Mozilla &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Mitchell_Baker&quot;&gt;Mitchell Baker&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;Mais tout n’a pas été rose tout du long. Même en tant qu’utilisateur, j’ai eu plusieurs fois des remontées acides, des grimaces de dégoût, et ce, indépendamment de mon licenciement économique en 2020 (causé par La Crise™, qui frappait fortement le monde de la publicité du fait des confinements liés au COVID19). Le rachat de &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;getpocket.com&quot;&gt;Pocket&lt;&#x2F;a&gt; a été pour moi très marquant : un produit pas open-source, qui a galéré à le devenir complètement (de mémoire, la mise à disposition du code serveur est arrivée au cours de l’année 2024, quasiment plus de 10 ans après le rachat). Pendant ce temps-là, une autre équipe en interne avait créé une liste d’articles à lire plus tard, &lt;em&gt;directement intégrée dans le navigateur&lt;&#x2F;em&gt;. Cette même équipe était en train de créer des fondations techniques très solides et une vision cohérente pour une éventuelle plateforme de &lt;em&gt;cloud personnel&lt;&#x2F;em&gt;, à une période où je rêvais d’un &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;nextcloud.com&quot;&gt;Nextcloud&lt;&#x2F;a&gt; ou d’un &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;cozy.io&#x2F;fr&#x2F;&quot;&gt;CozyCloud&lt;&#x2F;a&gt; pour tous·tes, et où je me racontais que Mozilla avait la possibilité de vraiment changer les choses. Si seulement nous mettions à disposition des services cloud personnels estampillés Mozilla, absolument tout libres, infogérés par Mozilla, nous aurions même un angle &lt;em&gt;business&lt;&#x2F;em&gt; possible derrière ça : fournir quelques méga- ou giga-octets de stockage gratuit, puis faire payer les suivants. Mais d’autres décisions techniques et organisationnelles m’ont également marqué. Les virages et détours au sujet de Firefox OS, qui a finalement vécu sa meilleure vie une fois extirpé de Mozilla. Le choix de désinvestir dans Rust et Servo (un moteur de navigateur réécrit de zéro, en Rust), quand d’autres projets qui me semblaient moins prometteurs mais beaucoup plus « dans l’air du temps » (entendez : 🤮 metaverse 🤮 ou IA 🤮 déjà) ont été largement préférés. Pas besoin de revenir sur les différents épisodes, ou de trop me justifier ; disons que les derniers en date, notamment la cacophonie sur la notice liée à la vie privée, et l’arrivée d’IA nativement intégrée, ont un peu fini de me dégoûter, et ont rendu très compliqué pour moi le fait de continuer de défendre Mozilla malgré tout.&lt;&#x2F;p&gt;
&lt;p&gt;Parce que oui, je continuerai à défendre &lt;em&gt;l’outsider&lt;&#x2F;em&gt; Mozilla, tant au niveau technologique ou au niveau organisationnel. Technologique, parce que c’est l’un des trois derniers participants à encore créer un &lt;em&gt;moteur de rendu&lt;&#x2F;em&gt;, en gros toute la machinerie qui permet de faire un navigateur Web, et l’un des deux à tenter de combattre l’hégémonie de celui de Google. Organisationnel, parce qu’il s’agit (toujours, malgré tout) d’une entreprise « à mission »&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-2-1&quot;&gt;&lt;a href=&quot;#fn-2&quot;&gt;2&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; adossée à une fondation. Pas d’actionnaires auprès de qui il serait nécessaire de rendre des comptes ou d’offrir des dividendes ; pas d’objectif de s’enrichir à milliards. Juste une nécessité de gagner assez d’argent pour se maintenir en vie, et &lt;em&gt;faire des trucs™&lt;&#x2F;em&gt; : payer les devs, proposer des initiatives d’éducation populaire au numérique et aux enjeux de protection de la vie privée.&lt;&#x2F;p&gt;
&lt;p&gt;Du moins, ça c’est là où je les attendrais ; en pratique, l’entreprise a tendance à s’éparpiller, à s’accrocher à n’importe quel qualificateur porteur de &lt;em&gt;hype&lt;&#x2F;em&gt; de la Silicon Valley, et essayer d’en tirer un profit pour se désolidariser du financement de Google pour la publicité offerte. D’où le metaverse ; d’où les initiatives un peu surprenantes sur l’IA. Je comprends le pourquoi et partage la motivation de prendre de la distance par rapport au financement via Google. Ça en devient absurde : aux USA, des procès &lt;em&gt;antitrust&lt;&#x2F;em&gt; ont actuellement lieu pour empêcher Google de financer les navigateurs à coups de millions pour être le moteur de recherche principal, ce afin de permettre aux &lt;em&gt;autres&lt;&#x2F;em&gt; moteurs de recherche d’être plus compétitifs™. Mais c’est une telle source de revenus pour Mozilla, que &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.omgubuntu.co.uk&#x2F;2025&#x2F;05&#x2F;mozilla-says-google-search-deal-vital-to-firefoxs-survival&quot;&gt;Mozilla en vient à &lt;em&gt;défendre&lt;&#x2F;em&gt; Google pour que Big G puisse continuer ce genre de pratiques&lt;&#x2F;a&gt;, sinon quoi la survie de Firefox (via son financement) serait menacée directement. Paradoxe de la compétition en effet. Bref, vivement que Mozilla puisse être plus tranquille financièrement, et idéalement pas en vendant de l’IA ou des produits inutiles ou en prenant des décisions qui ont l’air &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.zdnet.com&#x2F;article&#x2F;the-firefox-i-loved-is-gone-how-to-protect-your-privacy-on-it-now&#x2F;&quot;&gt;au mieux absurdes au pire effrayantes&lt;&#x2F;a&gt;, et aussi je voudrais la paix dans le monde et un poney, ce sera tout, un café et l’addition merci.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;ok-bon-alors-on-va-ou-on-fait-quoi&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#ok-bon-alors-on-va-ou-on-fait-quoi&quot; aria-label=&quot;Anchor link for: ok-bon-alors-on-va-ou-on-fait-quoi&quot;&gt;🔗&lt;&#x2F;a&gt;ok bon, alors on va où, on fait quoi ?&lt;&#x2F;h2&gt;
&lt;p&gt;Les plus malin·gnes auront lu le titre de l’article, et les plus pressé·es pourront aller au sous-titre suivant. En attendant, un peu d’état de l’art sur les navigateurs existants.&lt;&#x2F;p&gt;
&lt;p&gt;Si vous êtes sur un MacOS ou un iOS, je pense assez sincèrement qu’Apple fait un bon boulot de protection de la vie privée avec Safari, et ça peut être une solution &lt;em&gt;satisfaisante&lt;&#x2F;em&gt;, si vous ignorez le fait que l’entreprise cherche à tout prix à vous enclaver dans son jardin doré, et fera tout pour éviter de vous faire voir ailleurs. Après tout, une application Web, c’est une application qui n’est pas dans l’App Store d’Apple, et qui donc ne rapporte pas d’argent à Apple, pourquoi s’embêter à soutenir une telle plateforme 😏&lt;&#x2F;p&gt;
&lt;p&gt;Et sinon, eh bien on ne va pas aller chez Chrome non plus, hein. Ce serait comme dire « ohlala la démocratie est en danger en France, si c’est comme ça je me casse en Chine &#x2F; Corée du Nord &#x2F; États-Unis&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-3-1&quot;&gt;&lt;a href=&quot;#fn-3&quot;&gt;3&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; ».&lt;&#x2F;p&gt;
&lt;p&gt;Émerge donc un besoin de cohérence, pour moi : je veux continuer à utiliser &lt;em&gt;Gecko&lt;&#x2F;em&gt;, le moteur de rendu de Firefox, et je veux continuer à utiliser un navigateur qui protège ma vie privée (et ne s’engouffre pas dans l’IA à la première occasion). Il existe bien des variantes de Firefox (&lt;em&gt;forks&lt;&#x2F;em&gt;) dans le monde du libre : &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.gnu.org&#x2F;software&#x2F;gnuzilla&#x2F;&quot;&gt;IceCat&lt;&#x2F;a&gt; (anciennement &lt;em&gt;IceWeasel&lt;&#x2F;em&gt;), &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;librewolf.net&#x2F;&quot;&gt;LibreWolf&lt;&#x2F;a&gt; me viennent à l’esprit, mais il en existe également d’autres d’après &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Gecko_(software)#Usage&quot;&gt;cette page Wikipédia&lt;&#x2F;a&gt;. Ces &lt;em&gt;forks&lt;&#x2F;em&gt; ont tendance à réutiliser principalement les mêmes concepts de navigateurs tels que nous les connaissons, et s’évertuent à supprimer les fonctionnalités dont l’impact est considéré comme étant néfaste vis-à-vis de la vie privée : télémétrie, toutes les méthodes directes ou indirectes de &lt;em&gt;tracking&lt;&#x2F;em&gt; même si c’est pour Mozilla, IA intégrée, et j’en passe et des meilleures.&lt;&#x2F;p&gt;
&lt;p&gt;D’aucuns disent à juste titre qu’abandonner Firefox pour aller sur un &lt;em&gt;fork&lt;&#x2F;em&gt;, c’est comme arrêter de boire de l’eau du robinet et ne consommer que l’eau mise en bouteille par le voisin un peu conspi et réac. Il y a effectivement un risque inhérent que les devs de ces projets s’amusent à mettre en place des bouts de code malveillants dans leur projet, ou le fassent par inadvertance. Le résultat est le même : les utilisateur·ices de ces forks seraient alors mis en danger, et l’utilisation d’un navigateur alternatif serait même contre-productive quant à l’intention initiale de mieux protéger sa vie privée des caprices des créateur·ices de Firefox. Mais la même question se poserait vis-à-vis de Firefox lui-même, si l’on ne savait pas que ce dernier est maintenu par la vénérable entreprise Mozilla ; la question de la confiance en le code est réglée, au vu des principes de l’entreprise. Ou « était » réglée ? L’intégration d’outils d’IA pourrait relever d’une trahison au niveau du code (la recette de fabrication de Firefox), remettant justement en question la confiance accordée par les utilisateur·ices. Je ne tiens pas à plonger dans l’aspect philosophique de la question. Pour moi, le fait que Firefox ou ses &lt;em&gt;forks&lt;&#x2F;em&gt; soient &lt;em&gt;open-source&lt;&#x2F;em&gt; garantit qu’il n’y aura pas de mauvaise surprise, parce que des gens plus paranoïaques que moi vont vérifier que le code ne fait pas n’importe quoi, à plus ou moins grande échelle. Cependant, même avec les plus belles intentions du monde, il est important que ces &lt;em&gt;forks&lt;&#x2F;em&gt; se remettent à jour sur Firefox régulièrement et rapidement. Je reviens là-dessus plus bas.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;www.davidrevoy.com&#x2F;data&#x2F;images&#x2F;blog&#x2F;2025&#x2F;2025-02-27_how-my-firefox-became-a-librewolf.jpg&quot; alt=&quot;Illustration BD de LibreWolf par le talenteux David Revoy&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;Pour un usage général, je pense que l’option &lt;em&gt;LibreWolf&lt;&#x2F;em&gt; ferait très bien le boulot : pas d’intégration d’outils de protection de la vie privée qui rendraient l’usage imbitable pour des néophytes, des bons choix par défaut, très sensés, recommandé dans le dessin ci-dessus par le brillant &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.davidrevoy.com&#x2F;&quot;&gt;David Revoy&lt;&#x2F;a&gt;, bref, c’est Très Bien&#x2F;10.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;pendant-ce-temps-la-a-vera-cruz&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#pendant-ce-temps-la-a-vera-cruz&quot; aria-label=&quot;Anchor link for: pendant-ce-temps-la-a-vera-cruz&quot;&gt;🔗&lt;&#x2F;a&gt;pendant ce temps-là, à Vera Cruz,&lt;&#x2F;h2&gt;
&lt;p&gt;Pendant ce temps-là, donc, du côté des navigateurs pas open-source et conçus par des entreprises qui veulent engranger de la Grosse Caillasse™, il y a un petit nouveau, nommé &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;arc.net&#x2F;&quot;&gt;Arc Browser&lt;&#x2F;a&gt;. Bien entendu, basé sur Chrome. Mais qui apporte un peu de renouveau et de fraîcheur dans le monde des navigateurs. Par exemple, cette barre d’onglets &lt;em&gt;verticale&lt;&#x2F;em&gt; sur le côté, sans barre d’URL au dessus du site, pour une esthétique assez minimaliste, qui donne toute la place au site. Des &lt;em&gt;espaces&lt;&#x2F;em&gt; différents, avec des répertoires d’onglets, pour contenir et regrouper les innombrables onglets ouverts navigués dans des contextes différents, et limiter l’expansion de la charge cognitive de tout avoir sous les yeux tout le temps. La possibilité de découper la vue de la page en deux, pour pouvoir naviguer sur deux sites différents, côte-à-côte.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;arc.net&#x2F;theme-picker.png&quot; alt=&quot;Capture d’écran du navigateur Arc&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;Miam miam ; enfin un peu d’innovation dans les navigateurs, qui n’est pas sans rappeler &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;vivaldi.com&#x2F;fr&#x2F;&quot;&gt;le navigateur Vivaldi&lt;&#x2F;a&gt; (aussi basé sur Chrome 🥲). Au final, quels sont ses défauts, pour moi ? D’une part il est basé sur Chrome, et participe de l’hégémonie de son moteur de rendu. D’autre part, il est maintenu par une entreprise qui injecte de l’IA à toutes balles dans la version mobile du navigateur. Enfin, il n’est pas disponible sous Linux, système d’exploitation que j’affectionne tout particulièrement pour l’utiliser au quotidien 🤓&lt;&#x2F;p&gt;
&lt;h2 id=&quot;la-lettre-d-amour-rohlala-enfin-on-n-en-pouvait-plus&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#la-lettre-d-amour-rohlala-enfin-on-n-en-pouvait-plus&quot; aria-label=&quot;Anchor link for: la-lettre-d-amour-rohlala-enfin-on-n-en-pouvait-plus&quot;&gt;🔗&lt;&#x2F;a&gt;la lettre d’amour (rohlala enfin, on n’en pouvait plus)&lt;&#x2F;h2&gt;
&lt;p&gt;Et c’est là qu’entre en jeu &lt;del&gt;notre seigneur et sauveur&lt;&#x2F;del&gt;&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-5-1&quot;&gt;&lt;a href=&quot;#fn-5&quot;&gt;4&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; le &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;zen-browser.app&#x2F;&quot;&gt;Zen Browser&lt;&#x2F;a&gt;. Pas besoin de tergiverser, vu que sa conception semble directement inspirée d’&lt;em&gt;Arc&lt;&#x2F;em&gt;, il contient peu ou prou les mêmes fonctionnalités, et même mieux :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;Ce navigateur est &lt;strong&gt;basé sur Firefox et donc Gecko&lt;&#x2F;strong&gt;. En soi, il soutient donc indirectement la puissance technologique de Mozilla. Plus pragmatiquement, il est &lt;em&gt;compatible avec l’écosystème d’addons Firefox&lt;&#x2F;em&gt; (si vous n’avez pas encore installé &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;addons.mozilla.org&#x2F;en-US&#x2F;firefox&#x2F;addon&#x2F;ublock-origin&#x2F;&quot;&gt;uBlock Origin&lt;&#x2F;a&gt;, allez-y, prenez votre temps, je vous attends tranquillement ici) et permet de &lt;em&gt;se synchroniser avec un compte Mozilla&lt;&#x2F;em&gt; classique (pour partager ses favoris &#x2F; mots de passe etc. avec les autres instances de &lt;em&gt;Zen Browser&lt;&#x2F;em&gt;… ou de Firefox !).&lt;&#x2F;li&gt;
&lt;li&gt;Il est &lt;strong&gt;disponible sur Linux, MacOS et Windows&lt;&#x2F;strong&gt;, donc globalement partout.&lt;&#x2F;li&gt;
&lt;li&gt;Il y a donc une &lt;strong&gt;barre d’onglets verticale&lt;&#x2F;strong&gt;, sur le côté, qui laisse toute la place au contenu. Pas de barre d’URL en haut ; celle-ci existe bien, mais elle est plus discrète, en haut de la pile des onglets. Il est même possible de déplacer cette barre d’onglets sur la droite de l’écran. J’adore ça, personnellement, puisque ça donne la place principale sur mon écran pour l’application Web que je suis en train de regarder, et ça a même tendance à me faire croire que l’application Web est en fait une app native.&lt;&#x2F;li&gt;
&lt;li&gt;En appuyant sur &lt;code&gt;Ctrl+Alt+C&lt;&#x2F;code&gt;, j’enclenche le &lt;strong&gt;mode compact&lt;&#x2F;strong&gt;, qui cache même cette barre d’onglets, pour n’avoir vraiment plus que le site en face des yeux. C’est très reposant et très &lt;em&gt;zen&lt;&#x2F;em&gt; (j’approuve le nom &lt;em&gt;de facto&lt;&#x2F;em&gt;).&lt;&#x2F;li&gt;
&lt;li&gt;Il est aussi possible d’&lt;strong&gt;épingler&lt;&#x2F;strong&gt; des sites favoris, ce qui va n’afficher que leur icône, en très gros, tout en haut de la barre d’onglets. Idéal pour les sites sur lesquels je reviens souvent, notamment le tchat Matrix, mes emails, le lecteur musical etc.&lt;&#x2F;li&gt;
&lt;li&gt;Le navigateur essaie très activement de &lt;strong&gt;décharger automatiquement les onglets&lt;&#x2F;strong&gt;, notamment ceux qui n’ont pas été visités depuis un petit moment (alors que le navigateur était ouvert). Dans la barre d’onglets verticaux, il est même possible de séparer les onglets qui ne doivent pas être déchargés (au dessus d’une petite barre horizontale) de ceux qui peuvent l’être à n’importe quel moment (en dessous de cette même barre horizontale). Autant vous dire que la &lt;em&gt;mémoire vive&lt;&#x2F;em&gt; (RAM) de votre ordinateur appréciera 🥰&lt;&#x2F;li&gt;
&lt;li&gt;La &lt;strong&gt;gestion des onglets en espaces&lt;&#x2F;strong&gt; permet de compartimentaliser et de séparer les différents contextes liés à des onglets. Après quelques semaines d’utilisation sur cette machine, j’ai 3 espaces : un pour la navigation « au quotidien » que j’essaie de vider régulièrement, un pour &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;kresus.org&#x2F;&quot;&gt;Kresus&lt;&#x2F;a&gt; (🎶 un gestionnaire de finances personnelles libre, qu’il est trop bien 🎶) avec des liens vers des &lt;em&gt;merge requests&lt;&#x2F;em&gt; à relire ou finir, un pour des pages et sites que j’aimerais lire ou explorer plus tard, mais moins prioritaires.&lt;&#x2F;li&gt;
&lt;li&gt;&lt;strong&gt;Le découpage de la vue en plusieurs sous-vues&lt;&#x2F;strong&gt; (&lt;em&gt;split view&lt;&#x2F;em&gt;) est hyper pratique pour consulter deux pages en même temps. On clique sur deux onglets (ou plus), puis par exemple on presse &lt;code&gt;Ctrl+Alt+V&lt;&#x2F;code&gt; pour les afficher côte-à-côte verticalement. Dans mes cas d’utilisation récurrents, je noterai le fait de prendre des notes (dans &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;silverbullet.md&#x2F;&quot;&gt;Silverbullet&lt;&#x2F;a&gt; par exemple) pendant que je regarde une vidéo instructive sur Youtube (et oui) ; ou de faire une comparaison de deux sites au niveau de leur contenu. Il est possible de découper la vue jusque 4 fois, donc j’imagine aussi bien un cas d’usage de &lt;em&gt;tableau de bord&lt;&#x2F;em&gt; avec plein de sites importants qu’on veut garder sous les yeux.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;video width=&quot;100%&quot; height=&quot;100%&quot; controls&gt;
    &lt;source src=&quot;https:&#x2F;&#x2F;zen-browser.app&#x2F;_astro&#x2F;SplitViews.Cc2YPNMw.webm&quot; type=&quot;video&#x2F;webm&quot;&gt;
&lt;&#x2F;video&gt;
&lt;ul&gt;
&lt;li&gt;L’une de mes fonctionnalités favorites, c’est &lt;strong&gt;l’ouverture d’un lien en mode aperçu&lt;&#x2F;strong&gt; (&lt;em&gt;glance&lt;&#x2F;em&gt;). Pour cela, on clique sur un lien en maintenant pressé &lt;code&gt;Alt&lt;&#x2F;code&gt;. Cela ouvre une plus petite vue de ce site &lt;em&gt;au-dessus du site que l’on visite&lt;&#x2F;em&gt;, et cette petite vue peut être fermée via une presse de la touche &lt;code&gt;Échap&lt;&#x2F;code&gt; ou en cliquant en dehors de cette vue ; ce qui nous ramène au site précédent. Il est également possible d’étendre une vue &lt;em&gt;aperçu&lt;&#x2F;em&gt; en un véritable onglet, pour naviguer plus normalement dessus. Ça ne paraît pas grand chose comme ça, mais pour moi c’est &lt;em&gt;très très utile&lt;&#x2F;em&gt; : souvent je suis sur des sites avec beaucoup de liens, et j’en suis parfois certains pour aller voir ce qui se trame derrière, avant de revenir au site. Avec ce mode d’aperçu, je reste &lt;em&gt;dans le contexte&lt;&#x2F;em&gt; de ce que je regardais avant, et je passe moins de temps à jongler entre plein d’onglets ; je n’ai qu’un seul onglet ouvert tout du long de ma navigation.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;video width=&quot;100%&quot; height=&quot;100%&quot; controls&gt;
    &lt;source src=&quot;https:&#x2F;&#x2F;zen-browser.app&#x2F;_astro&#x2F;Glance.DMRcYdXi.webm&quot; type=&quot;video&#x2F;webm&quot;&gt;
&lt;&#x2F;video&gt;
&lt;ul&gt;
&lt;li&gt;Le &lt;em&gt;fork&lt;&#x2F;em&gt; est &lt;strong&gt;maintenu par des passionné·es, pas par une entreprise&lt;&#x2F;strong&gt; ; ce qui pourrait être un désavantage, mais l’activité autour du projet est très élevée. D’une part, beaucoup de &lt;em&gt;commits&lt;&#x2F;em&gt; récents dans &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;zen-browser&#x2F;desktop&#x2F;commits&#x2F;dev&#x2F;&quot;&gt;l’historique du dépôt de code&lt;&#x2F;a&gt;…&lt;&#x2F;li&gt;
&lt;li&gt;…Et ce qui est le plus important, selon moi : &lt;strong&gt;les mises à jour de &lt;em&gt;rattrapage&lt;&#x2F;em&gt; de Firefox sont hyper rapides&lt;&#x2F;strong&gt;. Le problème est le suivant, quand vous maintenez un navigateur basé sur Firefox, il est nécessaire de récupérer &lt;em&gt;régulièrement&lt;&#x2F;em&gt; les changements de votre projet source (on parle d’&lt;em&gt;upstream&lt;&#x2F;em&gt;), ici Firefox, afin de les intégrer dans votre projet. Si ce n’est pas fait assez rapidement, vous risquez très rapidement d’utiliser un logiciel qui a des failles de sécurité rendues publiques. &lt;em&gt;Or&lt;&#x2F;em&gt;, de ce que j’observe, les devs de Zen Browser ont tendance à intégrer les nouvelles versions de Firefox &lt;em&gt;très&lt;&#x2F;em&gt; vite ; la dernière version de Firefox (la 138.0.1) est sortie le &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.mozilla.org&#x2F;en-US&#x2F;firefox&#x2F;138.0.1&#x2F;releasenotes&#x2F;&quot;&gt;1er mai&lt;&#x2F;a&gt;, la version correspondante de Zen Browser (basée sur la 138.0.1) est sortie &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;zen-browser&#x2F;desktop&#x2F;releases&#x2F;tag&#x2F;1.12b&quot;&gt;le 3 mai&lt;&#x2F;a&gt; (voire le soir du 2 mai). Pour moi, c’est gage de qualité, d’autant plus que c’est donc maintenu par des gens qui font ça &lt;em&gt;sur leur temps libre, pour le plaisir&lt;&#x2F;em&gt;, c’est particulièrement honorable 🙏&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h2 id=&quot;oui-mais-quid-de-ceci-benjamin-quid-de-cela&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#oui-mais-quid-de-ceci-benjamin-quid-de-cela&quot; aria-label=&quot;Anchor link for: oui-mais-quid-de-ceci-benjamin-quid-de-cela&quot;&gt;🔗&lt;&#x2F;a&gt;oui mais quid de ceci, Benjamin, quid de cela ?&lt;&#x2F;h2&gt;
&lt;p&gt;Ohlala je vous vois arriver avec vos gros sabots. Non, le &lt;em&gt;Zen Browser&lt;&#x2F;em&gt; n’est pas parfait, et c’est OK. Oui, c’est risqué qu’un navigateur, un des logiciels les plus centraux dans nos utilisations bureautiques modernes, soit maintenu par une poignée de gentils geeks avec trop de temps libre. Oui, Firefox pourrait sûrement faire la même chose avec 15 personnalisations de &lt;code&gt;about:config&lt;&#x2F;code&gt;, un thème personnalisé en CSS et une plâtrée d’addons &lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-4-1&quot;&gt;&lt;a href=&quot;#fn-4&quot;&gt;5&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;. Je reviendrais sûrement sur Firefox, si jamais l’activité autour de &lt;em&gt;Zen Browser&lt;&#x2F;em&gt; venait à diminuer, ou bien au premier mouvement de drague à destination de l’IA, ou bien à la première faille de sécurité pas ou mal gérée.&lt;&#x2F;p&gt;
&lt;p&gt;En attendant, c’est &lt;strong&gt;rafraîchissant&lt;&#x2F;strong&gt; d’avoir de nouveau du plaisir à utiliser un navigateur Web moderne, avec des fonctionnalités innovantes, tout en ne faisant aucun compromis sur ma vie privée. J’utilise maintenant &lt;em&gt;Zen Browser&lt;&#x2F;em&gt; depuis quelques semaines sur une machine, et &lt;strong&gt;je suis tout bonnement ravi&lt;&#x2F;strong&gt;, au point que je pense l’installer un peu partout rapidement, en complément&#x2F;remplacement de Firefox.&lt;&#x2F;p&gt;
&lt;p&gt;J’ai tendance à être de moins en moins prosélyte quant à l’utilisation de logiciels, libres ou non. Chacun·e fait bien comme iel veut, on ne part pas toustes du même endroit, on a bien le droit de prioriser d’autres sujets avant celui de l’utilisation de tel ou tel logiciels. Mais c’est une des premières fois depuis un moment que je remarque un logiciel libre si cool et innovant que j’adore l’utiliser, que j’ai envie d’en parler et de vous le partager ; voilà qui est chose faite 😊 Pour télécharger le navigateur &lt;em&gt;Zen Browser&lt;&#x2F;em&gt;, c’est &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;zen-browser.app&#x2F;download&#x2F;&quot;&gt;par ici&lt;&#x2F;a&gt;. Si comme moi vous êtes fans, n’hésitez-pas à soutenir le développement avec &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;zen-browser.app&#x2F;donate&#x2F;&quot;&gt;un don&lt;&#x2F;a&gt; !&lt;&#x2F;p&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn-1&quot;&gt;
&lt;p&gt;bac +5 les enfants, comme disait le cowboy &lt;a href=&quot;#fr-1-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-2&quot;&gt;
&lt;p&gt;j’emploie des guillemets ici, parce qu’il me semble que le terme a une signification bien précise aux USA, mais ce n’est pas celle que j’utilise ici. &lt;a href=&quot;#fr-2-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-3&quot;&gt;
&lt;p&gt;too soon 🥲 &lt;a href=&quot;#fr-3-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-5&quot;&gt;
&lt;p&gt;roh le cliché de libriste, c’est moche. &lt;a href=&quot;#fr-5-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-4&quot;&gt;
&lt;p&gt;il y a quelques semaines j’ai essayé de supprimer la barre d’URL en haut de Firefox pour la cacher, ou ne l’afficher qu’au survol ; sachez qu’il n’existe pas de solution officielle, et toutes les solutions officieuses (à base de &lt;em&gt;Greasemonkey&lt;&#x2F;em&gt; ou autres billevesées) sont fragiles comme un égo de mec. &lt;a href=&quot;#fr-4-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;&#x2F;section&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Publish a Zola blog with Gitlab CI, real fast</title>
        <published>2024-11-02T00:00:00+00:00</published>
        <updated>2024-11-02T00:00:00+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/publish-zola-with-gitlab-ci/"/>
        <id>https://bouvier.cc/tech/publish-zola-with-gitlab-ci/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/publish-zola-with-gitlab-ci/">&lt;p&gt;The two or three regular visitors of this blog (hi friends!) might have noticed I’ve changed the
design, as of this year. I’ve actually moved from using the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;getpelican.com&#x2F;&quot;&gt;Pelican&lt;&#x2F;a&gt;
static blog generator to using &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.getzola.org&#x2F;&quot;&gt;Zola&lt;&#x2F;a&gt;. This article shows the continuous
integration I’ve set up to automatically build and push content to my server, Real Fast™.&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;
&lt;p&gt;Was there anything wrong with &lt;code&gt;Pelican&lt;&#x2F;code&gt;? Not really. It has served me well over the years, and its
naming coming from an anagram of &lt;code&gt;calepin&lt;&#x2F;code&gt;, French for a “small notebook”, is still just genius.
Installing on a new machine was a bit of a pain as I needed to recall which Python &#x2F; virtualenv
&#x2F; etc. commands were required to install dependencies, but over the years I’ve made a Makefile to simplify that. And it
didn’t happen that often though; at most I’d run such a command once every two years or so (which
is, arguably, already a lot if that’s the frequency at which one switches machines).&lt;&#x2F;p&gt;
&lt;p&gt;I mostly moved to Zola because I’m a zealot &#x2F; member of the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;rustjerk&#x2F;comments&#x2F;av5pog&#x2F;higherres_rust_evangelism_strike_force_image&#x2F;&quot;&gt;Rust evangelism strike
force&lt;&#x2F;a&gt;
on my spare time, and want to support tools created with Rust, which are usually blazing 😎 fast 🚀
&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-2-1&quot;&gt;&lt;a href=&quot;#fn-2&quot;&gt;1&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;, in addition to being super safe. And, having a statically linked binary is super convenient,
and notably super nice for continuous integration and deployment purposes. Indeed, it’s trivial to
cache a single binary in CI, and not have to worry about installing&#x2F;caching dependencies and so on.
In addition to that, Zola supports front-matter annotations with TOML &lt;em&gt;or&lt;&#x2F;em&gt; YAML, so it offered a
nice migration path from Pelican, who uses YAML for frontmatters.&lt;&#x2F;p&gt;
&lt;p&gt;I’ve been using &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;about.gitlab.com&#x2F;&quot;&gt;Gitlab&lt;&#x2F;a&gt;, notably the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;framagit.org&#x2F;&quot;&gt;Framagit&lt;&#x2F;a&gt;
instance hosted by the good fellows at &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;soutenir.framasoft.org&quot;&gt;Framasoft&lt;&#x2F;a&gt; (give them
money to support them!), for the repository hosting the sources of my blog. As such, I’ve wanted to
be able to push to the repository, and have the CI build and publish to my website.&lt;&#x2F;p&gt;
&lt;p&gt;Now, let me explain how I’ve did it. These instructions are valid for &lt;em&gt;GitLab Community Edition
v17.5.1&lt;&#x2F;em&gt; ; some things may change in newer versions, so I can’t guarantee they’ll work forever.
There’s a first stage that will build the public website using a cached Zola binary, if possible,
or grab it from the Github releases website otherwise.
The second stage&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-1-1&quot;&gt;&lt;a href=&quot;#fn-1&quot;&gt;2&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; will upload it to the server, using &lt;code&gt;rsync&lt;&#x2F;code&gt;. I’d recommend creating a new user
just for this task, with limited SSH access to a single directory, that is, where the generated
HTML will live. With caching, each stage takes at most 10 seconds to run, which I find…
acceptable 😁.&lt;&#x2F;p&gt;
&lt;p&gt;Here’s the &lt;code&gt;.gitlab-ci.yml&lt;&#x2F;code&gt; file I’ve checked in, heavily commented for your (and my future self’s) best
understanding:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;yaml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;default&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  image&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; debian:stable-slim&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;variables&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # The runner will be able to pull your Zola theme when the strategy is&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # set to &amp;quot;recursive&amp;quot;.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  GIT_SUBMODULE_STRATEGY&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;recursive&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # If you don&amp;#39;t set a version here, your site will be built with the latest&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # version of Zola available in GitHub releases.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # Use the semver (x.y.z) format to specify a version. For example: &amp;quot;0.17.2&amp;quot; or &amp;quot;0.18.0&amp;quot;.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  ZOLA_VERSION&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;    description&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;The version of Zola used to build the site.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;    value&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;0.19.1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;build&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  stage&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; build&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # Cache the Zola binary based on its version, to avoid conflicts between different versions.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  cache&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;    key&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; $ZOLA_VERSION&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;    paths&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;     # $CI_PROJECT_DIR is the current working directory in subsequent steps.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;      -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;$CI_PROJECT_DIR&#x2F;zola&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  script&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;      if [ ! -e &amp;quot;$CI_PROJECT_DIR&#x2F;zola&amp;quot; ]; then&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        echo &amp;quot;Downloading Zola…&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        # Download enough to use `wget`.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        apt-get update --assume-yes &amp;amp;&amp;amp; apt-get install --assume-yes --no-install-recommends wget ca-certificates&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        if [ $ZOLA_VERSION ]; then&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;          zola_url=&amp;quot;https:&#x2F;&#x2F;github.com&#x2F;getzola&#x2F;zola&#x2F;releases&#x2F;download&#x2F;v$ZOLA_VERSION&#x2F;zola-v$ZOLA_VERSION-x86_64-unknown-linux-gnu.tar.gz&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;          if ! wget --quiet --spider $zola_url; then&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;            echo &amp;quot;A Zola release with the specified version could not be found.&amp;quot;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;            exit 1;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;          fi&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        else&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;          github_api_url=&amp;quot;https:&#x2F;&#x2F;api.github.com&#x2F;repos&#x2F;getzola&#x2F;zola&#x2F;releases&#x2F;latest&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;          zola_url=$(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;            wget --output-document - $github_api_url |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;            grep &amp;quot;browser_download_url.*linux-gnu.tar.gz&amp;quot; |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;            cut --delimiter : --fields 2,3 |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;            tr --delete &amp;quot;\&amp;quot; &amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;          )&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        fi&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        wget $zola_url&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        tar -xzf *.tar.gz&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;      else&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;        echo &amp;quot;Reusing cached Zola…&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;      fi&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; |&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;      $CI_PROJECT_DIR&#x2F;zola build&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # The built artifacts will be put in the `public&#x2F;` directory, and reused during the next stage.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  artifacts&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;    paths&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;      -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; public&#x2F;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;    expire_in&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; 1 day&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;deploy&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  stage&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; deploy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  only&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; main&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  dependencies&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; build&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #F7768E;&quot;&gt;  script&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # Install rsync and ssh, if needs be.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; apt-get update -qq &amp;amp;&amp;amp; apt-get install -y -qq rsync&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; &amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;which ssh-agent || ( apt-get update -y &amp;amp;&amp;amp; apt-get install openssh-client -y )&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; eval $(ssh-agent -s)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # Set SSH private key, and define the right permissions to not trigger security errors.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; echo &amp;quot;$SSH_PRIVATE_KEY&amp;quot; | tr -d &amp;#39;\r&amp;#39; | ssh-add - &amp;gt; &#x2F;dev&#x2F;null&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; mkdir -p ~&#x2F;.ssh&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; chmod 700 ~&#x2F;.ssh&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # Set SSH known hosts, and define the right permissions to not trigger security errors.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; echo &amp;quot;$SSH_KNOWN_HOSTS&amp;quot; &amp;gt; ~&#x2F;.ssh&#x2F;known_hosts&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; chmod 644 ~&#x2F;.ssh&#x2F;known_hosts&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # Run rsync.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;  # Note: -avzh = append &#x2F; verbose &#x2F; compress &#x2F; human-readable&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;  -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; rsync -avzh --delete public&#x2F;* -e &amp;quot;ssh -p $SSH_PORT&amp;quot; $SSH_USERNAME@$SSH_HOST:$SSH_TARGET_DIR&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;After you’ve set this up, you need to fill some secrets on your CI, by going to &lt;code&gt;Settings&lt;&#x2F;code&gt;,
then &lt;code&gt;CI&#x2F;CD&lt;&#x2F;code&gt;, then fill all the following variables under &lt;code&gt;Variables&lt;&#x2F;code&gt;. Make sure to create them
as &lt;code&gt;Masked&lt;&#x2F;code&gt; variables, if not &lt;code&gt;Masked and hidden&lt;&#x2F;code&gt;, unless you’d like the SSH private key to
leak 🤡.&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;SSH_HOST&lt;&#x2F;code&gt;: server’s host, e.g. &lt;code&gt;mysupersecretserver.bouvier.cc&lt;&#x2F;code&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;SSH_PORT&lt;&#x2F;code&gt;: the port to be used for connecting with SSH, e.g. 22 by default.&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;SSH_KNOWN_HOSTS&lt;&#x2F;code&gt;: a copy of a simplified &lt;code&gt;~&#x2F;.ssh&#x2F;known_hosts&lt;&#x2F;code&gt; file, stripped down to only
contain lines related to the above &lt;code&gt;SSH_HOST&lt;&#x2F;code&gt;. This is to avoid having the CI task to confirm “Are
you sure you want to trust this host” when connecting for the first time to the server.&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;SSH_TARGET_DIR&lt;&#x2F;code&gt;: the final directory where the generated HTML should live, e.g. &lt;code&gt;&#x2F;var&#x2F;www&#x2F;myblog&lt;&#x2F;code&gt;.&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;SSH_USERNAME&lt;&#x2F;code&gt;: the username of the (Unix) user used in the CI task to upload generated artifacts
to your Web server…&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;SSH_PRIVATE_KEY&lt;&#x2F;code&gt;: …and its private key.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;Hope this was useful!&lt;&#x2F;p&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn-2&quot;&gt;
&lt;p&gt;Don’t pay attention to the emojis, they’re a meme at this point. &lt;a href=&quot;#fr-2-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-1&quot;&gt;
&lt;p&gt;Using two stages is inherited from my previous setup using Pelican, and might be overkill
since the each stage takes at most 10 seconds to run, with a hot cache. &lt;a href=&quot;#fr-1-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;&#x2F;section&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Hetzner: cheap auction servers, and how to find them</title>
        <published>2024-07-08T00:00:00+00:00</published>
        <updated>2024-07-08T00:00:00+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/hetzner-auction-server-script/"/>
        <id>https://bouvier.cc/tech/hetzner-auction-server-script/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/hetzner-auction-server-script/">&lt;p&gt;Lately, I’ve had to find a hosting server to replace the server we rented for delire.party,
our self-hosting collective of friends. Indeed, OVH decided, with no extra explanations, to
decomission our Kimsufi server at the end of the year, and to have us pay a few euros more per
month until then. This kind of practices irritates me, so I’ve decided to pull out
the plug (metaphorically speaking — the OVH server will still be online for the time of the
migration) and have our next server live at Hetzner.&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;
&lt;p&gt;Hetzner is a German company, they’ve been relying on &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.hetzner.com&#x2F;unternehmen&#x2F;nachhaltigkeit&#x2F;&quot;&gt;renewable energy for a
while&lt;&#x2F;a&gt;, and they have this interesting system
of server auctions, with prices lowering until a server has been rented out. For a regular human
being, that would mean checking the website multiple times a day to find a cheap server with the
right specification. I, for one, am a nerd, so I’ve looked for ways to automate this.&lt;&#x2F;p&gt;
&lt;p&gt;There are a few scripts for this on the Internet, but I made a relatively short one that was
perfectly suited for my constraints. I wanted to only see servers with a price tag less than 50€
per month, and at least 2TB of NVME storage. I also wanted email notifications, so that I can stay
in the comfort of my email client to check out the new deals. The script doesn’t account for deals
it’s already seen, but using it for three days (running it once per day) was sufficient to find a
satisfying deal, so I didn’t even have to bother implementing that.&lt;&#x2F;p&gt;
&lt;p&gt;The script is written in JavaScript (based on the one I’ve taken inspiration from), and it has two
dependencies: &lt;code&gt;request&lt;&#x2F;code&gt; to run the HTTP requests, and &lt;code&gt;nodemailer&lt;&#x2F;code&gt; to send me the email.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; nodemailer&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; require&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;nodemailer&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; request&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; require&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;request&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; dataUrl&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; = &amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;https:&#x2F;&#x2F;www.hetzner.com&#x2F;_resources&#x2F;app&#x2F;jsondata&#x2F;live_data_sb.json&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    smtp_host&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;mail.example.com&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    smtp_port&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; null&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    smtp_username&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; null&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    smtp_password&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; null&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    smtp_accept_unauthorized&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; true&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    email_from&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;hetzner-deals@example.com&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    email_to&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;your-email@example.com&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;!&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;smtp_host&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    console&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Please specity SMTP host&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    process&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;exit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;!&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;email_to&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    console&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Please specify the email address to send notifications to&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    process&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;exit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; transporter&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; nodemailer&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;createTransport&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;({&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    host&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;smtp_host&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    port&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;smtp_port&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    auth&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;        user&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;smtp_username&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;        pass&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;smtp_password&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;    tls&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;        rejectUnauthorized&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; !&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;smtp_accept_unauthorized&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;})&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; jar&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; request&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;jar&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;()&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;request&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; request&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;defaults&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;({&lt;&#x2F;span&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt; jar&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; jar&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; })&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;request&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;get&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;dataUrl&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; resp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; body&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; =&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; ||&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; resp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;statusCode&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; !==&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 200&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        console&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Error: &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        console&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Response code: &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; resp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;statusCode&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        process&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;exit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;2&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; json&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; JSON&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;parse&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;body&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; []&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    for&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; of&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; json&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;server&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; Here, `s` is the data representing a single server.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; You can look at it and make up your own filtering&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; criteria based on its shape.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; console.log(s);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; This holds the total size of NVME storage.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;        let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; sumNvme&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;serverDiskData&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;nvme&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;reduce&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;((&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt;a&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; b&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt; =&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; a&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; b&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 0&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; Rough approximation of the monthly price, based on the hourly price.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;        let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; monthlyPrice&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;hourly_price&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; *&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 24&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; *&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 31&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; renting an IPv4 costs around 2.02&#x2F;month&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        monthlyPrice&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; +=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 2.02&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; round to second decimal&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        monthlyPrice&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; ((&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;100&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; *&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; monthlyPrice&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 0&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; &#x2F;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 100&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;sumNvme&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; &amp;gt;=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 2000&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; &amp;amp;&amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; monthlyPrice&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; &amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 50&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;push&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;- &lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;${&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;id}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;push&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;    RAM=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;${&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;ram_size}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;GB&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;push&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;    NVME=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;${&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;sumNvme&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;GB&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;push&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;    PRICE=~&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;${&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;monthlyPrice&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;€&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;push&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;    (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;${&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;s&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;information}&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;push&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;    var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; mailOptions&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;        from&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;email_from&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;        to&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; config&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;email_to&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;        subject&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Hetzner Server deal found&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;        text&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Found deals:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;\n\n&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; results&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;join&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;\n&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    transporter&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;sendMail&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;mailOptions&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; function&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; info&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            console&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Error sending email notification: &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            process&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;exit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;3&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; else&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            console&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;log&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Email notification sent: &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; info&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;response&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    })&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;})&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Hope this may be useful to some of you too!&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>cargo-machete, find unused dependencies quickly</title>
        <published>2022-04-27T23:17:23+00:00</published>
        <updated>2022-04-27T23:17:23+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/cargo-machete/"/>
        <id>https://bouvier.cc/tech/cargo-machete/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/cargo-machete/">&lt;p&gt;&lt;code&gt;cargo-machete&lt;&#x2F;code&gt; is a new Cargo tool that detects unused dependencies in Rust
projects, in a fast (yet imprecise) way. As of today you can install it with
&lt;code&gt;cargo install cargo-machete&lt;&#x2F;code&gt; and then run it with &lt;code&gt;cargo machete&lt;&#x2F;code&gt; from any
folder that contains a workspace or crate, to find if you have potentially
unused dependencies. Beware, it can report a few false positives!&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;&lt;h2 id=&quot;problem-statement&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#problem-statement&quot; aria-label=&quot;Anchor link for: problem-statement&quot;&gt;🔗&lt;&#x2F;a&gt;Problem statement&lt;&#x2F;h2&gt;
&lt;p&gt;When developers hack on code, it’s a pretty common to reuse software that
already exists and has been written, optimized, and battle-tested by many
others. In fact, that’s a core idea of the open-source movement, and one
historical reason for its existence.&lt;&#x2F;p&gt;
&lt;p&gt;When zooming in into the Rust programming language case, my opinion is that it
is also a key reason why Rust has been so successful: having plenty of crates
doing everything you might need, already implemented for you and at hand’s
reach on &lt;code&gt;crates.io&lt;&#x2F;code&gt;. Plus, having the wonder of a one-does-it-all Cargo tool
that makes it very easy to use those crates as dependencies in your project.
&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-1-1&quot;&gt;&lt;a href=&quot;#fn-1&quot;&gt;1&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;&lt;&#x2F;p&gt;
&lt;p&gt;However, this comes with a price: sometimes you add a dependency because it’s
useful at a particular point in time. Much later, it’s not useful anymore, but
you may have forgotten about it. And then, the dependency remains as a zombie
in your &lt;code&gt;Cargo.toml&lt;&#x2F;code&gt; file. Cargo will include it in the compilation graph,
despite the compilation artifacts not being used at all. The unused dependency
will just stay there, silently weep, waiting for you to recall it exists.&lt;&#x2F;p&gt;
&lt;p&gt;Of course, the problem can even become worse: maybe you maintain several crates
that have unused dependencies. Or maybe you work with many crates as part of a
workspace, and each may have unused dependencies. Or simply you use many
dependencies yourself, and some may include unused dependencies. If you’ve
published your crates and others use those, then everyone could also compile
unused dependencies. At the scale of the entire Rust crates ecosystem, it can
have a huge impact on the compile times, produced heat and wasted energy.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;have-you-heard-about-our-lord-and-savior-cargo-udeps&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#have-you-heard-about-our-lord-and-savior-cargo-udeps&quot; aria-label=&quot;Anchor link for: have-you-heard-about-our-lord-and-savior-cargo-udeps&quot;&gt;🔗&lt;&#x2F;a&gt;Have you heard about our lord and savior, &lt;code&gt;cargo-udeps&lt;&#x2F;code&gt;&lt;&#x2F;h2&gt;
&lt;p&gt;There’s already a nice tool for this in the ecosystem:
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;est31&#x2F;cargo-udeps&quot;&gt;&lt;code&gt;cargo-udeps&lt;&#x2F;code&gt;&lt;&#x2F;a&gt;. It will compile your
crate (or workspace) and then infer from the compiled artifacts what
dependencies are used by your project, and thus show you which dependencies are
&lt;em&gt;unused&lt;&#x2F;em&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;That’s great, but the way it works forces a few tradeoffs:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;it requires to compile the whole crate with the Rustc &lt;code&gt;nightly&lt;&#x2F;code&gt; compiler. For
me that means recompiling the whole project from scratch, most of the time,
since I’m mostly using stable rustc as my daily driver.&lt;&#x2F;li&gt;
&lt;li&gt;if you compile for multiple targets (i.e. different combinations of CPU
flavor, OS, environment, etc.), you’d need to run &lt;code&gt;cargo-udeps&lt;&#x2F;code&gt; on each of
those to find per-target unused dependencies. For instance, if a dependency
is only configured when compiling for x86_64 machines, then it may be flagged
as unused on every other configuration.&lt;&#x2F;li&gt;
&lt;li&gt;most of all, since it look at compilation artifacts, it cannot know if a
specific dependency is directly used by your crate, or indirectly, leading to
somehwat mystifying results in the case of workspaces.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;Let’s dive a bit deeper into the last item, which I’ll refer to as &lt;em&gt;the
transitively-used dependencies problem&lt;&#x2F;em&gt;. Say you have your project &lt;code&gt;AAA&lt;&#x2F;code&gt; that
contains a dependency to &lt;code&gt;serde&lt;&#x2F;code&gt; in its &lt;code&gt;Cargo.toml&lt;&#x2F;code&gt; file, while it’s not
directly used by your code. In fact, if you did a text-search of &lt;code&gt;serde&lt;&#x2F;code&gt; in
&lt;code&gt;AAA&lt;&#x2F;code&gt;’s code with &lt;code&gt;grep&lt;&#x2F;code&gt;, you wouldn’t find a single match&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-2-1&quot;&gt;&lt;a href=&quot;#fn-2&quot;&gt;2&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;. But now &lt;code&gt;AAA&lt;&#x2F;code&gt;
is using another crate, &lt;code&gt;AB&lt;&#x2F;code&gt;, that itself depends on &lt;code&gt;serde&lt;&#x2F;code&gt;. &lt;code&gt;cargo-udeps&lt;&#x2F;code&gt;
will see that &lt;code&gt;serde&lt;&#x2F;code&gt; is used &lt;em&gt;overall&lt;&#x2F;em&gt;, so it cannot let you know that &lt;code&gt;AAA&lt;&#x2F;code&gt;’s
&lt;code&gt;Cargo.toml&lt;&#x2F;code&gt; file references an unused dependency to &lt;code&gt;serde&lt;&#x2F;code&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;cargo-machete&#x2F;unused.png&quot; alt=&quot;Graph of crates containing one unused crate&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;How is this a problem? After all, if the workspace uses &lt;code&gt;serde&lt;&#x2F;code&gt; even
indirectly, then we &lt;em&gt;will&lt;&#x2F;em&gt; have to compile it at some point, so it’s not like
it’s &lt;em&gt;really&lt;&#x2F;em&gt; unused.&lt;&#x2F;p&gt;
&lt;p&gt;First of all, the &lt;code&gt;AAA&lt;&#x2F;code&gt; crate might be using a different version of &lt;code&gt;serde&lt;&#x2F;code&gt;
than the &lt;code&gt;AB&lt;&#x2F;code&gt; crate, and this could result in different copies of the same
crate in your workspace. Note there are other nice tools that automatically
detect this kind of situation (hi there
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;EmbarkStudios&#x2F;cargo-deny&#x2F;&quot;&gt;cargo-deny&lt;&#x2F;a&gt;).&lt;&#x2F;p&gt;
&lt;p&gt;Second, the order in which crates are compiled has an impact on compilation
parallelism, and having unused dependencies may add spurious synchronization
points in the compilation graph. When a Rust crate gets compiled by Cargo,
Cargo proceeds in two phases:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;first, it collects information so as to unlock the compilation of other
crates further down the road that may depend on this particular one. I don’t
know precisely what it entails, but one can make educated guesses: parse the
code, analyze which items are &lt;code&gt;pub&lt;&#x2F;code&gt;lic, compute memory layouts for public
types, collect type information and so on and so forth.&lt;&#x2F;li&gt;
&lt;li&gt;then, it does the actual compilation: optimize and generate the actual
machine code for that particular crate, that will be later linked with other
artifacts to form the final executable program.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;The advantage of this two-phases scheme is that once Cargo is done with phase 1
for a particular crate, it can kick off the same process for other crates up
the dependency tree, while it runs phase 2 concurrently. With a multi-core
machine as is the norm on desktop computers, it’s almost certain that this will
bring speedups!&lt;&#x2F;p&gt;
&lt;p&gt;For instance, consider the following &lt;code&gt;Cargo.toml&lt;&#x2F;code&gt; file from our previous
example project:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;toml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;dependencies&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;serde&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; = &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;1.0&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Then a possible compilation graph could look like that:&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;cargo-machete&#x2F;phases.png&quot; alt=&quot;Compilation graph showing phases&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;In this case, &lt;code&gt;ab&lt;&#x2F;code&gt; phase 1 can start as soon as &lt;code&gt;serde&lt;&#x2F;code&gt; phase 1 has finished,
while &lt;code&gt;serde&lt;&#x2F;code&gt;’s compilation phase 2 happens in the background.&lt;&#x2F;p&gt;
&lt;p&gt;If you’re interested in reducing the overall compile times of your Rust
project, I would strongly suggest to &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;doc.rust-lang.org&#x2F;nightly&#x2F;cargo&#x2F;reference&#x2F;unstable.html#timings&quot;&gt;go read Rust’s documentation around
timings
visualization&lt;&#x2F;a&gt;.
Crates which spend lots of time in the first phase (or more generally, in both
phases) are basically pipelining bottlenecks, so identifying&#x2F;removing&#x2F;working
around them overall speeds up compile times.&lt;&#x2F;p&gt;
&lt;p&gt;Back to our small unused dependency problem: an unused dependency in your
&lt;code&gt;Cargo.toml&lt;&#x2F;code&gt; may block the compilation of other crates up the dependency tree,
and thus may slow down the whole compilation process by creating useless check
points.&lt;&#x2F;p&gt;
&lt;p&gt;Consider a crate &lt;code&gt;C&lt;&#x2F;code&gt; that depends on crates &lt;code&gt;A&lt;&#x2F;code&gt; and &lt;code&gt;B&lt;&#x2F;code&gt;, with &lt;code&gt;B&lt;&#x2F;code&gt; actually
unused:&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;cargo-machete&#x2F;pipeline-stall.png&quot; alt=&quot;Pipeline stall&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;Here, the compilation of the crate &lt;code&gt;C&lt;&#x2F;code&gt; could start way earlier, but it’s
blocking waiting for the compilation of &lt;code&gt;B&lt;&#x2F;code&gt; to finish first, while it’s not
even used!&lt;&#x2F;p&gt;
&lt;h2 id=&quot;solving-this-the-naive-way&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#solving-this-the-naive-way&quot; aria-label=&quot;Anchor link for: solving-this-the-naive-way&quot;&gt;🔗&lt;&#x2F;a&gt;Solving this, the naive way&lt;&#x2F;h2&gt;
&lt;p&gt;So when I was trying to confirm whether crates found by &lt;code&gt;cargo-udeps&lt;&#x2F;code&gt; were
actually used or not in my Rust projects, the thing I’d do would be to &lt;code&gt;grep&lt;&#x2F;code&gt;
(or better, use the blazingly fast Rust replacement
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;BurntSushi&#x2F;ripgrep&quot;&gt;ripgrep&lt;&#x2F;a&gt;) the crate’s name in the
project. After all, the crate’s name is in the source directory, if and only if
the crate is used, right?&lt;&#x2F;p&gt;
&lt;p&gt;The answer is… mostly, yes. If we exclude dynamic code loading via mechanisms
like &lt;code&gt;dlopen&lt;&#x2F;code&gt; or WebAssembly, then there aren’t so many ways to use other
crates &lt;em&gt;directly&lt;&#x2F;em&gt;, in Rust code. In fact, we can exhaustively enumerate all the
syntax items to use other dependencies in Rust:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;use&lt;&#x2F;span&gt;&lt;span&gt; my_crate&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;use&lt;&#x2F;span&gt;&lt;span&gt; your_crate &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;as&lt;&#x2F;span&gt;&lt;span&gt; my_crate&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;use {&lt;&#x2F;span&gt;&lt;span&gt; your_crate &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;as&lt;&#x2F;span&gt;&lt;span&gt; my_crate &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;};&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;extern&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt; crate&lt;&#x2F;span&gt;&lt;span&gt; my_crate&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; main&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;() {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;    my_crate&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;something&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;();&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;I’ve looked at a bit of Rust code now, and I haven’t seen other direct forms;
if I am missing any, please let me know! Now, these are the most &lt;em&gt;frequent&lt;&#x2F;em&gt;
ways to use a dependency, but there are in fact other ways:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;build.rs&lt;&#x2F;code&gt; scripts can generate code that could use other crates, and that
would not be visible through a text search in the &lt;code&gt;src&#x2F;&lt;&#x2F;code&gt; directory, as the
generated code is somewhere inside the &lt;code&gt;target&#x2F;build&#x2F;&lt;&#x2F;code&gt; directory.&lt;&#x2F;li&gt;
&lt;li&gt;macros (procedural or not) can expand to code that’s using other crates,
while the source code doesn’t &lt;em&gt;explicitly&lt;&#x2F;em&gt; mention them. For instance, the
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;Luthaf&#x2F;log-once&quot;&gt;&lt;code&gt;log_once&lt;&#x2F;code&gt;&lt;&#x2F;a&gt; crate uses the &lt;code&gt;log&lt;&#x2F;code&gt; macros
in its own macros, but &lt;code&gt;log_once&lt;&#x2F;code&gt; doesn’t depend on &lt;code&gt;log&lt;&#x2F;code&gt; explicitly. It’s a
bold and smart move: it breaks the coupling with the specific version of
&lt;code&gt;log&lt;&#x2F;code&gt; , and as long as the high-level API of &lt;code&gt;log&lt;&#x2F;code&gt; is stable (which is the
case), then &lt;code&gt;log_once&lt;&#x2F;code&gt; works with &lt;em&gt;any&lt;&#x2F;em&gt; version of &lt;code&gt;log&lt;&#x2F;code&gt;.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;And then, there’s still a bit of room for some false positives:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;raw text submatches: e.g. if a crate is named &lt;code&gt;bar&lt;&#x2F;code&gt;, then &lt;code&gt;foobar::&lt;&#x2F;code&gt; would be
a match if we’re doing a raw &lt;code&gt;grep&lt;&#x2F;code&gt; search&lt;&#x2F;li&gt;
&lt;li&gt;text search isn’t syntaxic analysis, and we wouldn’t know if a match is in a
comment (&lt;code&gt;&#x2F;&#x2F; use foo;&lt;&#x2F;code&gt;), or a string (&lt;code&gt;String::from(&quot;use foo;&quot;)&lt;&#x2F;code&gt;).&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;But that would do &lt;em&gt;most of the job&lt;&#x2F;em&gt;, wouldn’t it? In particular, compared to
&lt;code&gt;cargo-udeps&lt;&#x2F;code&gt;, this approach doesn’t suffer from the &lt;em&gt;transitively-used
dependencies&lt;&#x2F;em&gt; problem. If you look for a crate’s name in the &lt;code&gt;src&#x2F;&lt;&#x2F;code&gt; directory
and it’s not there, it’s likely not used by your crate. The End.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;a-tedious-process-calls-for-automation-so-i-made-a-tool&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#a-tedious-process-calls-for-automation-so-i-made-a-tool&quot; aria-label=&quot;Anchor link for: a-tedious-process-calls-for-automation-so-i-made-a-tool&quot;&gt;🔗&lt;&#x2F;a&gt;A tedious process calls for automation, so I made a tool&lt;&#x2F;h2&gt;
&lt;p&gt;And I’ve called it &lt;code&gt;cargo-machete&lt;&#x2F;code&gt;. Like a machete, it is very useful for
quickly weeding out things, but it is very imprecise and you wouldn’t trust it
at 100%.&lt;&#x2F;p&gt;
&lt;p&gt;The gist of it is:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;find directories that might contain Rust projects, as indicated by the
presence of a &lt;code&gt;Cargo.toml&lt;&#x2F;code&gt; file&lt;&#x2F;li&gt;
&lt;li&gt;for each dependency, create an absolutely ugly regular expression that
matches any of the syntaxic forms presented above. The regular expression
does better than just raw text search, in particular it doesn’t run into the
text submatch issue.
&lt;ul&gt;
&lt;li&gt;then for each file in the project, try to match the regular expression
against each line of any source file, and stop at the first successful
match (which means the dependency &lt;em&gt;is&lt;&#x2F;em&gt; used)&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;This tool is &lt;em&gt;fast&lt;&#x2F;em&gt;, because it combines the core library behind &lt;code&gt;ripgrep&lt;&#x2F;code&gt; for
matching regular expressions, with &lt;em&gt;rayon&lt;&#x2F;em&gt; for running it in parallel across
all the dependencies of a project. On my machine, the problem is CPU-bound,
because of the execution of the regular expression (and maybe thanks to my NVME
storage too). That’s only one data point, but on this particular beefy desktop
I use, it scans the entirety of the &lt;code&gt;rust-lang&#x2F;rust&lt;&#x2F;code&gt; repository in 1.08
seconds, or all of &lt;code&gt;BytecodeAlliance&#x2F;wasmtime&lt;&#x2F;code&gt; in 0.58 seconds.&lt;&#x2F;p&gt;
&lt;p&gt;The tool is &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;cargo-machete&quot;&gt;&lt;em&gt;open source&lt;&#x2F;em&gt;&lt;&#x2F;a&gt;, of course.&lt;&#x2F;p&gt;
&lt;p&gt;As is the tradition for Cargo tools, it can be installed with:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cargo install cargo&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;-&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;machete&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;and then can be used, from any directory that contains Rust code (be it a
workspace, a single project, or a directory on top of many Rust projects), with
the following line:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cargo machete&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Here’s an output example:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;span&gt; cargo machete&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Looking&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; for crates in this directory and analyzing their dependencies...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;&#x2F;home&#x2F;ben&#x2F;code&#x2F;cargo-machete&#x2F;integration-tests&#x2F;with-bench&#x2F;Cargo.toml&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; --&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; no package, must be a workspace&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;just-unused&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; --&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; &#x2F;home&#x2F;ben&#x2F;code&#x2F;cargo-machete&#x2F;integration-tests&#x2F;just-unused&#x2F;Cargo.toml:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;	log&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;unused-transitive&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; --&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; &#x2F;home&#x2F;ben&#x2F;code&#x2F;cargo-machete&#x2F;integration-tests&#x2F;unused-transitive&#x2F;Cargo.toml:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;	lib1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Done!&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;There are &lt;em&gt;false positives&lt;&#x2F;em&gt;: code generated via macros or build scripts aren’t
inspected as they’re not in the &lt;code&gt;src&#x2F;&lt;&#x2F;code&gt; directory and &lt;code&gt;cargo-machete&lt;&#x2F;code&gt; doesn’t
run any compile step. For instance, if a project depends on &lt;code&gt;log&lt;&#x2F;code&gt; , but uses it
only through &lt;code&gt;log_once&lt;&#x2F;code&gt;, then &lt;code&gt;cargo-machete&lt;&#x2F;code&gt; will incorrectly flag &lt;code&gt;log&lt;&#x2F;code&gt; as an
unused dependency.&lt;&#x2F;p&gt;
&lt;p&gt;The good news is that, thanks to a contribution from &lt;code&gt;@daniel5151&lt;&#x2F;code&gt;, you can
specify &lt;em&gt;known false positives&lt;&#x2F;em&gt; in the &lt;code&gt;Cargo.toml&lt;&#x2F;code&gt; file of your crate,
allowing use of &lt;code&gt;cargo-machete&lt;&#x2F;code&gt; in CI setups:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;toml&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;package&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;metadata&lt;&#x2F;span&gt;&lt;span&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cargo-machete&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;]&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ignored&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; [&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;log&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt; # false positive, used by log_once! macro&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;As far as I know, the risk for &lt;em&gt;false negatives&lt;&#x2F;em&gt; (i.e. crates that are unused,
but the tool thinks they’re used) is pretty low. One such instance would be a
multi-line string containing one of the &lt;code&gt;use&lt;&#x2F;code&gt; forms, but that seems rather
unlikely to be present in most Rust projects.&lt;&#x2F;p&gt;
&lt;p&gt;The tool is still a bit rough, but it’s been already quite useful for some
projects I’ve been working on! In a particular work project, most unused
dependencies were transitively used and compiled, but the rejiggering of the
compilation graph lead to a 5% compile time speedup overall. Good impact over
effort ratio.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;what-about-other-languages&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#what-about-other-languages&quot; aria-label=&quot;Anchor link for: what-about-other-languages&quot;&gt;🔗&lt;&#x2F;a&gt;What about other languages?&lt;&#x2F;h2&gt;
&lt;p&gt;What makes this possible in Rust, and could it be extended to other languages?&lt;&#x2F;p&gt;
&lt;p&gt;Dynamic languages by nature dynamically load code, but there are still ways to
try to automate detecting unused dependencies same as &lt;code&gt;cargo-machete&lt;&#x2F;code&gt; does.
Consider JavaScript and its &lt;code&gt;require&lt;&#x2F;code&gt; function, that can dynamically evaluate a
string that’s a path to a file with code we want to import. Since there’s an
infinity of ways to create a string, we can’t just perfectly rely on finding
&lt;code&gt;require(&quot;abc&quot;)&lt;&#x2F;code&gt; and assume that if not present, then &lt;code&gt;abc&lt;&#x2F;code&gt; isn’t used. Ditto
with &lt;code&gt;import&lt;&#x2F;code&gt; statements, which can evaluate dynamic sources. That being said,
if JS code is restricted to use &lt;code&gt;require&lt;&#x2F;code&gt; statements with only static strings
or static &lt;code&gt;import&lt;&#x2F;code&gt; statements, then this may work too! Although when
restricting to static &lt;code&gt;require&lt;&#x2F;code&gt;s, even just &lt;em&gt;loading&lt;&#x2F;em&gt; the code in NodeJS would
be sufficient to find unused dependencies with perfect accuraccy.&lt;&#x2F;p&gt;
&lt;p&gt;Back to static languages, where I constrain the problem to non-dynamic
dependencies (loaded via &lt;code&gt;dlopen&lt;&#x2F;code&gt; etc.). In a language like C or C++, there are
no unified module systems or package description (yet! although &lt;code&gt;cmake&lt;&#x2F;code&gt; might
be a de-facto standard). We can still apply this to header files, and look for
their inclusion via &lt;code&gt;#include&lt;&#x2F;code&gt; statements. Macros and preprocessed code would
also throw a wrench in the process. Then some human intervention would still be
required to eliminate the .c files, but I haven’t thought about it too much.&lt;&#x2F;p&gt;
&lt;p&gt;Static analysis of compiled binaries might be simpler, for that matter. If we
consider the problem for WebAssembly, we can frame it as “which imported
functions are not used in the module”, potentially eliminating an entire range
of host functions. In the simplest case, we could just look at the code
section, through the function bodies, and see if there’s any reference to
indices of every single imported function in &lt;code&gt;call&lt;&#x2F;code&gt; opcodes. Then, there can be
function &lt;code&gt;Table&lt;&#x2F;code&gt;s referencing those, so we have to make sure no table elements
reference the function. And if any table is mutable and publicly exposed via an
export, then a user of the wasm module may reference any function declared in
the wasm module, including imported functions, so all bets are off. Note
dead-code elimination in wasm would be pretty similar and suffer from the same
limitations: after all, a function dependency is just another kind of function,
in wasm! Each format may have such idiosyncrasies like that. Static analysis of
final binaries (as opposed to libraries) might be possible and reliable,
though.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;closing-thoughts&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#closing-thoughts&quot; aria-label=&quot;Anchor link for: closing-thoughts&quot;&gt;🔗&lt;&#x2F;a&gt;Closing thoughts&lt;&#x2F;h2&gt;
&lt;p&gt;For the sake of completeness, I should mention the existence of a rustc
crate-wide lint for this, &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;rust-lang&#x2F;rust&#x2F;pull&#x2F;72342&quot;&gt;since May 2020 or
so&lt;&#x2F;a&gt;:
&lt;code&gt;#![warn(unused_crate_dependencies)]&lt;&#x2F;code&gt;. This tells about unused crate
dependencies directly as a Rust warning, which in my opinion would be the ideal
end goal! Unfortunately, some Github comments suggest it suffers from having
too many false positives, and still it requires compiling the code.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;code&gt;@est31&lt;&#x2F;code&gt;, of &lt;code&gt;cargo-udeps&lt;&#x2F;code&gt;’s fame, has been working on a &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;rust-lang&#x2F;cargo&#x2F;pull&#x2F;8437&quot;&gt;better
solution&lt;&#x2F;a&gt;. It seems to not be so
far from completion, so between this and the Rust lint, I’m hopeful that there
could be a time where we have a solution that is perfectly precise, with
neither false positives nor false negatives.&lt;&#x2F;p&gt;
&lt;p&gt;In the meanwhile, I hope that &lt;code&gt;cargo-machete&lt;&#x2F;code&gt; can be useful to some of you, or
that it inspires others to make similar quick-and-dirty tools, in Rust or in
other languages. Thanks for reading this far, and please &lt;a href=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;cargo-machete&#x2F;@mailto:benjamin+cargomachete@bouvier.cc&quot;&gt;get in
touch&lt;&#x2F;a&gt; if you have any thoughts about this!&lt;&#x2F;p&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn-1&quot;&gt;
&lt;p&gt;If you don’t know about the &lt;code&gt;cargo-edit&lt;&#x2F;code&gt; tool that allows you to add a
dependency in one line with &lt;code&gt;cargo add serde&lt;&#x2F;code&gt; to your project: &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;killercup&#x2F;cargo-edit&quot;&gt;now you
do&lt;&#x2F;a&gt;. &lt;a href=&quot;#fr-1-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-2&quot;&gt;
&lt;p&gt;Wait for it… &lt;a href=&quot;#fr-2-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;&#x2F;section&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>A primer on code generation in Cranelift</title>
        <published>2021-02-17T19:00:42+00:00</published>
        <updated>2021-02-17T19:00:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/cranelift-codegen-primer/"/>
        <id>https://bouvier.cc/tech/cranelift-codegen-primer/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/cranelift-codegen-primer/">&lt;script src=&quot;https:&#x2F;&#x2F;cdn.jsdelivr.net&#x2F;npm&#x2F;mermaid&#x2F;dist&#x2F;mermaid.min.js&quot;&gt;&lt;&#x2F;script&gt;
&lt;script&gt;mermaid.initialize({startOnLoad:true});&lt;&#x2F;script&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bytecodealliance&#x2F;wasmtime&#x2F;tree&#x2F;main&#x2F;cranelift#cranelift-code-generator&quot;&gt;Cranelift&lt;&#x2F;a&gt; is a code generator written in the Rust programming language that aims to be a fast code generator, which outputs machine code that runs at reasonable speeds.&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;
&lt;p&gt;The Cranelift compilation model consists in compiling functions one by one, holding extra information about external entities, like external functions, memory addresses, and so on. This model allows for concurrent and parallel compilation of individual functions, which supports the goal of fast compilation. It was designed this way to allow for just-in-time (JIT) compilation of WebAssembly binary code in Firefox, although its scope has broadened a bit. Nowadays it is used in a few different WebAssembly runtimes, including &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bytecodealliance&#x2F;wasmtime#wasmtime&quot;&gt;Wasmtime&lt;&#x2F;a&gt; and &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;wasmer.io&#x2F;&quot;&gt;Wasmer&lt;&#x2F;a&gt;, but also as an alternative backend for Rust debug compilation, thanks to &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bjorn3&#x2F;rustc_codegen_cranelift&quot;&gt;cg_clif&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;A classic compiler design usually includes running a parser to translate the source to some form of intermediate representations, then run optimization passes onto them, then feeds this to the machine code generator.&lt;&#x2F;p&gt;
&lt;p&gt;This blog post focuses on the final step, namely the concepts that are involved in code generation, and what they map to in Cranelift. To make things more concrete, we’ll take a specific instruction, and see how it’s translated, from its creation down to code generation. At each step of the process, I’ll provide a short (&lt;em&gt;ahem&lt;&#x2F;em&gt;) high-level explanation of the concepts involved, and I’ll show what they map to in Cranelift, using the example instruction. While this is not a tutorial detailing how to add new instructions in Cranelift, this should be an interesting read for anyone who’s interested in compilers, and this could be an entry point if you’re interested in hacking on the Cranelift &lt;code&gt;codegen&lt;&#x2F;code&gt; crate.&lt;&#x2F;p&gt;
&lt;p&gt;This is our plan for this blog post: each squared box represents data, each
rounded box is a process. We’re going to go through each of them below.&lt;&#x2F;p&gt;
&lt;div class=&quot;mermaid&quot;&gt;
graph TD;
    clif[Optimized CLIF];
    vcode[VCode];
    final_vcode[Final VCode];
    machine_code[Machine code artifacts];
    lowering([Lowering]);
    regalloc([Register allocation]);
    codegen([Machine code generation]);
    clif --&gt; lowering --&gt; vcode --&gt; regalloc --&gt; final_vcode --&gt; codegen --&gt; machine_code
&lt;&#x2F;div&gt;
&lt;h2 id=&quot;intermediate-representations&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#intermediate-representations&quot; aria-label=&quot;Anchor link for: intermediate-representations&quot;&gt;🔗&lt;&#x2F;a&gt;Intermediate representations&lt;&#x2F;h2&gt;
&lt;p&gt;Compilers use &lt;strong&gt;intermediate representations&lt;&#x2F;strong&gt; (&lt;em&gt;IR&lt;&#x2F;em&gt;) to represent source code. Here we’re interested in representations of the &lt;em&gt;data flow&lt;&#x2F;em&gt;, that is instructions themselves and only that. The IRs contain information about the instructions themselves, their operands, type specialization information, and any additional metadata that might be useful. IRs usually map to a certain level of abstraction, and as such, they are useful for solving different problems that require different levels of abstraction. Their shape (which data structures) and numbers often have a huge impact on the performance of the compiler itself (that is, how fast it is at compiling).&lt;&#x2F;p&gt;
&lt;p&gt;In general, most programming languages use IRs internally, and yet, these are invisible to the programmers. The reason is that source code is usually first &lt;em&gt;parsed&lt;&#x2F;em&gt; (tokenized, verified) and then translated into an IR. The &lt;em&gt;abstract syntax tree&lt;&#x2F;em&gt;, aka AST, is one such IR representing the source code itself, in a format that’s very close to the source code itself. Since the raison d’être of Cranelift is to be a code generator, having a text format is secondary, and only useful for testing and debugging purposes. That’s why embedders directly create and manipulate Cranelift’s IR.&lt;&#x2F;p&gt;
&lt;p&gt;At the time of writing, Cranelift has two IRs to represent the function’s code:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;one external, high-level intermediate representation, called &lt;strong&gt;CLIF&lt;&#x2F;strong&gt; (for &lt;em&gt;Cranelift IR format&lt;&#x2F;em&gt;),&lt;&#x2F;li&gt;
&lt;li&gt;one internal, low-level intermediate representation called &lt;strong&gt;VCode&lt;&#x2F;strong&gt; (for &lt;em&gt;virtual-registerized code&lt;&#x2F;em&gt;).&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h2 id=&quot;clif-ir&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#clif-ir&quot; aria-label=&quot;Anchor link for: clif-ir&quot;&gt;🔗&lt;&#x2F;a&gt;CLIF IR&lt;&#x2F;h2&gt;
&lt;p&gt;CLIF is the IR that Cranelift embedders create and manipulate. It consists of high-level typed operations that are convenient to use and&#x2F;or can be simply translated to machine code. It is in &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Static_single_assignment_form&quot;&gt;static single assignment (SSA) form&lt;&#x2F;a&gt;: each value referenced by an operation (SSA value) is defined only once, and may have as many uses as desired. CLIF is practical to use and manipulate for classic compilers optimization passes (e.g. &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Loop-invariant_code_motion&quot;&gt;LICM&lt;&#x2F;a&gt;), as it is generic over the target architecture which we’re compiling to.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; builder&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;ins&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;().&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;iconst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;types&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;I64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 42&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; builder&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;ins&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;().&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;iconst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;types&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;I64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 1337&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; sum&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; builder&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;ins&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;().&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;iadd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;An example of Rust code that would generate CLIF IR: using an IR builder, two constant 64-bits integer SSA values x and y are created, and then added together. The result is stored into the &lt;code&gt;sum&lt;&#x2F;code&gt; SSA value, which can then be consumed by other instructions.&lt;&#x2F;p&gt;
&lt;p&gt;The code for the IR builder we’re manipulating above is automatically generated by the &lt;code&gt;cranelift-codegen&lt;&#x2F;code&gt; build script. The build script uses a domain specific &lt;em&gt;meta&lt;&#x2F;em&gt; language (DSL)&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-2-1&quot;&gt;&lt;a href=&quot;#fn-2&quot;&gt;1&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; that defines the instructions, their input and output operands, which input types are allowed, how the output type is inferred, etc. We won’t take a look at this &lt;em&gt;today&lt;&#x2F;em&gt;: this is a bit too far from code generation, but this could be material for another blog post.&lt;&#x2F;p&gt;
&lt;p&gt;As an example of a full-blown CLIF generator, there is &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bytecodealliance&#x2F;wasmtime&#x2F;tree&#x2F;main&#x2F;cranelift&#x2F;wasm&quot;&gt;a crate&lt;&#x2F;a&gt; in the Cranelift project that allows translating from the WebAssembly binary format to CLIF. The Cranelift backend for Rustc uses its own CLIF generator that translates from one of the Rust compiler’s IRs.&lt;&#x2F;p&gt;
&lt;p&gt;Finally, it’s time to reveal what’s going to be our running example! The Chosen One is the &lt;code&gt;iadd&lt;&#x2F;code&gt; CLIF operation, which allows to add two integers of any length together, with wrapping semantics. It is both simple to understand what it does, and exhibits interesting behaviors on the two architectures we’re interested in. So, let’s continue down the pipeline!&lt;&#x2F;p&gt;
&lt;h2 id=&quot;vcode-ir&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#vcode-ir&quot; aria-label=&quot;Anchor link for: vcode-ir&quot;&gt;🔗&lt;&#x2F;a&gt;VCode IR&lt;&#x2F;h2&gt;
&lt;p&gt;Later on, the CLIF intermediate representation is &lt;em&gt;lowered&lt;&#x2F;em&gt;, i.e. transformed from a high-level one into a lower-level one. Here lower level means a form more specialized for a machine architecture. This lower IR is called &lt;em&gt;VCode&lt;&#x2F;em&gt; in Cranelift. The values it references are called &lt;em&gt;virtual registers&lt;&#x2F;em&gt; (more on the &lt;em&gt;virtual&lt;&#x2F;em&gt; bit below). They’re not in SSA form anymore: each virtual register may be redefined as many times as we want. This IR is used to encode register allocation constraints and it guides machine code generation. As a matter of fact, since this information is tied to the machine code’s representation itself, this IR is also target-specific: there’s one flavor of VCode per each CPU architecture we’re compiling to.&lt;&#x2F;p&gt;
&lt;p&gt;Let’s get back to our example, that we’re going to compile on two instruction set architectures:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;ARM 64-bits (aka aarch64), which is used in most mobile devices but start to become mainstream on laptops (Apple’s Mac M1, some Chromebooks)&lt;&#x2F;li&gt;
&lt;li&gt;Intel’s x86 64-bits (aka x86_64, also abbreviated x64), which is used in most desktop and laptop machines).&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;An integer addition machine instruction on aarch64 will take three operands: two input operands (one of which must be a register), and another third output register operand. While on the x86_64 architecture, the equivalent instruction involves a total of two registers: one that is a read-only source register, and another that is an in-out modified register, containing both the second source and the destination register. We’ll get back to this.&lt;&#x2F;p&gt;
&lt;p&gt;So considering &lt;code&gt;iadd&lt;&#x2F;code&gt;, let’s look at (one of&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-4-1&quot;&gt;&lt;a href=&quot;#fn-4&quot;&gt;2&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;) the VCode instruction that’s used to represent integer additions on aarch64 (as defined in &lt;code&gt;cranelift&#x2F;codegen&#x2F;src&#x2F;isa&#x2F;aarch64&#x2F;inst&#x2F;mod.rs&lt;&#x2F;code&gt;):&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F;&#x2F; An ALU operation with two register sources and a register destination.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;AluRRR&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    alu_op&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; ALUOp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Writable&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;},&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Some details here:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;alu_op&lt;&#x2F;code&gt; defines the sub-opcode used in the ALU (Arithmetic Logic Unit). It will be &lt;code&gt;AluOp::Add64&lt;&#x2F;code&gt; for a 64-bits integer addition.&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;rn&lt;&#x2F;code&gt; and &lt;code&gt;rm&lt;&#x2F;code&gt; are the conventional aarch64 names for the two input registers.&lt;&#x2F;li&gt;
&lt;li&gt;&lt;code&gt;rd&lt;&#x2F;code&gt; is the destination register. See how it’s marked as &lt;code&gt;Writable&lt;&#x2F;code&gt;, while the two others are not? &lt;code&gt;Writable&lt;&#x2F;code&gt; is a plain Rust wrapper that makes sure that we &lt;em&gt;can&lt;&#x2F;em&gt; statically differentiate read-only registers from writable registers; a neat trick that allows us to catch more issues at compile-time.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;All this information is directly tied to the machine code representation of an addition instruction on aarch64: each field is later used to select some bytes that will be generated during code generation.&lt;&#x2F;p&gt;
&lt;p&gt;As said before, the VCode is specific to each architecture, so x86_64 has a different VCode representation for the same instruction (as defined in &lt;code&gt;cranelift&#x2F;codegen&#x2F;src&#x2F;isa&#x2F;x64&#x2F;inst&#x2F;mod.rs&lt;&#x2F;code&gt;):&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F;&#x2F; Integer arithmetic&#x2F;bit-twiddling: (add sub and or xor mul adc? sbb?) (32 64) (reg addr imm) reg&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;AluRmiR&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    is_64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; bool&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    op&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; AluRmiROpcode&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    src&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; RegMemImm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Writable&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;},&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Here, the sub-opcode is defined as part of the &lt;code&gt;AluRmiROpcode&lt;&#x2F;code&gt; enum (the comment hints at which other x86 machine instructions are generated by this same VCode). See how there’s only one &lt;code&gt;src&lt;&#x2F;code&gt; (source) register (or memory or immediate operand), while the instruction conceptually takes two inputs? That’s because it’s expected that the &lt;code&gt;dst&lt;&#x2F;code&gt; (destination) register is &lt;em&gt;modified&lt;&#x2F;em&gt;, that is, both read (so it’s the second input operand) and written to (so it’s the result register). In equivalent C code, the x86’s add instruction doesn’t actually do &lt;code&gt;a = b + c&lt;&#x2F;code&gt;. What it does is &lt;code&gt;a += b&lt;&#x2F;code&gt;, that is, one of the sources is &lt;em&gt;consumed&lt;&#x2F;em&gt; by the instruction. This is an artifact inherited from the design of older x86 machines in the 1970’s, when instructions were designed around an accumulator model (and representing efficiently three operands in a CISC architecture would make the encoding larger and harder than it is).&lt;&#x2F;p&gt;
&lt;h2 id=&quot;instruction-selection-lowering&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#instruction-selection-lowering&quot; aria-label=&quot;Anchor link for: instruction-selection-lowering&quot;&gt;🔗&lt;&#x2F;a&gt;Instruction selection (lowering)&lt;&#x2F;h2&gt;
&lt;p&gt;As said before, converting from the high-level IR (CLIF) to the low-level IR (VCode) is called lowering. Since VCode is target-dependent, this process is also target-dependent. That’s where we consider which machine instructions get eventually used for a given CLIF opcode. There are many ways to achieve the same machine state results for given semantics, but some of these ways are faster than other, and&#x2F;or require fewer code bytes to achieve. The problem can be summed up like this: given some CLIF, which VCode can we create to generate the fastest and&#x2F;or smallest machine code that carries out the desired semantics? This is called &lt;em&gt;instruction selection&lt;&#x2F;em&gt;, because we’re selecting the VCode instructions among a set of different possible instructions.&lt;&#x2F;p&gt;
&lt;p&gt;How do these IR map to each other? A given CLIF node may be lowered into 1 to N VCode instructions. A given VCode instruction may lead to the code generation of 1 to M machine instructions. There are no rules governing the maximum of entities mapped. For instance, the integer addition CLIF opcode &lt;code&gt;iadd&lt;&#x2F;code&gt; on 64-bits inputs maps to a single VCode instruction on aarch64. The VCode instruction then causes a single code instruction to be generated.&lt;&#x2F;p&gt;
&lt;p&gt;Other CLIF opcodes may generate more than a single machine instruction eventually. Consider the CLIF opcode for signed integer division &lt;code&gt;idiv&lt;&#x2F;code&gt;. Its semantics define that it traps for zero inputs and in case of integer overflow&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-3-1&quot;&gt;&lt;a href=&quot;#fn-3&quot;&gt;3&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;. On aarch64, this is lowered into:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;one VCode instruction that checks if the input is zero and trap otherwise&lt;&#x2F;li&gt;
&lt;li&gt;two VCode instructions for comparing the input values against the minimal integer value and -1&lt;&#x2F;li&gt;
&lt;li&gt;one VCode instruction to trap if the two input values match what we checked against&lt;&#x2F;li&gt;
&lt;li&gt;and one VCode instruction that does the actual division operation.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;Each of these VCode instruction then generates one or more machine code instructions, resulting in a bit of a longer sequence.&lt;&#x2F;p&gt;
&lt;p&gt;Let’s look at the lowering of &lt;code&gt;iadd&lt;&#x2F;code&gt; on aarch64 (in &lt;code&gt;cranelift&#x2F;codegen&#x2F;src&#x2F;isa&#x2F;aarch64&#x2F;lower_inst.rs&lt;&#x2F;code&gt;), edited and simplified for clarity. I’ve added comments in the code, explaining what each line does:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Opcode&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Iadd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&amp;gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Get the destination register.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; get_output_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; outputs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;0&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;]).&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;only_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;().&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;unwrap&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;();&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Get the controlling type of the addition (32-bits int or 64-bits int or&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; int vector, etc.).&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; ty&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; ty&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;unwrap&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;();&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Force one of the inputs into a register, not applying any signed- or&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; zero-extension.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; put_input_in_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; inputs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;0&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;],&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; NarrowValueMode&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;None&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Try to see if we can encode the second operand as an immediate on&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; 12-bits, maybe by negating it;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Otherwise, put it into a register.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; negated&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;) =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; put_input_in_rse_imm12_maybe_negated&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        inputs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;],&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;        ty_bits&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ty&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;),&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;        NarrowValueMode&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;None&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    );&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Select the ALU subopcode, based on possible negation and controlling&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; type.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; alu_op&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; if !&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;negated&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;        choose_32_64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ty&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; ALUOp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Add32&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; ALUOp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Add64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; else&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;        choose_32_64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ty&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; ALUOp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Sub32&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; ALUOp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Sub64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    };&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Emit the VCode instruction in the VCode stream.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;emit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;alu_inst_imm12&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;alu_op&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;));&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;In fact, the &lt;code&gt;alu_inst_imm12&lt;&#x2F;code&gt; wrapper can create one VCode instruction among a set of possible ones (since we’re trying to select &lt;em&gt;the best one&lt;&#x2F;em&gt;). For the sake of simplicity, we’ll assume that &lt;code&gt;AluRRR&lt;&#x2F;code&gt; is going to be generated, i.e. the selected instruction is the one using only register encodings for the input values.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;register-allocation&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#register-allocation&quot; aria-label=&quot;Anchor link for: register-allocation&quot;&gt;🔗&lt;&#x2F;a&gt;Register allocation&lt;&#x2F;h2&gt;
&lt;div class=&quot;mermaid&quot;&gt;
graph TD
    vcode_vreg[VCode with virtual registers]
    regalloc([Register allocation])
    vcode_rreg[VCode with real registers]
    codegen([Code generation])
    machine_code(Machine code)
    vcode_vreg --&gt; regalloc --&gt; vcode_rreg --&gt; codegen --&gt; machine_code
&lt;&#x2F;div&gt;
&lt;h3 id=&quot;vcode-registers-and-stack-slots&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#vcode-registers-and-stack-slots&quot; aria-label=&quot;Anchor link for: vcode-registers-and-stack-slots&quot;&gt;🔗&lt;&#x2F;a&gt;VCode, registers and stack slots&lt;&#x2F;h3&gt;
&lt;p&gt;Hey, ever wondered what the V in VCode meant? Back to the drawing board. While a program may reference a theoretically unlimited number of instructions, each referencing a theoretically unlimited number of values as inputs and outputs, the physical machine only has a fixed set of containers for those values:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;either they must live in machine &lt;strong&gt;registers&lt;&#x2F;strong&gt;: very fast to access in the CPU, take some CPU real estate, thus are costly, so there are usually few of them.&lt;&#x2F;li&gt;
&lt;li&gt;or they must live in the process’ &lt;strong&gt;stack memory&lt;&#x2F;strong&gt;: it’s slower to access, but we can have virtually any amount of stack &lt;em&gt;slots&lt;&#x2F;em&gt;.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;asm&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;mov&lt;&#x2F;span&gt;&lt;span&gt; %&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;edi&lt;&#x2F;span&gt;&lt;span&gt;,-&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;0x4&lt;&#x2F;span&gt;&lt;span&gt;(%&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;rbp&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;mov&lt;&#x2F;span&gt;&lt;span&gt; %&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;rsi&lt;&#x2F;span&gt;&lt;span&gt;,-&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;0x10&lt;&#x2F;span&gt;&lt;span&gt;(%&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;rbp&lt;&#x2F;span&gt;&lt;span&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;mov&lt;&#x2F;span&gt;&lt;span&gt; -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;0x4&lt;&#x2F;span&gt;&lt;span&gt;(%&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;rbp&lt;&#x2F;span&gt;&lt;span&gt;),%&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;eax&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;&lt;em&gt;In this example of x86 machine code, %edi, %rsi, %rbp, %eax are all registers; stack slots are memory addresses computed as the frame pointer (%rbp) plus an offset value (which happens to be negative here). Note that stack slots may be referred to by the stack pointer (%rsp) in general.&lt;&#x2F;em&gt;&lt;&#x2F;p&gt;
&lt;h3 id=&quot;defining-the-register-allocation-problem&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#defining-the-register-allocation-problem&quot; aria-label=&quot;Anchor link for: defining-the-register-allocation-problem&quot;&gt;🔗&lt;&#x2F;a&gt;Defining the register allocation problem&lt;&#x2F;h3&gt;
&lt;p&gt;The problem of mapping the IR values (in VCode these are the &lt;code&gt;Reg&lt;&#x2F;code&gt;) to machine “containers” is called &lt;strong&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Register_allocation&quot;&gt;register allocation&lt;&#x2F;a&gt;&lt;&#x2F;strong&gt; (aka regalloc). Inputs to register allocation can be as numerous as we want them, and map to “virtual” values, hence we call them &lt;em&gt;virtual registers&lt;&#x2F;em&gt;. And… that’s where the V from VCode comes from: the instructions in VCode reference values that are &lt;em&gt;virtual&lt;&#x2F;em&gt; registers before register allocation, so we say the code is in &lt;em&gt;virtualized&lt;&#x2F;em&gt; register form. The output of register allocation is a set of new instructions, where the virtual registers have been replaced by &lt;em&gt;real registers&lt;&#x2F;em&gt; (the physical ones, limited in quantity) or stack slots references (and other additional metadata).&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; Before register allocation, with unlimited virtual registers:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v2 = v0 + v1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v3 = v2 * 2&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v4 = v2 + 1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v5 = v4 + v3&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;return v5&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; One possible register allocation, on a machine that has 2 registers %r0, %r1:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r0 = %r0 + %r1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r1 = %r0 * 2&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r0 = %r0 + 1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r1 = %r0 + %r1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;return %r1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;When all is well, the virtual registers don’t conceptually &lt;em&gt;live&lt;&#x2F;em&gt; at the same time, and they can be put into physical registers. Issues arise when there’s not enough physical registers to contain all the virtual registers that live at the same time, which is the case for… a very large majority of programs. Then, register allocation must decide which registers continue to live in registers at a given program point, and which should be &lt;strong&gt;spilled&lt;&#x2F;strong&gt; into a stack slot, effectively &lt;em&gt;storing&lt;&#x2F;em&gt; them onto the stack for later use. This later reuse will imply to &lt;strong&gt;reload&lt;&#x2F;strong&gt; them from the stack slot, using a &lt;em&gt;load&lt;&#x2F;em&gt; machine instruction. The complexity resides in choosing which registers should be spilled, at which program point they should be spilled, and at which program points we should reload them, if we need to do so. Making good choices there will have a large impact on the speed of the generated code, since memory accesses to the stack imply an additional runtime cost. For instance, a variable that’s frequently used in a hot loop should live in a register for the whole loop’s lifetime, and not be spilled&#x2F;reloaded in the middle of the loop.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; Before register allocation, with unlimited virtual registers:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v2 = v0 + v1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v3 = v0 + v2&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v4 = v3 + v1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;return v4&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; One possible register allocation, on a machine that has 2 registers %r0, %r1.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; We need to spill one value, because there&amp;#39;s a point where 3 values are live at the same time!&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;spill %r1 --&amp;gt; stack_slot(0)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r1 = %r0 + %r1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r1 = %r0 + %r1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;reload stack_slot(0) --&amp;gt; %r0&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r1 = %r1 + %r0&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;return %r1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;And, since we like to have our cake and eat it too, the register allocator itself should be &lt;em&gt;fast&lt;&#x2F;em&gt;: it should not take an unbounded amount of time to make these allocation decisions. Register allocation has the good taste to be a &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;NP-completeness&quot;&gt;NP-complete&lt;&#x2F;a&gt; problem. Concretely, this means that implementations cannot find the &lt;em&gt;best&lt;&#x2F;em&gt; solutions given arbitrary inputs, but they’ll estimate &lt;em&gt;good&lt;&#x2F;em&gt; solutions based on heuristics, in worst-case quadratic time over the size of the input. All of this makes it so that register allocation has its own whole research field, and has been extensively studied for some time now. It is a fascinating problem.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;register-allocation-in-cranelift&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#register-allocation-in-cranelift&quot; aria-label=&quot;Anchor link for: register-allocation-in-cranelift&quot;&gt;🔗&lt;&#x2F;a&gt;Register allocation in Cranelift&lt;&#x2F;h3&gt;
&lt;p&gt;Back to Cranelift. The register allocation contract is that if a value &lt;em&gt;must&lt;&#x2F;em&gt; live in a real register at a given program point, then it &lt;em&gt;does&lt;&#x2F;em&gt; live where it should (unless register allocation is impossible). At the start of code generation for a VCode instruction, we are guaranteed that the input values live in real registers, and that the output real register is available before the next VCode instruction.&lt;&#x2F;p&gt;
&lt;p&gt;You might have noticed that the VCode instructions only refer to registers, and not stack slots. But where are the stack slots, then? The trick is that the stack slots are &lt;em&gt;invisible&lt;&#x2F;em&gt; to VCode. Register allocation may create an arbitrary number of spills, reloads, and register moves&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-5-1&quot;&gt;&lt;a href=&quot;#fn-5&quot;&gt;4&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; around VCode instructions, to ensure that their register allocation constraints are met. This is why the output of register allocation is a new list of instructions, that includes not only the initial instructions filled with the actual registers, but also additional spill, reload and move (VCode) instructions added by regalloc.&lt;&#x2F;p&gt;
&lt;p&gt;As said before, this problem is so sufficiently complex, involved and independent from the rest of the code (assuming the right set of interfaces!) that its code lives in a separate crate, &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bytecodealliance&#x2F;regalloc.rs&quot;&gt;&lt;code&gt;regalloc.rs&lt;&#x2F;code&gt;&lt;&#x2F;a&gt;, with its own fuzzing and testing infrastructure. I hope to shed some light on it at some point too.&lt;&#x2F;p&gt;
&lt;p&gt;What’s interesting to us today is the register allocation &lt;em&gt;constraints&lt;&#x2F;em&gt;. Consider the aarch64 integer add instruction &lt;code&gt;add rd, rn, rm&lt;&#x2F;code&gt;: &lt;code&gt;rd&lt;&#x2F;code&gt; is the output virtual register that’s written to, while &lt;code&gt;rn&lt;&#x2F;code&gt; and &lt;code&gt;rm&lt;&#x2F;code&gt; are the inputs, thus read from. We need to inform the register allocation algorithm about these constraints. In regalloc jargon, “read to” is known as &lt;em&gt;used&lt;&#x2F;em&gt;, while “written to” is known as &lt;em&gt;defined&lt;&#x2F;em&gt;. Here, the aarch64 VCode instruction &lt;code&gt;AluRRR&lt;&#x2F;code&gt; does &lt;em&gt;use&lt;&#x2F;em&gt; &lt;code&gt;rn&lt;&#x2F;code&gt; and &lt;code&gt;rm&lt;&#x2F;code&gt;, and it &lt;em&gt;def&lt;&#x2F;em&gt;ines &lt;code&gt;rd&lt;&#x2F;code&gt;. This usage information is &lt;em&gt;collected&lt;&#x2F;em&gt; in the &lt;code&gt;aarch64_get_regs&lt;&#x2F;code&gt; function (&lt;code&gt;cranelift&#x2F;codegen&#x2F;src&#x2F;isa&#x2F;aarch64&#x2F;inst&#x2F;mod.rs&lt;&#x2F;code&gt;):&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; aarch64_get_regs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; collector&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; RegUsageCollector&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    match&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;        &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;AluRRR&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;, .. } =&amp;gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            collector&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;add_def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            collector&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;add_use&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            collector&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;add_use&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; etc.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Then, after register allocation has assigned the physical registers, we need to instruct it how to replace virtual register mentions by physical register mentions. This is done in the &lt;code&gt;aarch64_map_regs&lt;&#x2F;code&gt; function (same file as above):&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; aarch64_map_regs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;RUM&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; RegUsageMapper&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; mapper&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;RUM&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; ...&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    match&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;        &amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;AluRRR&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;            ref&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt; mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;            ref&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt; mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;            ref&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt; mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;            ..&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;        } =&amp;gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;            map_def&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;mapper&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;            map_use&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;mapper&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;            map_use&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;mapper&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; etc.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Note this is reflecting quite precisely what the usage collector did: we’re replacing the virtual register mention for the defined register &lt;code&gt;rd&lt;&#x2F;code&gt; with the information (which real register) provided by the &lt;code&gt;RegUsageMapper&lt;&#x2F;code&gt;. These two functions must stay in sync, otherwise here be dragons! (and bugs very hard to debug!)&lt;&#x2F;p&gt;
&lt;h3 id=&quot;register-allocation-on-x86&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#register-allocation-on-x86&quot; aria-label=&quot;Anchor link for: register-allocation-on-x86&quot;&gt;🔗&lt;&#x2F;a&gt;Register allocation on x86&lt;&#x2F;h3&gt;
&lt;p&gt;On Intel’s x86, register allocation may be a bit trickier: in some cases, the lowering needs to be carefully written so it satisfies some register allocation constraints that are very specific to this architecture. In particular, x86 has &lt;em&gt;fixed register constraints&lt;&#x2F;em&gt; as well as &lt;em&gt;tied operands&lt;&#x2F;em&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;For this specific part, we’ll look at the integer shift-left instruction, which is equivalent to C’s &lt;code&gt;x &amp;lt;&amp;lt; y&lt;&#x2F;code&gt;. Why this particular instruction? It exhibits both properties that we’re interested in studying here. The lowering of &lt;code&gt;iadd&lt;&#x2F;code&gt; is similar, albeit slightly simpler, as it &lt;em&gt;only&lt;&#x2F;em&gt; involves tied operands.&lt;&#x2F;p&gt;
&lt;h4 id=&quot;fixed-register-constraints&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#fixed-register-constraints&quot; aria-label=&quot;Anchor link for: fixed-register-constraints&quot;&gt;🔗&lt;&#x2F;a&gt;Fixed register constraints&lt;&#x2F;h4&gt;
&lt;p&gt;On the one hand, some instructions expect their inputs to be in &lt;em&gt;fixed&lt;&#x2F;em&gt; registers, that is, specific registers arbitrarily predefined by the architecture manual. For the example of the shift instruction, if the count is not statically known at compile time (it’s not a shift by a constant value), then the amount by which we’re shifting must be in the &lt;code&gt;rcx&lt;&#x2F;code&gt; register&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-8-1&quot;&gt;&lt;a href=&quot;#fn-8&quot;&gt;5&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;Now, how do we make sure that the input value actually is in &lt;code&gt;rcx&lt;&#x2F;code&gt;? We can mark &lt;code&gt;rcx&lt;&#x2F;code&gt; as used in the &lt;code&gt;get_regs&lt;&#x2F;code&gt; function so regalloc knows about this, but nothing ensures that the input &lt;em&gt;resides&lt;&#x2F;em&gt; in it at the beginning of the instruction. To resolve this, we’ll introduce a &lt;strong&gt;move instruction&lt;&#x2F;strong&gt; during lowering, that is going to copy the input value into &lt;code&gt;rcx&lt;&#x2F;code&gt;. Then we’re sure it lives there, and register allocation knows it’s used: we’re good to go!&lt;&#x2F;p&gt;
&lt;p&gt;In a nutshell, this shows how lowering and register allocation play together:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;during lowering, we introduce a move from a dynamic shift input value to &lt;code&gt;rcx&lt;&#x2F;code&gt; before the actual shift&lt;&#x2F;li&gt;
&lt;li&gt;in the register usage function, we mark &lt;code&gt;rcx&lt;&#x2F;code&gt; as used&lt;&#x2F;li&gt;
&lt;li&gt;(nothing to do in the register mapping function: &lt;code&gt;rcx&lt;&#x2F;code&gt; is a real register already)&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h4 id=&quot;tied-operands&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#tied-operands&quot; aria-label=&quot;Anchor link for: tied-operands&quot;&gt;🔗&lt;&#x2F;a&gt;Tied operands&lt;&#x2F;h4&gt;
&lt;p&gt;On the other hand, some instructions have operands that are both read and written at the same time: we call them &lt;em&gt;modified&lt;&#x2F;em&gt; in Cranelift and regalloc.rs, but they’re also known as &lt;em&gt;tied operands&lt;&#x2F;em&gt; in the compiler literature. It’s not just that there’s a register that must be read, and a register that must be written to: they &lt;em&gt;must&lt;&#x2F;em&gt; be the same register. How do we model this, then?&lt;&#x2F;p&gt;
&lt;p&gt;Consider a naive solution. We take the input virtual register, and decide it’s allocated to the same register as the output (modified) register. Unfortunately, if the chosen virtual register was going to be reused by another later VCode instruction, then its value would be overwritten (clobbered) by the current instruction. This would result in incorrect code being generated, so this is not acceptable. In general we can’t clobber the value that was in an input value during lowering, because that’s the role of regalloc to make this kind of decisions.&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; Before register allocation, with virtual registers:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v2 = v0 + v1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;v3 = v0 + 42&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; After register allocation, on a machine with two registers %r0 and %r1:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;&#x2F;&#x2F; assign v0 to %r0, v1 to %r1, v2 to %r0&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;%r0 += v1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;... = %r0 + 42 &#x2F;&#x2F; ohnoes! the value in %r0 is v2, not v0 anymore!&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;The right solution is, again, to &lt;em&gt;copy&lt;&#x2F;em&gt; this input virtual register into the output virtual register, right before the instruction. This way, we can still reuse the untouched input register in other instructions without modifying it: only the copy is written to.&lt;&#x2F;p&gt;
&lt;p&gt;Pfew! We can now look at the entire lowering for the shift left instruction, edited and commented for clarity:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; Read the instruction operand size from the output&amp;#39;s type.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; size&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; dst_ty&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;bytes&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;() as&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; u8&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; Put the left hand side into a virtual register.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; lhs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; put_input_in_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; inputs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;0&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;]);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; Put the right hand side (shift amount) into either an immediate (if it&amp;#39;s&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; statically known at compile time), or into a virtual register.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;count&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rhs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;) =&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    if let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Some&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;) =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;get_input_as_source_or_const&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;insn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;).&lt;&#x2F;span&gt;&lt;span&gt;constant &lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;{&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; Mask count, according to Cranelift&amp;#39;s semantics.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; cst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; = (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; as&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; u8&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;) &amp;amp; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;dst_ty&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;bits&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;() as&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; u8&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; -&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;        (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Some&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;),&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; None&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; else&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;        (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;None&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Some&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;put_input_in_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; inputs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;])))&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    };&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; Get the destination virtual register.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; get_output_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; outputs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;0&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;]).&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;only_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;().&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;unwrap&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;();&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; Copy the left hand side into the (modified) output operand, to satisfy the&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; mod constraint.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;emit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;mov_r_r&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;true&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; lhs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;));&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; If the shift count is statically known: nothing particular to do. Otherwise,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; we need to put it in the RCX register.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; count&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;is_none&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;() {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; w_rcx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Writable&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;from_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;regs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;rcx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;());&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Copy the shift count (which is in rhs) into RCX.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;emit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;mov_r_r&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;true&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rhs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;unwrap&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(),&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; w_rcx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;));&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; Generate the actual shift instruction.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ctx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;emit&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;shift_r&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;size&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; ShiftKind&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ShiftLeft&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; count&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;));&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;And this is how we tell the register usage collector about our constraints:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ShiftR&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; num_bits&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;, .. } =&amp;gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; num_bits&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;is_none&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;() {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; if the shift count is dynamic, mark RCX as used.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        collector&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;add_use&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;regs&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;rcx&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;());&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; In all the cases, the destination operand is modified.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    collector&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;add_mod&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(*&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Only the modified register needs to be mapped to its allocated physical register:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;ShiftR&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; { ref&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt; mut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;, .. } =&amp;gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;    map_mod&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;mapper&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; dst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;);&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h3 id=&quot;virtual-registers-copies-and-performance&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#virtual-registers-copies-and-performance&quot; aria-label=&quot;Anchor link for: virtual-registers-copies-and-performance&quot;&gt;🔗&lt;&#x2F;a&gt;Virtual registers copies and performance&lt;&#x2F;h3&gt;
&lt;p&gt;Do these virtual register copies sound costly to you? In theory, they could lead to the code generation of a move instructions, increasing the size of the code generated and causing a small runtime cost. In practice,
register allocation, through its interface, knows how to identify move instructions, their source and their destination. By analyzing them, it can see when a source isn’t used after a given move instruction, and thus allocate the same register for the source and the destination of the move. Then, when Cranelift generates the code, it will avoid generating a move from a physical register to the same one&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-7-1&quot;&gt;&lt;a href=&quot;#fn-7&quot;&gt;6&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;. As a matter of fact, creating a VCode copy doesn’t necessarily mean that it will generate a machine code move instruction later: it is present just in case regalloc &lt;em&gt;needs&lt;&#x2F;em&gt; it, but it can be avoided when it’s spurious.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;code-generation&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#code-generation&quot; aria-label=&quot;Anchor link for: code-generation&quot;&gt;🔗&lt;&#x2F;a&gt;Code generation&lt;&#x2F;h2&gt;
&lt;p&gt;Oh my, we’re getting closer to actually being able to run the code! Once register allocation has run, we can generate the actual machine code for the VCode instructions. Cool kids call this step of the pipeline &lt;em&gt;codegen&lt;&#x2F;em&gt;, for code generation. This is the part where we decipher the architecture manuals provided by the CPU vendors, and generate the raw machine bytes for our machine instructions. In Cranelift, this means filling a code buffer (there’s a &lt;code&gt;MachBuffer&lt;&#x2F;code&gt; sink interface for this!), returned along some internal relocations&lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-6-1&quot;&gt;&lt;a href=&quot;#fn-6&quot;&gt;7&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; and additional metadata. Let’s see what happens for our integer addition, when the times come to generate the code for its VCode equivalent &lt;code&gt;AluRRR&lt;&#x2F;code&gt; on &lt;code&gt;aarch64&lt;&#x2F;code&gt; (in &lt;code&gt;cranelift&#x2F;codegen&#x2F;src&#x2F;isa&#x2F;aarch64&#x2F;inst&#x2F;emit.rs&lt;&#x2F;code&gt;):&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; We match on the VCode&amp;#39;s identity here:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;amp;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Inst&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;AluRRR&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; alu_op&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; } =&amp;gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; First select the top 11 bits based on the ALU subopcode.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; top11&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; match&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; alu_op&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        ALUOp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Add32&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 0b00001011_000&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        ALUOp&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;::&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Add64&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 0b10001011_000&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; etc&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    };&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Then decide the bits 10 to 15, based on the ALU subopcode as well.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    let&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; bit15_10&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; match&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; alu_op&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;        &#x2F;&#x2F; other cases&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        _&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 0b000000&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    };&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; Then use an helper and pass forward the allocated physical registers&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;    &#x2F;&#x2F; values.&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;    sink&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;put4&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;enc_arith_rrr&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;top11&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; bit15_10&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;));&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;And what’s this &lt;code&gt;enc_arith_rrr&lt;&#x2F;code&gt; doing, then?&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;rust&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;fn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; enc_arith_rrr&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;bits_31_21&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; bits_15_10&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Writable&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;Reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;) -&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; u32&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;    (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;bits_31_21&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; &amp;lt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 21&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;bits_15_10&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; &amp;lt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 10&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; machreg_to_gpr&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;rd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;to_reg&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;())&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;machreg_to_gpr&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;rn&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; &amp;lt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 5&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;machreg_to_gpr&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;rm&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; &amp;lt;&amp;lt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 16&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Encoding the instruction parts (operands, register mentions) is a lot of bit twiddling and fun. We do so for each VCode instruction, until we’ve generated the whole function’s body. If you remember correctly, at this point register allocation may have added some spills&#x2F;reloads&#x2F;move instructions. From the codegen’s point of view, these are just regular instructions with precomputed operands (either real registers, or memory operands involving the stack pointer), so they’re not treated particularly and they’re just generated the same way other VCode instructions are.&lt;&#x2F;p&gt;
&lt;p&gt;More work is done by the codegen backend then, to optimize blocks placement, compute final branch offsets, etc. If you’re interested by this, I strongly encourage you to go read &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;cfallin.org&#x2F;blog&#x2F;2021&#x2F;01&#x2F;22&#x2F;cranelift-isel-2&#x2F;&quot;&gt;this blog post&lt;&#x2F;a&gt; by Chris Fallin. After this, we’re finally done: we’ve produced a code buffer, as well as external relocations (to other functions, memory addresses, etc.) for a single function. The code generator’s task is complete: the final steps consist in linking and, optionally, producing an executable binary.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;mission-accomplished&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#mission-accomplished&quot; aria-label=&quot;Anchor link for: mission-accomplished&quot;&gt;🔗&lt;&#x2F;a&gt;Mission accomplished!&lt;&#x2F;h2&gt;
&lt;p&gt;So, we’re done for today! Thanks for reading this far, hope it has been a useful and pleasant read to you! Feel free to reach out to me on the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&quot;&gt;twitterz&lt;&#x2F;a&gt; if you have additional remarks&#x2F;questions, and to go contribute on &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bytecodealliance&#x2F;wasmtime&quot;&gt;Wasmtime&#x2F;Cranelift&lt;&#x2F;a&gt; if this sort of things is interesting to you 😇. Until next time, take care of yourselves!&lt;&#x2F;p&gt;
&lt;p&gt;Thanks to &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;cfallin.org&quot;&gt;Chris Fallin&lt;&#x2F;a&gt; for reading and suggesting improvements to this blog post.&lt;&#x2F;p&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn-2&quot;&gt;
&lt;p&gt;Really, Rust &lt;em&gt;is&lt;&#x2F;em&gt; the DSL. It was Python code before, that had the advantage to be faster to update. Yet it was doing a lot of magic behind the curtain, which wasn’t very friendly for new people trying to learn and use Cranelift. Despite a statically typed language helping for exploration through tooling, this meta-language is to partially disappear in the long run, see Chris’ &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;cfallin.org&#x2F;blog&#x2F;2020&#x2F;09&#x2F;18&#x2F;cranelift-isel-1&#x2F;&quot;&gt;blog post&lt;&#x2F;a&gt; on this topic. &lt;a href=&quot;#fr-2-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-4&quot;&gt;
&lt;p&gt;Aarch64 connoisseurs may notice that there are other ways to encode an addition. Say, if one of the input operands was the result of a bit shift instruction by an immediate value, then it’s possible to &lt;em&gt;embed&lt;&#x2F;em&gt; the shift within the add, so we end up with fewer machine instructions (and lower the register pressure). This other possible encoding is sufficiently different in terms of register allocation and code generation that it justifies having its own VCode instruction. &lt;code&gt;AluRRR&lt;&#x2F;code&gt; is simpler in the sense that it’s only concerned with register inputs and outputs, thus a perfect example for this post. &lt;a href=&quot;#fr-4-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-3&quot;&gt;
&lt;p&gt;What’s an integer overflow for signed integer division? Consider an integer value represented on &lt;code&gt;N&lt;&#x2F;code&gt; bits. If you try to divide the smallest integer value &lt;code&gt;-2**N&lt;&#x2F;code&gt; by &lt;code&gt;-1&lt;&#x2F;code&gt;, it should return &lt;code&gt;2**N&lt;&#x2F;code&gt;, but this is out of range, since the biggest signed integer value we can represent on &lt;code&gt;N&lt;&#x2F;code&gt; bits is &lt;code&gt;(2**N) - 1&lt;&#x2F;code&gt;! So this will overflow and be set to &lt;code&gt;-2**N&lt;&#x2F;code&gt;, which is the initial value, but not the correct result. Good luck debugging this without a software trap! &lt;a href=&quot;#fr-3-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-5&quot;&gt;
&lt;p&gt;Register moves may be introduced because a successor block (in the control flow graph) expects a given virtual register to live in a particular real register, or because a particular instruction requires a virtual register to be allocated to a &lt;em&gt;fixed&lt;&#x2F;em&gt; real register that’s busy: regalloc can then temporarily divert the busy register into another unused register. &lt;a href=&quot;#fr-5-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-8&quot;&gt;
&lt;p&gt;The &lt;code&gt;c&lt;&#x2F;code&gt; in &lt;code&gt;rcx&lt;&#x2F;code&gt; actually stands for &lt;code&gt;count&lt;&#x2F;code&gt;; this is a property inherited from former CPU designs. &lt;a href=&quot;#fr-8-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-7&quot;&gt;
&lt;p&gt;Unless this move carries sign- or zero-extending semantics, which is the case for e.g. x86’s 32-bits &lt;code&gt;mov&lt;&#x2F;code&gt; instructions on a 64-bits architecture. &lt;a href=&quot;#fr-7-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-6&quot;&gt;
&lt;p&gt;Relocations are placeholders for information we don’t have &lt;em&gt;yet&lt;&#x2F;em&gt; access to. For instance, when we’re generating jump instructions, the jump targets offsets are not determined yet. So we record where the jump instruction is in the code stream, as well as which control flow block it should jump into, so we can &lt;em&gt;patch it&lt;&#x2F;em&gt; later when the final offsets are known: that’s the content of our relocation. &lt;a href=&quot;#fr-6-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;&#x2F;section&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Botzilla, a multi-purpose Matrix bot tuned for Mozilla</title>
        <published>2020-11-12T18:49:42+00:00</published>
        <updated>2020-11-12T18:49:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/botzilla/"/>
        <id>https://bouvier.cc/tech/botzilla/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/botzilla/">&lt;p&gt;In this post I reflect on my personal history of writing chat bots, and then
present a panel of features that the bot has, some user-facing ones, some
others that embody what I esteem to be a sane, well-behaved Matrix bot.&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;
&lt;p&gt;Over the last year, Mozilla has decided to shut down the IRC network and
replace it with a more modern platform. To my greatest delight, &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;exple.tive.org&#x2F;blarg&#x2F;2019&#x2F;12&#x2F;19&#x2F;over-the-line&#x2F;&quot;&gt;the Matrix
ecosystem has been
selected&lt;&#x2F;a&gt; among all the
possible replacements. For those who might not know Matrix, it’s a modern,
decentralized protocol, using plain HTTP JSON-formatted endpoints,
well-documented, and it implements both features that are common in recent
messaging systems (e.g. file attachments, message edits and deletions), as well
as those needed to handle large groups (e.g. moderation tools, private rooms,
invite-only rooms).&lt;&#x2F;p&gt;
&lt;h2 id=&quot;but-first-some-history&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#but-first-some-history&quot; aria-label=&quot;Anchor link for: but-first-some-history&quot;&gt;🔗&lt;&#x2F;a&gt;but first, some history&lt;&#x2F;h2&gt;
&lt;p&gt;Back in 2014 when I was an intern at Mozilla, I made a silly IRC JavaScript bot
that would quote the @horsejs twitter account, when asked to do so. Then a few
other useless features were added: “karma” tracking &lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-1-1&quot;&gt;&lt;a href=&quot;#fn-1&quot;&gt;1&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt;, being a karma
guardian angel (lowering the karma of people lowering the karma of some
predefined people), keeping track of contextless quotes from misc people…&lt;&#x2F;p&gt;
&lt;p&gt;Over time, it slowly transformed into an IRC bot &lt;em&gt;framework&lt;&#x2F;em&gt;, with &lt;em&gt;modules&lt;&#x2F;em&gt;
you could attach and configure at startup, setting which rooms the bot would
join, what should be the cooldowns for message sending (more on this later),
and so much more! Hence it was renamed &lt;em&gt;meta-bot&lt;&#x2F;em&gt;.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;an-aside-on-the-morality-of-bots&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#an-aside-on-the-morality-of-bots&quot; aria-label=&quot;Anchor link for: an-aside-on-the-morality-of-bots&quot;&gt;🔗&lt;&#x2F;a&gt;an aside on the morality of bots&lt;&#x2F;h3&gt;
&lt;p&gt;I find making bots a fun activity, since once you’ve passed the step of
connecting and sending messages, the rest is mostly easy (&lt;em&gt;cough cough regular
expressions cough cough&lt;&#x2F;em&gt;) and creative work. And it’s unfortunately easy to be
reckless too.&lt;&#x2F;p&gt;
&lt;p&gt;At this time, I never considered the potentially bad effects of quoting text
from a random source, viz. fetching tweets from the @horsejs account. If the
source would return a message that was inconsiderate, rude, or even worse,
aggressive, then the bot would replicate this behavior. It is a real issue
because although the bot doesn’t think by itself and doesn’t &lt;em&gt;mean&lt;&#x2F;em&gt; any harm,
its programmers can do better, and they should try to avoid these issues at all
costs. A chat bot replicates the culture of the engineers who made it on one
hand, but also contributes to propagating this culture in the chat rooms it
participates in, &lt;em&gt;normalizing&lt;&#x2F;em&gt; it to the chat participants.&lt;&#x2F;p&gt;
&lt;p&gt;My bot happened to be well-behaved most of the time… until one time where it
was not. After noticing the incident and expressing my deepest apologies, I
deactivated the module and went through the whole list of modules, to make sure
none could cause any harm, in any possible way. I should have known better in
the first place! I am really not trying to signal my own virtue, since I failed
in a way that should have been predictable. I hope by writing this that other
people may reflect about the actions of their bots as well, in case they could
be misbehaving like this.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;the-former-fleet-of-mozilla-bots&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#the-former-fleet-of-mozilla-bots&quot; aria-label=&quot;Anchor link for: the-former-fleet-of-mozilla-bots&quot;&gt;🔗&lt;&#x2F;a&gt;the former fleet of mozilla bots&lt;&#x2F;h3&gt;
&lt;p&gt;There were a few other useful IRC bots (of which I wasn’t the author) hanging
out in the Mozilla IRC rooms, notably
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;globau&#x2F;firebot&quot;&gt;Firebot&lt;&#x2F;a&gt; and
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;wiki.mozilla.org&#x2F;Mrgiggles&quot;&gt;mrggigles&lt;&#x2F;a&gt;. The latter probably started as
a joke too, to enumerate puns from a list in the JavaScript channel. Then it
outgrew its responsibilities by helping with a handful of requests: who can
review this or this file in Mozilla’s source code? what’s the status of the
continuous integration trees? can this particular C++ function used in Gecko
cause a garbage collection?&lt;&#x2F;p&gt;
&lt;p&gt;When we moved over to Matrix, the bots unfortunately became outdated, since the
communication protocol (IRC) they were using was different. We could have
ported them to the Matrix protocol, but the Not-Invented-Here syndrom was
strong with this one: I’ve been making bots for a while, and I was personally
interested in the Matrix protocol and trying out the JS facilities offered by
the Matrix ecosystem.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;botzilla-features&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#botzilla-features&quot; aria-label=&quot;Anchor link for: botzilla-features&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;Botzilla features&lt;&#x2F;strong&gt;&lt;&#x2F;h2&gt;
&lt;p&gt;So I’ve decided to write &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;botzilla&quot;&gt;Botzilla&lt;&#x2F;a&gt;, a
successor in spirit to &lt;em&gt;meta-bot&lt;&#x2F;em&gt; and &lt;em&gt;mrgiggles&lt;&#x2F;em&gt;, written in TypeScript. This
is a very &lt;em&gt;unofficial&lt;&#x2F;em&gt; bot, tailored for Mozilla’s needs but probably useful in
other contexts. I’ve worked on it informally as a side-project, on my &lt;em&gt;copious&lt;&#x2F;em&gt;
spare time. Crafting tools that show useful to other people has been sufficient
a reward to motivate me to work on it, so it’s been quite fun!&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;logo.png&quot; alt=&quot;Botzilla’s logo&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;&lt;em&gt;Botzilla’s logo, courtesy of &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;nical.github.io&#x2F;index.html&quot;&gt;Nical&lt;&#x2F;a&gt;&lt;&#x2F;em&gt;&lt;&#x2F;p&gt;
&lt;p&gt;Let’s take a look at all the features that the bot offers, at this point.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;uuid-generate-unique-ids&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#uuid-generate-unique-ids&quot; aria-label=&quot;Anchor link for: uuid-generate-unique-ids&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;uuid: Generate unique IDs&lt;&#x2F;strong&gt;&lt;&#x2F;h3&gt;
&lt;p&gt;This was a feature of Firebot, and easy enough to replicate, so this was the
test feature for the Matrix bot. When saying &lt;code&gt;!uuid&lt;&#x2F;code&gt;, the bot will
automatically generate a unique id (using uuid v4), guaranteed GMO-free and
usable in any context that would require it. This was the first module,
designed to test the framework.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;uuid.png&quot; alt=&quot;Demo of uuid&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;h3 id=&quot;treestatus-inform-about-ci-tree-status&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#treestatus-inform-about-ci-tree-status&quot; aria-label=&quot;Anchor link for: treestatus-inform-about-ci-tree-status&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;treestatus: Inform about CI tree status&lt;&#x2F;strong&gt;&lt;&#x2F;h3&gt;
&lt;p&gt;Mozilla developers tend to interact a lot with the continuous integration
trees, because code is sometimes landed, sometimes backed out (sorry&#x2F;thank you
sheriffs!), sometimes merged across branches. This leads to the integration
trees being closed. Before we had the feature to automatically land patch
stacks when the trees reopened, it was useful to be able to get the open&#x2F;close
status of a tree. Asking &lt;code&gt;!treestatus&lt;&#x2F;code&gt; will answer with a list of the status of
some &lt;em&gt;common&lt;&#x2F;em&gt; trees. It is also possible to request the status of a particular
tree, e.g. for the “mozilla-central” tree, by asking &lt;code&gt;!treestatus mozilla-central&lt;&#x2F;code&gt; (or just &lt;code&gt;central&lt;&#x2F;code&gt;, as a handy shortcut).&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;treestatus.png&quot; alt=&quot;Demo of treestatus&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;h3 id=&quot;expand-bug-status&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#expand-bug-status&quot; aria-label=&quot;Anchor link for: expand-bug-status&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;Expand bug status&lt;&#x2F;strong&gt;&lt;&#x2F;h3&gt;
&lt;p&gt;If you have ever interacted with Mozilla’s code, there’s chances that you’ve
used Bugzilla, and mentioned bug numbers in conversations. The bot caches any
message containing &lt;code&gt;bug XXX&lt;&#x2F;code&gt; and will respond with a link to this bug, the
nickname of the person assigned to this bug if there’s one, and the summary of
this bug, if it’s public. This is by far the most used and useful module, since
it doesn’t require a special incantation, but will react automatically to a lot
of messages written with no particular intent (see below where it’s explained
how to not be spammy, though).&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;expand-bug.png&quot; alt=&quot;Demo of expand-bug&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;h3 id=&quot;who-can-review-x&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#who-can-review-x&quot; aria-label=&quot;Anchor link for: who-can-review-x&quot;&gt;🔗&lt;&#x2F;a&gt;Who Can Review X?&lt;&#x2F;h3&gt;
&lt;p&gt;This was a very nice feature that mrgiggles had: ask for potential reviewers
for a particular file in the Gecko source tree and get a list of most recent
reviewers. Botzilla replicates this, when seeing the trigger: &lt;code&gt;who can review js&#x2F;src&#x2F;wasm&#x2F;WasmJS.cpp?&lt;&#x2F;code&gt;. The list of potential reviewers is extracted from
Mercurial logs, looking for the N last reviewers of this particular file.&lt;&#x2F;p&gt;
&lt;p&gt;As a bonus, there’s no need to pass the full path to the file, if the file’s
name is unique in the tree’s source code. Botzilla will trigger a search in
Searchfox, and will use the unique name in the result list, if there’s such a
unique result. The previous example thus can be shortened to &lt;code&gt;who can review WasmJS.cpp?&lt;&#x2F;code&gt; since the file’s name is unique in the whole code base.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;who-can-review.png&quot; alt=&quot;Demo of who can review&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;h3 id=&quot;github-gitlab-issues-p-m-rs&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#github-gitlab-issues-p-m-rs&quot; aria-label=&quot;Anchor link for: github-gitlab-issues-p-m-rs&quot;&gt;🔗&lt;&#x2F;a&gt;{Github,Gitlab} {issues,{P,M}Rs}&lt;&#x2F;h3&gt;
&lt;p&gt;It is possible for a room administrator to “connect” a given Matrix room to a
Github repository. Later on, any mention of issues or pull requests by their
number, e.g. &lt;code&gt;#1234&lt;&#x2F;code&gt;, will make Botzilla react with the summary and a link to
the issue&#x2F;PR at stake.&lt;&#x2F;p&gt;
&lt;p&gt;This also works for Gitlab repositories, with slight differences: the
administrator has to precise what’s the root URL of the Gitlab instance (since
Gitlab can be selfhosted). Issues are caught when numbers follows a &lt;code&gt;#&lt;&#x2F;code&gt; sign,
while merge requests are caught when the numbers follow a &lt;code&gt;!&lt;&#x2F;code&gt; sign.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;gitlab.png&quot; alt=&quot;Demo of gitlab&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;h3 id=&quot;tweet-toot-post-on-twitter-mastodon&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#tweet-toot-post-on-twitter-mastodon&quot; aria-label=&quot;Anchor link for: tweet-toot-post-on-twitter-mastodon&quot;&gt;🔗&lt;&#x2F;a&gt;!tweet&#x2F;!toot: Post on Twitter&#x2F;Mastodon&lt;&#x2F;h3&gt;
&lt;p&gt;An administrator can configure a room to tie it up to a Twitter (respectively
Mastodon) user account, using API tokens. Then, any person with an
administrative role can post messages with &lt;code&gt;!tweet something shocking for the bird site&lt;&#x2F;code&gt;(respectively &lt;code&gt;!toot something heartful for the mammoth site&lt;&#x2F;code&gt;). This
makes it possible to allow other people to post on these social networks
without the need to give them the account’s password.&lt;&#x2F;p&gt;
&lt;p&gt;Unfortunately, the Twitter module hasn’t ever been tested, since when I’ve
tried to create a developer account, Twitter accepted it after a few days but
then never displayed the API tokens on the interface. The support also never
answered when I asked for help. Thankfully Mastodon can be self-hosted and thus
it is easier to test. I’m happy to report that it works quite well!&lt;&#x2F;p&gt;
&lt;h3 id=&quot;confession-and-histoire&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#confession-and-histoire&quot; aria-label=&quot;Anchor link for: confession-and-histoire&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;code&gt;confession&lt;&#x2F;code&gt; and histoire&lt;&#x2F;h3&gt;
&lt;p&gt;It is quite common in teams to set up regular standup meetings, where everyone
in the team announces what they’ve been working on in the last few days or
week. It also strikes me as important for personal recognition, including
towards management, to be able to &lt;em&gt;show off&lt;&#x2F;em&gt; (just a bit!) what you’ve
accomplished recently, and to remember this when times are harder (see also
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;jvns.ca&#x2F;blog&#x2F;brag-documents&#x2F;&quot;&gt;Julia Evans’ blog post on the topic&lt;&#x2F;a&gt;).&lt;&#x2F;p&gt;
&lt;p&gt;There’s a Botzilla module for this. Every time someone starts a message with
&lt;code&gt;confession:&lt;&#x2F;code&gt;, then everything after the colon will be saved in a database
(…wait for it!). Then, all the confessions are displayed on the
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;robotzilla.github.io&#x2F;histoire&quot;&gt;Histoire&lt;&#x2F;a&gt; &lt;sup class=&quot;footnote-reference&quot; id=&quot;fr-2-1&quot;&gt;&lt;a href=&quot;#fn-2&quot;&gt;2&lt;&#x2F;a&gt;&lt;&#x2F;sup&gt; website, with one
message feed per user. Note it is possible to send confessions privately to
Botzilla (that doesn’t affect the frontend though, which is open and public to
all!), or in a public channel. Public channels somehow equate to team members,
so channels also get their own pages on the frontend.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;confession.png&quot; alt=&quot;Demo of confession&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;botzilla&#x2F;histoire.png&quot; alt=&quot;Screenshot of Histoire&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;Now the fun&#x2F;cursed part is how all of this &lt;em&gt;works&lt;&#x2F;em&gt;. This was implemented in
&lt;em&gt;mrgiggles&lt;&#x2F;em&gt;, and I liked it a lot, since it required no kind of backend or
frontend server. How so? By (ab)using Github files as the database and Github
pages as the frontend. Sending a confession will trigger a request to a Github
endpoint to find a database file segregated by time, then it will trigger
another request to create&#x2F;modify it with the content of the confession. The
frontend then uses other requests to public Github APIs to read the confessions
before dynamically rendering those. Astute readers will notice that under a lot
of confession activity, the bot would be a bit slowed down by Github’s API use
rates. In this case, there’s some exponential backoff behavior before trying to
re-send unsaved confessions to Github. Overall it works great, and API
limitation rates have never quite been a problem.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;intrinsic-features-they-re-good-bots-bront&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#intrinsic-features-they-re-good-bots-bront&quot; aria-label=&quot;Anchor link for: intrinsic-features-they-re-good-bots-bront&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;Intrinsic features: they’re good bots, bront&lt;&#x2F;strong&gt;&lt;&#x2F;h2&gt;
&lt;p&gt;In addition to all the user-facing features, the bot has a few other
interesting attributes that are more relevant to consider from a framework
point of view. Hopefully some of these ideas can be useful for other bot
authors!&lt;&#x2F;p&gt;
&lt;h3 id=&quot;join-all-the-rooms&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#join-all-the-rooms&quot; aria-label=&quot;Anchor link for: join-all-the-rooms&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;Join All The Rooms!&lt;&#x2F;strong&gt;&lt;&#x2F;h3&gt;
&lt;p&gt;Every time the bot is invited to a channel, be it public or private, it will
join the channel, making it easy to use in general. It was implemented for free
by &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;turt2live&#x2F;matrix-js-bot-sdk&quot;&gt;the JS framework I’ve been
using&lt;&#x2F;a&gt;, and it is a
definitive improvement over the IRC version of the bot.&lt;&#x2F;p&gt;
&lt;p&gt;Sometimes Matrix rooms are upgraded to a new version of the room. The bot will
try to join the upgraded room if it can, keeping all its room settings intact
during the transition.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;thou-shalt-not-spam&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#thou-shalt-not-spam&quot; aria-label=&quot;Anchor link for: thou-shalt-not-spam&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;Thou shalt not spam&lt;&#x2F;strong&gt;&lt;&#x2F;h3&gt;
&lt;p&gt;To avoid spamming the channel, especially for modules that are &lt;em&gt;reactions&lt;&#x2F;em&gt; to
other messages (think: bug numbers, issues&#x2F;pull requests mentions), the bot has
had to learn how to keep quiet. There are two rules triggering the quieting
behavior:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;if the bot has already reacted less than N minutes ago (where N is a
configurable amount) in the same room,&lt;&#x2F;li&gt;
&lt;li&gt;or if it has already reacted to some entity in a message, and there’s been
fewer than M messages in between the last reaction and the last message
mentioning the same entity in the same room (M is also configurable)&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;If any of these two criteria is met, then the bot will keep quiet and it will
not react to another similar message. The combination of these two has proven
over time to be quite solid in my experience, based on observing the bot’s
behavior and public reactions to its behavior.&lt;&#x2F;p&gt;
&lt;p&gt;Some similar mechanism is used for the &lt;em&gt;confession&lt;&#x2F;em&gt; module: on a &lt;em&gt;first&lt;&#x2F;em&gt;
confession, the bot will answer with a message saying it has seen the
confession, including a link to where it is going to be posted, and will add an
emoji “eyes” reaction to the message. Posting this long form message could be
quite spammy, if there’s a lot of confessions around the same time. Under the
same criteria, it will just react with an “eyes” emoji to other confessions.
Later on, it’ll resend the full message, once both criterias aren’t blocking it
from doing so.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;decentralized-administration-self-service&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#decentralized-administration-self-service&quot; aria-label=&quot;Anchor link for: decentralized-administration-self-service&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;Decentralized administration self-service&lt;&#x2F;strong&gt;&lt;&#x2F;h3&gt;
&lt;p&gt;The bot can be administrated, by discussing with it using the &lt;code&gt;!admin&lt;&#x2F;code&gt; command.
This can happen in both a private conversation with it, or in public channels,
yet it is recommended to do so in private channels. To confirm that an admin
action has succeeded, it’ll use the thumbs-up emoji on the message doing the
particular action.&lt;&#x2F;p&gt;
&lt;p&gt;To have a single administrator for the bot would be quite the burden, and it is
not resilient to people switching roles, leaving the company, etc. Normally
you’d solve this by implementing your own access control lists. Fortunately,
Matrix already has a concept of &lt;em&gt;power levels&lt;&#x2F;em&gt; that assigns roles to users,
among which there are the administrator and moderator roles.&lt;&#x2F;p&gt;
&lt;p&gt;The bot will rely on this to decide to which requests it will answer. Somebody
marked as an administrator or a moderator of a room can administrate Botzilla
in this particular room, using &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;botzilla#admin&quot;&gt;the &lt;code&gt;!admin&lt;&#x2F;code&gt;
commands&lt;&#x2F;a&gt;. There’s still a
super-admin role, that must be defined in the configuration, in case things go
awry. While administrators only have power over the current room, a super-admin
can use its super-powers to change anything in any room. This decentralization
of the administrative roles makes it easy to have different settings for
different rooms, and to rely a bit less on single individuals.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;key-value-store&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#key-value-store&quot; aria-label=&quot;Anchor link for: key-value-store&quot;&gt;🔗&lt;&#x2F;a&gt;&lt;strong&gt;Key-value store&lt;&#x2F;strong&gt;&lt;&#x2F;h3&gt;
&lt;p&gt;In general, the bot contains a key-value store implemented in an sqlite
database, making it easy to migrate and add context that’s preserved across
restarts of the bot. This is used to store private information like user
repository information and settings for most rooms. Conceptually, each pair of
room and module has its own key-value store, so that there’s no risk of
confusion between different rooms and modules. There’s also a key-value
per-module store that’s applicable to all the rooms, to represent global
settings. If there’s some non-global (per room) settings for a room, these are
preferred over the global settings.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;self-documentation&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#self-documentation&quot; aria-label=&quot;Anchor link for: self-documentation&quot;&gt;🔗&lt;&#x2F;a&gt;Self-documentation&lt;&#x2F;h3&gt;
&lt;p&gt;Each chat module is implemented as a ECMAScript module and must export an help
string along the main reaction function. This is then captured and aggregated
as part of an &lt;code&gt;!help&lt;&#x2F;code&gt; command, that can be used to request help about usage of
the bot. The main help message will display the list of all the enabled
modules, and help about a specific module may be queried with e.g. &lt;code&gt;!help uuid&lt;&#x2F;code&gt;.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;future-work-and-conclusion&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#future-work-and-conclusion&quot; aria-label=&quot;Anchor link for: future-work-and-conclusion&quot;&gt;🔗&lt;&#x2F;a&gt;Future work and conclusion&lt;&#x2F;h2&gt;
&lt;p&gt;If I were to start again, I’d do a few things differently:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;now that the Rust ecosystem around the Matrix platform has matured a bit, I’d
probably write this bot in Rust. Starting from JavaScript and moving to
TypeScript has helped me catch a few static issues. I’d expect moving to Rust
would help handling Matrix events faster, provide end-to-end encryption
support for free, and be quite pleasant to use in general thanks to the
awesome Rust tooling.&lt;&#x2F;li&gt;
&lt;li&gt;use a real single-page app framework for the Histoire website. Maybe? I mean
I’m a big fan of VanillaJS, but using it means re-creating your own Web
framework like thing to make it nice and productive to use.&lt;&#x2F;li&gt;
&lt;li&gt;despite being a fun hack, using Github as a backend has algorithmic
limitations, that can make the web app sluggish. In particular, a combined
feed for N users on M &lt;em&gt;eras&lt;&#x2F;em&gt; (think: periods) will trigger NxM Github API
requests. Using a plain database with a plain API would probably be simpler
at this point. This is mitigated with an in-memory cache so only the first
time all the requests happen, but crafting my own requests would be more
expressive and efficient, and allow for more features too (like displaying
the list of rooms on the start view).&lt;&#x2F;li&gt;
&lt;li&gt;provide a (better) commands parser. Regular expressions in this context are a
bit feeble and limited. Also right now each module could in theory reuse the
same command triggers as another one, etc.&lt;&#x2F;li&gt;
&lt;li&gt;implement the chat modules in WebAssembly :-) In fact, I think there’s a
whole business model which would consist in having the bot framework
including a wasm VM, and interacting with different communication platforms
(not restricted to Matrix). Developers in such a bot platform could choose
which source language to use for developing their own modules. It ought to be
possible to define a clear, restricted, WASI-like capabilities-based
interface that gets passed to each chat module. In such a sandboxed
environment, the responsibility for hosting the bot’s code is decoupled from
the responsibility of writing modules. So a company could make the platform
available, and paying users would develop the modules and host them. Imagine
&lt;code&gt;git push&lt;&#x2F;code&gt;ing your chat modules and they get compiled to wasm and deployed on
the fly. But I digress! (Please do not forget to credit me with a large $$$
envelope&#x2F;a nice piece of swag if implementing this &lt;em&gt;at least&lt;&#x2F;em&gt; multi-billion
dollars idea.)&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;I’d like to finish by thanking the authors of the previous Mozilla bots, namely
&lt;strong&gt;sfink&lt;&#x2F;strong&gt; and &lt;strong&gt;glob&lt;&#x2F;strong&gt;: your puppets have been incredible sources of
inspiration. Also huge thanks to the people hanging in the &lt;code&gt;matrix-bot-sdk&lt;&#x2F;code&gt;
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;matrix.to&#x2F;#&#x2F;!matrix-bot-sdk:t2bot.io&quot;&gt;chat room&lt;&#x2F;a&gt;, who’ve answered questions and provided
help in a few occasions.&lt;&#x2F;p&gt;
&lt;p&gt;I hope you liked this presentation of Botzilla and its features! Of course, all
the code is free and open-source, including &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;botzilla&quot;&gt;the
bot&lt;&#x2F;a&gt; as well as &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;robotzilla&#x2F;histoire&quot;&gt;the histoire
frontend&lt;&#x2F;a&gt;. At this point it is
addressing most of the needs I had, so I don’t have immediate plans to extend
it further. I’d happily take contributions, though, so feel free to chime in if
you’d like to implement anything! It’s also a breeze to run on any machine,
thanks to Docker-based deployment. Have fun with it!&lt;&#x2F;p&gt;
&lt;section class=&quot;footnotes&quot;&gt;
&lt;ol class=&quot;footnotes-list&quot;&gt;
&lt;li id=&quot;fn-1&quot;&gt;
&lt;p&gt;Karma is an IRC idiosyncrasy, in which users rate up and down other users
using their nickname suffixed with ++ or –. Karma tracking consists in
keeping scores and displaying those. &lt;a href=&quot;#fr-1-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;li id=&quot;fn-2&quot;&gt;
&lt;p&gt;Histoire is the French for “history” and “story”. Inherited from Steve
Fink’s very own mrgiggles :-) &lt;a href=&quot;#fr-2-1&quot;&gt;↩&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;&#x2F;section&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Improving my Github workflow</title>
        <published>2019-10-10T18:00:42+00:00</published>
        <updated>2019-10-10T18:00:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/github-workflow/"/>
        <id>https://bouvier.cc/tech/github-workflow/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/github-workflow/">&lt;p&gt;Since I’ve been working on a &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;CraneStation&#x2F;Cranelift&quot;&gt;Github
project&lt;&#x2F;a&gt; for a while now, I
thought now would be a good time to gather ways to make it easier to work with
Github pull requests (PRs). In particular, it’s easy to drown yourself in the
incoming flow of Github emails.&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;
&lt;p&gt;This post is for you if:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;you get lost in tracking which pull requests need attention from you, be it
either review requests or just mentions.&lt;&#x2F;li&gt;
&lt;li&gt;you would like to strike a better work-life balance when it gets to Github
notifications.&lt;&#x2F;li&gt;
&lt;li&gt;you would like to filter Github email notifications in smarter ways.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;Here are a few tricks I’ve collected over the years that make it easier to deal
with a few things, focusing on Github notifications and emails, since they were
the largest issue for me. This is not an exhaustive list of all the nice
features Github has, or all the WebExtensions that could help with Github: it
is a few things that work for me and are worth sharing. Note that I go from the
most mundane to the more specific advices here.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;notifications-dashboard&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#notifications-dashboard&quot; aria-label=&quot;Anchor link for: notifications-dashboard&quot;&gt;🔗&lt;&#x2F;a&gt;Notifications dashboard&lt;&#x2F;h3&gt;
&lt;p&gt;If you’re working on several projects, Github can end up sending you too many
email notifications.&lt;&#x2F;p&gt;
&lt;p&gt;It’s possible to disable some kinds of notifications entirely &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;settings&#x2F;notifications&quot;&gt;in the
settings&lt;&#x2F;a&gt;, but that’s too radical
for my needs.&lt;&#x2F;p&gt;
&lt;p&gt;However, Github has a &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;notifications&quot;&gt;notification
dashboard&lt;&#x2F;a&gt; that displays all the activity
related to repositories you’re watching or issues&#x2F;pull-requests you’re involved
in. It’s easy to dismiss all the notifications of all projects at once, or per
project. There’s a tab on the left that allows to select more precisely your
level of involvement in the issue: did you participate in it? You can also save
some notifications for later, so they’re not deleted once you’ve clicked them;
they’ll appear under the “Saved for later” tab — I just discovered this!&lt;&#x2F;p&gt;
&lt;p&gt;Note that Github may also send these notifications by email, if you’ve decided
to do so. In this case, I’d strongly recommend allowing the downloads of images
in Github emails. Despite the bad effect on your privacy this might have by
allowing user tracking, it will also synchronize the notifications’ read state,
which is nice.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;github-workflow&#x2F;github-notification-dashboard.png&quot; alt=&quot;Notification dashboard count&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;See how I am totally in control of my notifications? Truth is, I don’t need
notifications in general, because I’m usually more interested in reviews I need
to receive and give.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;pull-requests-dashboard&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#pull-requests-dashboard&quot; aria-label=&quot;Anchor link for: pull-requests-dashboard&quot;&gt;🔗&lt;&#x2F;a&gt;Pull requests dashboard&lt;&#x2F;h3&gt;
&lt;p&gt;Github allows to assign a reviewer to a pull request. At Mozilla, we
require a formal review for each change in the code base, unless it’s really
not meaningful (like, removing trailing whitespaces). Even documentation and
tests changes may require a review, depending on the rules of the code module
you’re working on.&lt;&#x2F;p&gt;
&lt;p&gt;It is very common that a pull request is received with requests for additional
changes. In this case, it is important to explicitly &lt;strong&gt;re-request a review&lt;&#x2F;strong&gt;,
otherwise this breaks all the review tracking Github proposes.&lt;&#x2F;p&gt;
&lt;p&gt;Now Github has two interesting pages for this:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;a list of all the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;pulls&quot;&gt;pull requests you have created&lt;&#x2F;a&gt;
and that aren’t closed, so you can assign reviewers and follow PR’s progress
over time;&lt;&#x2F;li&gt;
&lt;li&gt;a list of all the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;pulls&#x2F;review-requested&quot;&gt;pull requests you have been assigned to as a
reviewer&lt;&#x2F;a&gt;.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h3 id=&quot;navigating-files-quicker-addon&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#navigating-files-quicker-addon&quot; aria-label=&quot;Anchor link for: navigating-files-quicker-addon&quot;&gt;🔗&lt;&#x2F;a&gt;Navigating files quicker (addon)&lt;&#x2F;h3&gt;
&lt;p&gt;When I know my way around a project, I’ll frequently need to see the content of
a particular file or directory, that might be a few directories deep. On
Github, this means going to the files view, clicking once per directory (at
most), and finding the file I want.&lt;&#x2F;p&gt;
&lt;p&gt;The pull request view doesn’t show the directory hierarchy and which files of
which directory have been touched, which is a light inconvenience too.&lt;&#x2F;p&gt;
&lt;p&gt;Good news, everyone! There is one WebExtension called
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.octotree.io&#x2F;&quot;&gt;Octotree&lt;&#x2F;a&gt; that adds a directory view within a panel
to the left of Github’s UI. By default, it’s folded and doesn’t take much
space; you need to hover it with the mouse to make it appear. On pull requests,
it will show files that have been modified with the diff summary for each file.
Note the website shows features from the PRO version, but there’s a free
version that addresses the needs detailed above.&lt;&#x2F;p&gt;
&lt;p&gt;This is an example of the Octotree panel on our project’s repository:&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;github-workflow&#x2F;github-octotree.png&quot; alt=&quot;Octotree example&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;To be honest, I haven’t investigated using the search bar, which could be quite
handy for this too, especially thanks to keyboard shortcuts.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;dealing-with-work-and-personal-projects&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#dealing-with-work-and-personal-projects&quot; aria-label=&quot;Anchor link for: dealing-with-work-and-personal-projects&quot;&gt;🔗&lt;&#x2F;a&gt;Dealing with work and personal projects&lt;&#x2F;h3&gt;
&lt;p&gt;If you’re using Github for personal and work related projects, you might have
been bothered by work emails coming into your personal mailbox. That has
happened to me in the past, causing some unnecessary mental load over the
weekend and unnecessarily breaking the state of relaxation.&lt;&#x2F;p&gt;
&lt;p&gt;Fortunately, Github allows you to &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;settings&#x2F;notifications#organization_routing&quot;&gt;redirect emails from a particular Github
Organization to a specific email
address&lt;&#x2F;a&gt;. Of
course, this only works when the repository is owned by an organization and
you’re part of this organization.&lt;&#x2F;p&gt;
&lt;p&gt;I’m lucky to work on such projects at the moment. It’s not a silver bullet
though, because some projects are sometimes owned by personal accounts, making
this trick useless. As far as I know, there are no good solutions in this case.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;email-filters&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#email-filters&quot; aria-label=&quot;Anchor link for: email-filters&quot;&gt;🔗&lt;&#x2F;a&gt;Email filters&lt;&#x2F;h3&gt;
&lt;p&gt;The biggest remaining offender certainly is Github emails, in general.
Fortunately, Github has made it easy to filter them. I’ll mention examples in
the Gmail email client, since that’s what we’re using at work, but these apply
to any other modern email client too.&lt;&#x2F;p&gt;
&lt;h4 id=&quot;filter-by-project&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#filter-by-project&quot; aria-label=&quot;Anchor link for: filter-by-project&quot;&gt;🔗&lt;&#x2F;a&gt;Filter by project&lt;&#x2F;h4&gt;
&lt;p&gt;Each email coming from a specific project comes with a mailing &lt;code&gt;list-id&lt;&#x2F;code&gt;, which
is a specific header that some email clients know how to interpret. For
instance, in Gmail, when you click on the small arrow next to the list of
recipients, you’ll see many details about the current email, including, if
there’s one, the “mailing-list” id, and a link to automatically create a filter
for this mailing-list. That allows you to create a particular directory&#x2F;tag in
which the filter can automatically put all the emails with this id.&lt;&#x2F;p&gt;
&lt;h4 id=&quot;filter-by-reason&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#filter-by-reason&quot; aria-label=&quot;Anchor link for: filter-by-reason&quot;&gt;🔗&lt;&#x2F;a&gt;Filter by reason&lt;&#x2F;h4&gt;
&lt;p&gt;In addition to filtering by project (and this is where Gmail tags &#x2F;
Thunderbird’s &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;support.mozilla.org&#x2F;en-US&#x2F;kb&#x2F;using-saved-searches&quot;&gt;saved
searches&lt;&#x2F;a&gt; truly
shine), it’s also possible to infer more information from the Github email
notifications, by looking at the list of recipients or custom email headers.&lt;&#x2F;p&gt;
&lt;p&gt;Indeed, when there’s a specific reason why an email was sent to you, Github
will add a (fake) recipient in the CC field, its address username being the
reason why the email was sent to you. For instance, in an email telling me that
somebody requested a review from me, the email address
&lt;code&gt;review_requested@github.com&lt;&#x2F;code&gt; will appear in the CC list. If you look at the
full message, you’ll also see the custom email header &lt;code&gt;X-GitHub-Reason&lt;&#x2F;code&gt; set to
&lt;code&gt;review_requested&lt;&#x2F;code&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;All the possible reasons are &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;help.github.com&#x2F;en&#x2F;articles&#x2F;about-email-notifications#filtering-email-notifications&quot;&gt;detailed in Github’s
documentation&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;These extra CC email addresses and email headers allow creating very powerful
filters that add supplementary tags to an email. For me, they relate directly
to the &lt;em&gt;importance&lt;&#x2F;em&gt; of the incoming email: reviews and mentions are usually
something I pay very close attention to, and thus they get filtered in a
special top-level tag in Gmail.&lt;&#x2F;p&gt;
&lt;p&gt;Here’s an example of all the information you might find about a given email in
Gmail: in particular, look at the CC list and mailing-list type ids.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;github-workflow&#x2F;github-email-example.png&quot; alt=&quot;Notification dashboard count&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;Note that Gitlab also adds some &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;docs.gitlab.com&#x2F;ee&#x2F;workflow&#x2F;notifications.html#email-headers&quot;&gt;similar custom
headers&lt;&#x2F;a&gt;
that can be filtered by some powerful email clients. I won’t go into detail
about those.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;one-more-thing-mozillian-edition&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#one-more-thing-mozillian-edition&quot; aria-label=&quot;Anchor link for: one-more-thing-mozillian-edition&quot;&gt;🔗&lt;&#x2F;a&gt;One more thing, Mozillian edition&lt;&#x2F;h3&gt;
&lt;p&gt;If you’re working on Mozilla code, Gecko and&#x2F;or external projects, there’s
this &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;mikeconley&#x2F;myqonly&#x2F;&quot;&gt;neat addon&lt;&#x2F;a&gt; that Mike Conley
made. It will add an icon to the Firefox button bar, showing you the number
of requests assigned to you on Github and Phabricator, as well as the number of
pending Bugzilla requests.&lt;&#x2F;p&gt;
&lt;p&gt;It requires a minimal setup step for Github (filling your username) and
Bugzilla (adding a Bugzilla API token), and then it Just Works. It smartly
reuses a Phabricator token from the current Firefox Container’s session, if
there’s one.&lt;&#x2F;p&gt;
&lt;p&gt;You may think that having such a display all the time might provoke anxiety
during non-working hours. And you’d be right to think so! So the author of the
addon has added a feature to &lt;strong&gt;not&lt;&#x2F;strong&gt; display this information outside working
hours, that you can define as you like. Great stuff!&lt;&#x2F;p&gt;
&lt;h3 id=&quot;that-s-it-folks&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#that-s-it-folks&quot; aria-label=&quot;Anchor link for: that-s-it-folks&quot;&gt;🔗&lt;&#x2F;a&gt;That’s it, folks!&lt;&#x2F;h3&gt;
&lt;p&gt;Thanks for reading this far! I hope this helped you to some extent, allowing
you to spend less time in Github and more time doing the actual work. If you
have more interesting tips for using Github effectively, feel free to add a
comment or ping me on &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&quot;&gt;twitter&lt;&#x2F;a&gt;!&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>A new blog theme!</title>
        <published>2019-06-13T19:00:42+00:00</published>
        <updated>2019-06-13T19:00:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/a-new-blog-theme/"/>
        <id>https://bouvier.cc/tech/a-new-blog-theme/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/a-new-blog-theme/">&lt;p&gt;Just a quick note! I’ve just updated my blog’s design and UI so it is more
convenient, modern and light. It is still using the
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;blog.getpelican.com&#x2F;&quot;&gt;Pelican&lt;&#x2F;a&gt; blog system, only the theme has
changed. The previous theme, &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;alexandrevicenzi&#x2F;flex&quot;&gt;Flex&lt;&#x2F;a&gt;,
was elegant and it has been quite pleasant to use, but it had a few subtle
drawbacks that revealed as time went. Here I explain what were my needs, which
other blog designs inspired me, and link to all the different resources I’ve
used.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;embeds-tracking-and-comments&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#embeds-tracking-and-comments&quot; aria-label=&quot;Anchor link for: embeds-tracking-and-comments&quot;&gt;🔗&lt;&#x2F;a&gt;Embeds: tracking and comments&lt;&#x2F;h2&gt;
&lt;p&gt;I use some embeds in my blog: since Pelican generates static files, it can’t
handle more complicated features like counting visitors or a comment system.
Soon, I’ve wanted to get a rough idea of the number of visitors and where they
come from, to understand which posts are the most useful (and of course, to
bless my ego). I’ve been using self-hosted, open-source systems for that
purpose, like &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;matomo.org&#x2F;&quot;&gt;Matomo&lt;&#x2F;a&gt; (née Piwik), because I don’t want
my blog’s visitors to get (even more) tracked by Google Analytics, which would
be counter to my values. This led to a meaningful contribution to add &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;alexandrevicenzi&#x2F;Flex&#x2F;pull&#x2F;15&quot;&gt;support
for Matomo&lt;&#x2F;a&gt; in the theme, so
other people don’t have to redo this work themselves!&lt;&#x2F;p&gt;
&lt;p&gt;Later, I switched over to the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;usefathom.com&#x2F;&quot;&gt;Fathom&lt;&#x2F;a&gt; analytics
system, because it’s lighter, it knows less about my visitors (only number of
visits&#x2F;visitors by page and the referrer), and its dashboard is a gazillion
times faster to load (all it loads are static files!) than Matomo’s. So I had
to tweak the theme for this too! And then, when I decided to add an embed for a
self-hosted, open-source comment system named &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;posativ.org&#x2F;isso&quot;&gt;isso&lt;&#x2F;a&gt;
(think of it as an alternative to the centralized Disqus), I needed to add
support for it as well. This meant digging into the theme and understanding how
it works, and each time I wanted to add a new embed, I had to repeat this
process.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;css-and-markup&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#css-and-markup&quot; aria-label=&quot;Anchor link for: css-and-markup&quot;&gt;🔗&lt;&#x2F;a&gt;CSS and markup&lt;&#x2F;h2&gt;
&lt;p&gt;After reading a well-written and quite interesting online class on typography
on &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;betterwebtype.com&#x2F;&quot;&gt;BetterWebType&lt;&#x2F;a&gt; (read it if you haven’t! it
helped me understanding why some websites were more pleasant to use, and put
explanations onto all the intuitions and feelings I had on the topic), I’ve
changed a few lines in the CSS source file, tweaking fonts, width, line height
etc.&lt;&#x2F;p&gt;
&lt;p&gt;A few other changes affecting the markup also happened over time, for which I
had to add CSS rules. Of course there were a few conflicts with preexisting
rules, since that’s pretty common in CSS; but the fact that I didn’t write the
CSS rules in the first place made it harder to understand how and where these
rules were used.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;writing-my-own&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#writing-my-own&quot; aria-label=&quot;Anchor link for: writing-my-own&quot;&gt;🔗&lt;&#x2F;a&gt;Writing my own&lt;&#x2F;h2&gt;
&lt;p&gt;I also got a bit bored of the design itself, especially the aside bar on the
left which felt a bit early 2010, as well as the overall sense of density. Some
other blogs’ strong takes inspired me a lot: in particular, I’d like to point
to my colleague and friend &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;blog.paul.cx&#x2F;&quot;&gt;Paul Adenot’s blog&lt;&#x2F;a&gt;, and its
refined, minimalistic design (as well as its amazing body font, look at the
K&#x2F;it or Q&#x2F;uo ligatures, they’re &lt;em&gt;insane&lt;&#x2F;em&gt;). There’s also the blog of &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;jvns.ca&#x2F;&quot;&gt;Julia
Evans&lt;&#x2F;a&gt;, which in addition to containing amazingly instructive
blog posts about system programming, shows all the blog posts as a long list in
the index page, making it clear, informative and helping discoverability.&lt;&#x2F;p&gt;
&lt;p&gt;With all of this in mind, since I had to acquire more control over the theme
anyway and I wanted it to look much different, I decided to write my own. Of
course, it’s &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;blog.benj.me&#x2F;tree&#x2F;sources&#x2F;themes&#x2F;modern&quot;&gt;free
software&lt;&#x2F;a&gt;.
This is what you should be seeing right now.&lt;&#x2F;p&gt;
&lt;p&gt;As an extra to all the items I’ve mentioned before, I’d like to give a shoutout
to the open-source &lt;em&gt;fonts&lt;&#x2F;em&gt; I’ve used. The header font is
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;velvetyne.fr&#x2F;fonts&#x2F;bluu&#x2F;&quot;&gt;bluu&lt;&#x2F;a&gt;, a font created by Jean-Baptiste
Morizot from the fantastic &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;velvetyne.fr&quot;&gt;Velvetyne&lt;&#x2F;a&gt; type foundry. Go
check it out if you’re a typo nerd! or if you’re looking for some original and
well-thought fonts. The body font is &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;rsms.me&#x2F;inter&#x2F;&quot;&gt;Inter UI&lt;&#x2F;a&gt;; it
feels quite modern and it has been designed with computer screens in mind.&lt;&#x2F;p&gt;
&lt;p&gt;Previous vs new home page:&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;a-new-blog-theme&#x2F;previous-index.png&quot; alt=&quot;Previous home page&quot; &#x2F;&gt;
&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;a-new-blog-theme&#x2F;new-index.png&quot; alt=&quot;New home page&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;Previous vs new post page:&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;a-new-blog-theme&#x2F;previous-post.png&quot; alt=&quot;Previous post page&quot; &#x2F;&gt;
&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;a-new-blog-theme&#x2F;new-post.png&quot; alt=&quot;New post page&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;h2 id=&quot;flex-thee-well&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#flex-thee-well&quot; aria-label=&quot;Anchor link for: flex-thee-well&quot;&gt;🔗&lt;&#x2F;a&gt;Flex thee well&lt;&#x2F;h2&gt;
&lt;p&gt;I still think Flex is a great theme to start with, and I’m thankful to the
authors for writing it in the first place, since it has served me well. Thanks!&lt;&#x2F;p&gt;
&lt;p&gt;I’d also like to thank people who gave me feedback and helped me write the CSS
(me can’t CSS) of this theme, including Flaburgan, Maiwann, Marien, Thomas
and other &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;framasoft.org&#x2F;en&quot;&gt;Framasoft&lt;&#x2F;a&gt; friends.&lt;&#x2F;p&gt;
&lt;p&gt;And thank &lt;strong&gt;you&lt;&#x2F;strong&gt; for reading this far! I think blogging and self-hosting are
still important and relevant, to avoid depending on commercial platforms, like
Medium, which repeatedly tries to get you to subscribe, displays thousands of
popovers and attempts to abuse your attention into reading always more (and
spending more time on their website).  It’s easy to host your own blog! Even if
you don’t have your own server, you can use Github or
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;docs.gitlab.com&#x2F;ee&#x2F;user&#x2F;project&#x2F;pages&#x2F;&quot;&gt;Gitlab&lt;&#x2F;a&gt; pages, with fancy
automatic systems that make publishing a breeze.&lt;&#x2F;p&gt;
&lt;p&gt;I’d be curious to hear your feedback on this new design, including code
improvements (remember? it’s free software!). Feel free to type in a comment,
or hit me up on &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&quot;&gt;twitter&lt;&#x2F;a&gt;!&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Laverna Webclipper, the story of a WebExtension</title>
        <published>2016-09-18T17:10:42+00:00</published>
        <updated>2016-09-18T17:10:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/laverna-webclipper-webextension/"/>
        <id>https://bouvier.cc/tech/laverna-webclipper-webextension/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/laverna-webclipper-webextension/">&lt;p&gt;Yesterday I’ve spent my afternoon on a very small side-project related to
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;laverna.cc&#x2F;app&quot;&gt;Laverna&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;Laverna is an offline-first, no-backend Web application that allows you to
write notes directly from your browser, classify them in notebooks, with a
live-preview Markdown editor and a powerful yet simple tagging system. Besides
being offline-first, it also allows you to sync between different devices,
using &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;dropbox.github.io&#x2F;dropbox-sdk-js&#x2F;&quot;&gt;Dropbox&lt;&#x2F;a&gt; as a backend or your
own instance of a &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;remotestorage.io&#x2F;&quot;&gt;RemoteStorage&lt;&#x2F;a&gt; server. All of
this makes it a powerful free and open-source alternative to software like
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;www.evernote.com&#x2F;&quot;&gt;Evernote&lt;&#x2F;a&gt; or even &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;www.onenote.com&#x2F;&quot;&gt;Microsoft
OneNote&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;a-webclipper-for-laverna&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#a-webclipper-for-laverna&quot; aria-label=&quot;Anchor link for: a-webclipper-for-laverna&quot;&gt;🔗&lt;&#x2F;a&gt;A WebClipper for Laverna&lt;&#x2F;h2&gt;
&lt;p&gt;There was already an official
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;laverna&#x2F;webclipper&quot;&gt;webclipper&lt;&#x2F;a&gt;, but when I’ve tried it, it
didn’t work. Looking at the code and the last commit date, it seemed a bit
deprecated; so I made my own that uses WebExtensions (since I really wanted to
learn a bit more about these), that you can find &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;laverna-webclipper&quot;&gt;here on
Github&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;The idea behind a web clipper is very simple: you go on a Web page that
contains some interesting content and you would like to keep it for later (for
example, to read it later, or to keep it as reference). So the addon
introduces a new click button in the browser bar that opens a new Laverna tab
and prefills the field with the parsed content, one-click away from being
saved. This is useful because some read-it-later services can’t have access to
content behind paid-walls, for instance.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;pub&#x2F;demo-laverna-webclipper.webm&quot;&gt;See it in action here.&lt;&#x2F;a&gt;&lt;&#x2F;p&gt;
&lt;p&gt;If you’re interested in getting this addon, you can go on the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;addons.mozilla.org&#x2F;en-US&#x2F;firefox&#x2F;addon&#x2F;laverna-clipper&#x2F;&quot;&gt;official addons
for Firefox
website&lt;&#x2F;a&gt; (as
of the day of this writing, the addon has not been validated yet).&lt;&#x2F;p&gt;
&lt;h2 id=&quot;the-implementation&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#the-implementation&quot; aria-label=&quot;Anchor link for: the-implementation&quot;&gt;🔗&lt;&#x2F;a&gt;The Implementation&lt;&#x2F;h2&gt;
&lt;p&gt;This project is using &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;mozilla&#x2F;readability&quot;&gt;Readability.js&lt;&#x2F;a&gt;
for retrieving the “interesting content” of a page, and then renders it to
Markdown (since Laverna uses this format) thanks to
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;neocotic&#x2F;html.md&quot;&gt;html.md&lt;&#x2F;a&gt;. The rest is plumbing :)&lt;&#x2F;p&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;wiki.mozilla.org&#x2F;WebExtensions&quot;&gt;WebExtensions&lt;&#x2F;a&gt; are a new safe and
portable way to write addons for Web browsers. Heavily inspired from Chrome’s
Addons APIs, the main thrust is that WebExtensions should Just Work© on any Web
browser supporting them (any Chromium-based, Edge and Firefox, as of today).&lt;&#x2F;p&gt;
&lt;p&gt;This is fantastic news! … when all the browsers will support it. As of today,
this addon will only work on Firefox, and here is why.&lt;&#x2F;p&gt;
&lt;p&gt;First, programming a WebExtension is &lt;em&gt;hard&lt;&#x2F;em&gt;. I mean &lt;strong&gt;really&lt;&#x2F;strong&gt; hard; debugging
it may quickly become a nightmare, because debug messages can appear in the
developer console (for the content script, which is the “client” part of your
addon), or in a specific addon debugger console (for background scripts), or
even in the &lt;code&gt;stdout&lt;&#x2F;code&gt; logs of your browser. Some issues, like parsing errors in
your JavaScript files, may be really hard to find in this context.&lt;&#x2F;p&gt;
&lt;p&gt;Second, at some point in the addon’s workflow, you need to inject the Markdown
content into the Laverna’s text editor. Laverna uses
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;codemirror.net&#x2F;&quot;&gt;CodeMirror&lt;&#x2F;a&gt; for their rich text editor, so dumping the
content is not just as simple as setting the &lt;code&gt;value&lt;&#x2F;code&gt; property of the target
&lt;code&gt;textarea&lt;&#x2F;code&gt;. You need to retrieve the CodeMirror instance somehow and call one of its
methods. The instance is fortunately saved in the
&lt;code&gt;.CodeMirror&lt;&#x2F;code&gt; property of the textarea, so you could retrieve it, if you really
were on the client page. But browsers don’t
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;developer.chrome.com&#x2F;extensions&#x2F;content_scripts#execution-environment&quot;&gt;allow&lt;&#x2F;a&gt;
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;Add-ons&#x2F;WebExtensions&#x2F;Content_scripts#Xray_vision_in_Firefox&quot;&gt;that&lt;&#x2F;a&gt;,
unless modifying the observed page itself, which I didn’t know about when writing this addon.&lt;&#x2F;p&gt;
&lt;p&gt;That being said, Firefox gives you a (hacky) way to do so (through
&lt;code&gt;wrappedJSObject&lt;&#x2F;code&gt;), so you can indeed access the original JS DOM object of the
observed page and retrieve the CodeMirror instance, then call its &lt;code&gt;setValue&lt;&#x2F;code&gt;
method. Pfew! Chrome doesn’t have this hack, which is merely the reason why it
doesn’t work there.&lt;&#x2F;p&gt;
&lt;p&gt;Unrelated to these issues: whenever one clicks on the clipper button, the page
is parsed, which has the unfortunate side-effect of clearing the parsed content
(that’s how Readability works). One can clone the entire DOM to prevent this
issue, but this API is not (yet?) available in WebExtensions, at least under
Firefox. To work around this, the page is reloaded in the background after
you’ve clicked the button. Filthy, right?&lt;&#x2F;p&gt;
&lt;h2 id=&quot;get-in-touch&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#get-in-touch&quot; aria-label=&quot;Anchor link for: get-in-touch&quot;&gt;🔗&lt;&#x2F;a&gt;Get in touch!&lt;&#x2F;h2&gt;
&lt;p&gt;If you have any remarks or suggestions, don’t hesitate to ping me on
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&#x2F;&quot;&gt;twitter&lt;&#x2F;a&gt; or
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;framasphere.org&#x2F;people&#x2F;315a5640ead10132c4cc2a0000053625&quot;&gt;diaspora&lt;&#x2F;a&gt;; I
love any kind of feedback, since this is usually the only way to get better and
understand what matters. If you’re interested in contributing to this addon,
let’s get in touch!&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Making asm.js&#x2F;WebAssembly compilation more parallel in Firefox</title>
        <published>2016-04-22T15:00:42+00:00</published>
        <updated>2016-04-22T15:00:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/"/>
        <id>https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/making-asmjs-webassembly-compilation-more-parallel/">&lt;p&gt;In December 2015, I’ve worked on reducing startup time of asm.js programs in
Firefox by making compilation more parallel. As our
JavaScript engine, Spidermonkey, uses the same compilation pipeline for both
asm.js and WebAssembly, this also benefitted WebAssembly compilation. Now is a
good time to talk about what it meant, how it got achieved and what are the
next ideas to make it even faster.&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;&lt;h1 id=&quot;what-does-it-mean-to-make-a-program-more-parallel&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#what-does-it-mean-to-make-a-program-more-parallel&quot; aria-label=&quot;Anchor link for: what-does-it-mean-to-make-a-program-more-parallel&quot;&gt;🔗&lt;&#x2F;a&gt;What does it mean to make a program “more parallel”?&lt;&#x2F;h1&gt;
&lt;p&gt;Parallelization consists of splitting a sequential program into smaller
independent tasks, then having them run on different CPU. If your program
is using &lt;code&gt;N&lt;&#x2F;code&gt; cores, it can be up to &lt;code&gt;N&lt;&#x2F;code&gt; times faster.&lt;&#x2F;p&gt;
&lt;p&gt;Well, in theory. Let’s say you’re in a car, driving on a 100 Km long road.
You’ve already driven the first 50 Km in one hour. Let’s say your car can
have unlimited speed from now on. What is the maximal average speed you can
reach, once you get to the end of the road?&lt;&#x2F;p&gt;
&lt;p&gt;People intuitively answer “If it can go as fast as I want, so nearby lightspeed
sounds plausible”. But this is not true! In fact, if you could teleport from
your current position to the end of the road, you’d have traveled 100 Km in one
hour, so your maximal theoritical speed is 100 Km per hour. This result is a
consequence of &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Amdahl%27s_law&quot;&gt;Amdahl’s law&lt;&#x2F;a&gt;.
When we get back to our initial problem, this means you can expect a &lt;code&gt;N&lt;&#x2F;code&gt; times
speedup if you’re running your program with &lt;code&gt;N&lt;&#x2F;code&gt; cores if, and only if your
program can be &lt;strong&gt;entirely&lt;&#x2F;strong&gt; run in parallel. This is usually not the case, and
that is why most wording refers to &lt;em&gt;speedups &lt;strong&gt;up to&lt;&#x2F;strong&gt; N times faster&lt;&#x2F;em&gt;, when it
comes to parallelization.&lt;&#x2F;p&gt;
&lt;p&gt;Now, say your program is already running some portions in parallel. To make it
faster, one can identify some parts of the program that are sequential, and make
them independent so that you can run them in parallel. With respect to our car
metaphor, this means augmenting the portion of the road on which you can run at
unlimited speed.&lt;&#x2F;p&gt;
&lt;p&gt;This is exactly what we have done with parallel compilation of asm.js programs
under Firefox.&lt;&#x2F;p&gt;
&lt;h1 id=&quot;a-quick-look-at-the-asm-js-compilation-pipeline&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#a-quick-look-at-the-asm-js-compilation-pipeline&quot; aria-label=&quot;Anchor link for: a-quick-look-at-the-asm-js-compilation-pipeline&quot;&gt;🔗&lt;&#x2F;a&gt;A quick look at the asm.js compilation pipeline&lt;&#x2F;h1&gt;
&lt;p&gt;I recommend to read this &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;blog.mozilla.org&#x2F;luke&#x2F;2014&#x2F;01&#x2F;14&#x2F;asm-js-aot-compilation-and-startup-performance&#x2F;&quot;&gt;blog
post&lt;&#x2F;a&gt;.
It clearly explains the differences between JIT (Just In Time) and AOT (Ahead
Of Time) compilation, and elaborates on the different parts of the engines
involved in the compilation pipeline.&lt;&#x2F;p&gt;
&lt;p&gt;As a TL;DR, keep in mind that &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;asmjs.org&#x2F;&quot;&gt;asm.js&lt;&#x2F;a&gt; is a strictly
validated, highly optimizable, typed subset of JavaScript. Once
validated, it guarantees high performance and stability (no garbage collector
involved!). That is ensured by
mapping every single JavaScript instruction of this subset to a few CPU
instructions, if not only a single instruction. This means an asm.js program
needs to get &lt;em&gt;compiled&lt;&#x2F;em&gt; to machine code, that is, translated from JavaScript to
the language your CPU directly manipulates (like what GCC would do for a C++
program). If you haven’t heard, the results are impressive and you can run
&lt;a href=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;making-asmjs-webassembly-compilation-more-parallel&#x2F;beta.unity3d.com&#x2F;jonas&#x2F;DT2&#x2F;&quot;&gt;video&lt;&#x2F;a&gt;
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.unrealengine.com&#x2F;html5&quot;&gt;games&lt;&#x2F;a&gt; directly in your browser, without
needing to install anything. No plugins. Nothing more than your usual, everyday
browser.&lt;&#x2F;p&gt;
&lt;p&gt;Because asm.js programs can be gigantic in size (in number of functions as well
as in number of lines of code), the first compilation of the entire program is
going to take some time. Afterwards, Firefox uses a caching mechanism that
prevents the need for recompilation and almost instaneously loads the code, so
subsequent loadings matter less*****. The end user will mostly wait for the
first compilation, thus this one needs to be fast.&lt;&#x2F;p&gt;
&lt;p&gt;Before the work explained below, the pipeline for compiling a single function
(out of an asm.js module) would look like this:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;parse the function, and as we parse, emit intermediate representation (IR)
nodes for the compiler infrastructure. SpiderMonkey has several IRs,
including the MIR (middle-level IR, mostly loaded with semantic) and the LIR
(low-level IR closer to the CPU memory representation: registers, stack,
etc.). The one generated here is the MIR. All of this happens on the main
thread.&lt;&#x2F;li&gt;
&lt;li&gt;once the entire IR graph is generated for the function, optimize the MIR
graph (i.e. apply a few optimization passes). Then, generate the LIR graph
before carrying out register allocation (probably the most costly task of the
pipeline). This can be done on supplementary helper threads, as the MIR
optimization and LIR generation for a given function doesn’t depend on other
ones.&lt;&#x2F;li&gt;
&lt;li&gt;since functions can call between themselves within an asm.js module, they
need references to each other. In assembly, a reference is merely an offset
to somewhere else in memory. In this initial implementation, code generation
is carried out on the main thread, at the cost of speed but for the sake of
simplicity.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;So far, only the MIR optimization passes, register allocation and LIR
generation were done in parallel. Wouldn’t it be nice to be able to do more?&lt;&#x2F;p&gt;
&lt;p&gt;&lt;strong&gt;*&lt;&#x2F;strong&gt; There are conditions for benefitting from the caching mechanism. In
particular, the script should be loaded
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Games&#x2F;Techniques&#x2F;Async_scripts&quot;&gt;asynchronously&lt;&#x2F;a&gt;
and it should be of a consequent size.&lt;&#x2F;p&gt;
&lt;h1 id=&quot;doing-more-in-parallel&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#doing-more-in-parallel&quot; aria-label=&quot;Anchor link for: doing-more-in-parallel&quot;&gt;🔗&lt;&#x2F;a&gt;Doing more in parallel&lt;&#x2F;h1&gt;
&lt;p&gt;Our goal is to make more work in parallel: so can we take out MIR generation
from the main thread? And we can take out code generation as well?&lt;&#x2F;p&gt;
&lt;p&gt;The answer happens to be &lt;em&gt;yes&lt;&#x2F;em&gt; to both questions.&lt;&#x2F;p&gt;
&lt;p&gt;For the former, instead of emitting a MIR graph as we parse the function’s
body, we emit a small, compact, pre-order representation of the function’s
body. In short, a new IR. As work was starting on
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;webassembly&#x2F;design&quot;&gt;WebAssembly&lt;&#x2F;a&gt; (wasm) at this time, and
since asm.js semantics and wasm semantics mostly match, the IR could just be
the wasm
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;WebAssembly&#x2F;design&#x2F;blob&#x2F;master&#x2F;BinaryEncoding.md&quot;&gt;encoding&lt;&#x2F;a&gt;,
consisting of the wasm opcodes plus a few specific asm.js ones*. Then, wasm
is translated to MIR in another thread.&lt;&#x2F;p&gt;
&lt;p&gt;Now, instead of parsing and generating MIR in a single pass, we would now parse
and generate wasm IR in one pass, and generate the MIR out of the wasm IR in
another pass. The wasm IR is very compact and much cheaper to generate than a
full MIR graph, because generating a MIR graph needs some algorithmic work,
including the creation of Phi nodes (join values after any form of branching).
As a result, it is expected that compilation time won’t suffer.  This was a
large refactoring: taking every single asm.js instructions, and encoding them
in a compact way and later decode these into the equivalent MIR nodes.&lt;&#x2F;p&gt;
&lt;p&gt;For the second part, could we generate code on other threads? One structure in
the code base, the &lt;em&gt;MacroAssembler&lt;&#x2F;em&gt;, is used to generate all the code and it
contains all necessary metadata about offsets. By adding more metadata there to
abstract internal calls &lt;strong&gt;**&lt;&#x2F;strong&gt;, we can describe the new scheme in terms of a
classic functional &lt;code&gt;map&lt;&#x2F;code&gt;&#x2F;&lt;code&gt;reduce&lt;&#x2F;code&gt;:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;the wasm IR is sent to a thread, which will return a MacroAssembler. That
is a &lt;code&gt;map&lt;&#x2F;code&gt; operation, transforming an array of wasm IR into an array of
MacroAssemblers.&lt;&#x2F;li&gt;
&lt;li&gt;When a thread is done compiling, we merge its MacroAssembler into one big
MacroAssembler. Most of the merge consists in taking all the offset metadata
in the thread MacroAssembler, fixing up all the offsets, and concatenate the
two generated code buffers. This is equivalent to a &lt;code&gt;reduce&lt;&#x2F;code&gt; operation,
merging each MacroAssembler within the module’s one.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;At the end of the compilation of the entire module, there is still some light
work to be done: offsets of internal calls need to be translated to their
actual locations. All this work has been done in &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;bugzilla.mozilla.org&#x2F;show_bug.cgi?id=1181612&quot;&gt;this bugzilla
bug&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;strong&gt;*&lt;&#x2F;strong&gt; In fact, at the time when this was being done, we used a different
superset of wasm. Since then, work has been done so that our asm.js frontend is
really just another wasm emitter.&lt;&#x2F;p&gt;
&lt;p&gt;**** ** referencing functions by their appearance order index in the module,
rather than an offset to the actual start of the function. This order is indeed
stable, from a function to the other.&lt;&#x2F;p&gt;
&lt;h1 id=&quot;results&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#results&quot; aria-label=&quot;Anchor link for: results&quot;&gt;🔗&lt;&#x2F;a&gt;Results&lt;&#x2F;h1&gt;
&lt;p&gt;Benchmarking has been done on a Linux x64 machine with 8 cores clocked at 4.2
Ghz.&lt;&#x2F;p&gt;
&lt;p&gt;First, compilation times of a few asm.js massive games:&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;beta.unity3d.com&#x2F;jonas&#x2F;DT2&#x2F;&quot;&gt;DeadTrigger2&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;beta.unity3d.com&#x2F;jonas&#x2F;AngryBots&#x2F;&quot;&gt;AngryBots&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;lukewagner&#x2F;PlatformerGamePacked&quot;&gt;Platformer game&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;www.unrealengine.com&#x2F;html5&quot;&gt;Tappy Chicken&lt;&#x2F;a&gt;&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;The X scale is the compilation time in seconds, so lower is better. Each value
point is the best one of three runs. For the new scheme, the corresponding
relative speedup (in percentage) has been added:&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;making-asmjs-webassembly-compilation-more-parallel&#x2F;2016-04-22_parallelization-times.png&quot; alt=&quot;Compilation times of various benchmarks&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;For all games, compilation is much faster with the new parallelization scheme.&lt;&#x2F;p&gt;
&lt;p&gt;Now, let’s go a bit deeper. The Linux CLI tool &lt;code&gt;perf&lt;&#x2F;code&gt; has a &lt;code&gt;stat&lt;&#x2F;code&gt; command
that gives you an average of the number of utilized CPUs during the program
execution. This is a great measure of threading efficiency: the more a CPU is
utilized, the more it is not idle, waiting for other results to come, and thus
useful. For a constant task execution time, the more utilized CPUs, the more
likely the program will execute quickly.&lt;&#x2F;p&gt;
&lt;p&gt;The X scale is the number of utilized CPUs, according to the &lt;code&gt;perf stat&lt;&#x2F;code&gt;
command, so higher is better. Again, each value point is the best one of three
runs.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;making-asmjs-webassembly-compilation-more-parallel&#x2F;2016-04-22_parallelization-cpu-utilized.png&quot; alt=&quot;CPU utilized on DeadTrigger2&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;With the older scheme, the number of utilized CPUs quickly rises up from 1 to 4
cores, then more slowly from 5 cores and beyond. Intuitively, this means that
with 8 cores, we almost reached the theoritical limit of the portion of the
program that can be made parallel (not considering the overhead introduced by
parallelization or altering the scheme).&lt;&#x2F;p&gt;
&lt;p&gt;But with the newer scheme, we get much more CPU usage even after 6 cores! Then
it slows down a bit, although it is still more significant than the slow rise
of the older scheme. So it is likely that with even more threads, we could have
even better speedups than the one mentioned beforehand. In fact, we have moved
the theoritical limit mentioned above a bit further: we have expanded the
portion of the program that can be made parallel. Or to keep on using the
initial car&#x2F;road metaphor, we’ve shortened the constant speed portion of the
road to the benefit of the unlimited speed portion of the road, resulting in a
shorter trip overall.&lt;&#x2F;p&gt;
&lt;h1 id=&quot;future-steps&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#future-steps&quot; aria-label=&quot;Anchor link for: future-steps&quot;&gt;🔗&lt;&#x2F;a&gt;Future steps&lt;&#x2F;h1&gt;
&lt;p&gt;Despite these improvements, compilation time can still be a pain, especially on
mobile. This is mostly due to the fact that we’re running a whole multi-million
line codebase through the backend of a compiler to generate optimized code.
Following this work, the next bottleneck during the compilation process is
parsing, which matters for asm.js in particular, which source is plain text.
Decoding WebAssembly is an order of magnitude faster though, and it can be made
even faster. Moreover, we have even more load-time optimizations coming down
the pipeline!&lt;&#x2F;p&gt;
&lt;p&gt;In the meanwhile, we keep on improving the WebAssembly backend. Keep track of
our progress on &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;bugzilla.mozilla.org&#x2F;show_bug.cgi?id=1188259&quot;&gt;bug
1188259&lt;&#x2F;a&gt;!&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Previous writings about Mozilla work</title>
        <published>2016-03-09T18:00:42+00:00</published>
        <updated>2016-03-09T18:00:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/previous-writing-about-mozilla-work/"/>
        <id>https://bouvier.cc/tech/previous-writing-about-mozilla-work/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/previous-writing-about-mozilla-work/">&lt;p&gt;I am currently a compiler engineer at Mozilla corporation, the company making
the Firefox browser among else. Our JavaScript virtual machine, Spidermonkey,
is split in several tiers, including an highly optimizing Just-In-Time (JIT)
compiler able to compile JavaScript to assembly at runtime. My previous work
has involved efficiently compiling Float32 arithmetic to hardware instructions
and implement a new SIMD API for the Web.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;about-float32-optimizations&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#about-float32-optimizations&quot; aria-label=&quot;Anchor link for: about-float32-optimizations&quot;&gt;🔗&lt;&#x2F;a&gt;About Float32 optimizations&lt;&#x2F;h2&gt;
&lt;p&gt;The full blog post is
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;blog.mozilla.org&#x2F;javascript&#x2F;2013&#x2F;11&#x2F;07&#x2F;efficient-float32-arithmetic-in-javascript&#x2F;&quot;&gt;there&lt;&#x2F;a&gt;.
It has been written in November 2013.&lt;&#x2F;p&gt;
&lt;p&gt;The main idea is that if you have float32 inputs to an operation; and you cast
them to doubles; and you apply an arithmetic operation to these inputs; and you
cast the result back to a float32, then you’d have the same result as if you
did the entire computation with float32 values and operations.&lt;&#x2F;p&gt;
&lt;p&gt;So we’ve introduced an operation in JavaScript that converts a Number to its
closest float32 IEEE754 representation: &lt;code&gt;Math.fround&lt;&#x2F;code&gt;. Said differently, the
above equivalence says that:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;function&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; f&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt;x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;    return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;function&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; g&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt;x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;    var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; xf&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Math&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;fround&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;    var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; yf&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Math&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;fround&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;    return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; Math&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;fround&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;xf&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; yf&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;}&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;&#x2F;&#x2F; For all x, y that can be represented exactly as float32:&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;assert&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;f&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; ===&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; g&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;x&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; y&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;))&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Yes, &lt;code&gt;===&lt;&#x2F;code&gt;. The same &lt;code&gt;===&lt;&#x2F;code&gt; you’ve been told &lt;strong&gt;not&lt;&#x2F;strong&gt; to use for floating-point
Numbers. But here, we have &lt;em&gt;bitwise&lt;&#x2F;em&gt; equality, so we can use strict equality*.&lt;&#x2F;p&gt;
&lt;p&gt;Processors have special instructions for carrying out float32 arithmetic, which
have higher throughput than the equivalent double ones. With this result in
mind, we could add a pass that would spot opportunities where the computations
are equivalent (thanks to &lt;code&gt;Math.fround&lt;&#x2F;code&gt; hints) and emit float32 instructions
instead of double instructions. This sped up a some numerical applications and
games engines by a few points.&lt;&#x2F;p&gt;
&lt;p&gt;* a careful reader would object that this is wrong for &lt;code&gt;x = y = NaN&lt;&#x2F;code&gt;, which
I’ve put away for the sake of simplicity.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;about-simd-js&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#about-simd-js&quot; aria-label=&quot;Anchor link for: about-simd-js&quot;&gt;🔗&lt;&#x2F;a&gt;About SIMD.js&lt;&#x2F;h2&gt;
&lt;p&gt;The full blog post is
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;blog.mozilla.org&#x2F;javascript&#x2F;2015&#x2F;03&#x2F;10&#x2F;state-of-simd-js-performance-in-firefox&#x2F;&quot;&gt;there&lt;&#x2F;a&gt;.
It has been written in March 2015.&lt;&#x2F;p&gt;
&lt;p&gt;Nowadays, processors have instructions sets that allow them to execute several
simple arithmetic operations at once. For instance, let’s say you have two
arrays of integers and you want to add each element to the corresponding one in
the other array. If both arrays have size &lt;code&gt;N&lt;&#x2F;code&gt;, this means you’ll have to carry
out &lt;code&gt;N&lt;&#x2F;code&gt; scalar additions. But processors can actually group these into bundles
of several additions, with SIMD; for the case of 32-bits wide integers, on most
modern processors, you need at most &lt;code&gt;Math.ceil(N &#x2F; 4)&lt;&#x2F;code&gt; instructions. The blog
post details what SIMD.js is and what bottlenecks we hit during implementation.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;conclusion&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#conclusion&quot; aria-label=&quot;Anchor link for: conclusion&quot;&gt;🔗&lt;&#x2F;a&gt;Conclusion&lt;&#x2F;h2&gt;
&lt;p&gt;This was a small reminder about previously written blog posts. If you’re into
JavaScript, compilers or low-level optimization, I can only recommend you to go
read the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;blog.mozilla.org&#x2F;javascript&#x2F;&quot;&gt;Mozilla’s JavaScript blog&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>A Tale Of Linux On The Desktop</title>
        <published>2016-01-27T21:00:42+00:00</published>
        <updated>2016-01-27T21:00:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/installing-linux/"/>
        <id>https://bouvier.cc/tech/installing-linux/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/installing-linux/">&lt;p&gt;So I’ve received a new desktop machine at home, on which only
Windows 10 was installed. I’ve decided to install Linux, for my day-to-day
hacking.  Unfortunately, when I’ve plugged the Ubuntu (nobody’s perfect) USB
drive to my computer, I had the surprise to see a black screen showing up just
after booting, and nothing else. Here are some notes taken during the
installation of Linux on this machine.&lt;&#x2F;p&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;installing-linux&#x2F;2016-01-27_installing-linux-twitter.png&quot; alt=&quot;Twitter fame&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;h2 id=&quot;graphics-not-working&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#graphics-not-working&quot; aria-label=&quot;Anchor link for: graphics-not-working&quot;&gt;🔗&lt;&#x2F;a&gt;Graphics not working&lt;&#x2F;h2&gt;
&lt;p&gt;Graphics don’t work, but we can still install Ubuntu in non-graphic mode
with an &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;www.ubuntu.com&#x2F;download&#x2F;alternative-downloads&quot;&gt;alternate install
image&lt;&#x2F;a&gt;, namely the
network installer. Let’s try that.&lt;&#x2F;p&gt;
&lt;p&gt;The machine has a Nvidia graphics card, so the open-source Nouveau graphics
driver is used by default. Suspecting proprietary drivers might solve the
problem, I decide to download them. Then, to my greatest surprise, I find out
network isn’t working, be it the wireless network or Ye Olde Ethernet network.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;ethernet-not-working&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#ethernet-not-working&quot; aria-label=&quot;Anchor link for: ethernet-not-working&quot;&gt;🔗&lt;&#x2F;a&gt;Ethernet not working&lt;&#x2F;h2&gt;
&lt;p&gt;The computer vendor’s website says the network card is an Atheros Killer 2400.
Looking that up on the web with my favorite &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;duckduckgo.com&quot;&gt;search
engine&lt;&#x2F;a&gt;, this &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;askubuntu.com&#x2F;questions&#x2F;670347&#x2F;is-there-any-way-to-install-atheros-e2400-drivers&quot;&gt;StackOverflow
page&lt;&#x2F;a&gt;
showed up. The solution to make ethernet work at the end of the installation is
the following:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;modprobe alx&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;echo 1969 e0a1 &amp;gt; &#x2F;sys&#x2F;bus&#x2F;pci&#x2F;drivers&#x2F;alx&#x2F;new_id&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;This enables the module &lt;code&gt;alx&lt;&#x2F;code&gt; and registers the device to the module. Now
ethernet is working. Fiuu. Let’s keep moving.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;install-all-the-drivers&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#install-all-the-drivers&quot; aria-label=&quot;Anchor link for: install-all-the-drivers&quot;&gt;🔗&lt;&#x2F;a&gt;Install All The Drivers!&lt;&#x2F;h2&gt;
&lt;p&gt;The &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;help.ubuntu.com&#x2F;community&#x2F;BinaryDriverHowto&#x2F;Nvidia&quot;&gt;ubuntu’s nvidia troubleshooting
page&lt;&#x2F;a&gt; gives you a
nice tool that show you what drivers are adapted to your hardware:
&lt;code&gt;ubuntu-drivers&lt;&#x2F;code&gt;. Here’s an output example given by this command:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;$ sudo ubuntu-drivers devices&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;== &#x2F;sys&#x2F;devices&#x2F;pci0000:00&#x2F;0000:00:01.0&#x2F;0000:01:00.0 ==&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;vendor   : NVIDIA Corporation&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;modalias : pci:v000010DEd00000FE9sv0000106Bsd00000130bc03sc00i00&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;driver   : xserver-xorg-video-nouveau - distro free builtin&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;driver   : nvidia-340-updates - distro non-free&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;driver   : nvidia-340 - distro non-free&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;driver   : nvidia-352 - distro non-free recommended&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;driver   : nvidia-352-updates - distro non-free&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;So this tells me what the recommanded driver for my nvidia card is. For bonus
credit, it also shows me a recommended driver for the builtin WiFi card. Once
I’ve installed it, it has been working like a charm!&lt;&#x2F;p&gt;
&lt;h2 id=&quot;moar-graphics-settings&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#moar-graphics-settings&quot; aria-label=&quot;Anchor link for: moar-graphics-settings&quot;&gt;🔗&lt;&#x2F;a&gt;Moar graphics settings&lt;&#x2F;h2&gt;
&lt;p&gt;After the nvidia driver has been installed, a
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;help.ubuntu.com&#x2F;community&#x2F;BinaryDriverHowto&#x2F;Nvidia#Screen_Blanks.2FMonitor_Turns_Off&quot;&gt;tweak&lt;&#x2F;a&gt;
was needed to ensure the driver outputs video on the DVI port instead of the
VGA port. Beforehand, the &lt;code&gt;xorg.conf&lt;&#x2F;code&gt; file was generated thanks to the
&lt;code&gt;nvidia-xsettings&lt;&#x2F;code&gt; command.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;final-boss-uefi&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#final-boss-uefi&quot; aria-label=&quot;Anchor link for: final-boss-uefi&quot;&gt;🔗&lt;&#x2F;a&gt;Final boss: UEFI&lt;&#x2F;h2&gt;
&lt;p&gt;Now that all minimal drivers are correctly installed and configured, let’s try
to reboot. Although I’ve installed the grub bootloader on the main disk, I
can’t find a way to access it, even by trying all the lines in the boot menu.&lt;&#x2F;p&gt;
&lt;p&gt;The reason is that I’ve boot up the USB drive in legacy mode, which is not UEFI
mode. As a matter of fact, Ubuntu has been installed in legacy mode but the
machine is booting with UEFI, so Linux can’t be seen from the bootloader.&lt;&#x2F;p&gt;
&lt;p&gt;Fortunately, this is Linux, and everything that has been done can be undone. If
you can find a way to boot with UEFI &lt;strong&gt;and&lt;&#x2F;strong&gt; log in under Linux (maybe
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;help.ubuntu.com&#x2F;community&#x2F;BasicChroot&quot;&gt;chrooting&lt;&#x2F;a&gt;), then you can
follow &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;help.ubuntu.com&#x2F;community&#x2F;UEFI#Converting_Ubuntu_into_UEFI_mode&quot;&gt;this
procedure&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
&lt;p&gt;I couldn’t boot in UEFI mode, but I could run the
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;sourceforge.net&#x2F;p&#x2F;boot-repair-cd&#x2F;home&#x2F;Home&#x2F;&quot;&gt;Boot-Repair-Disk&lt;&#x2F;a&gt; on USB
as UEFI, so I’ve followed the procedure there and converted my installation
into UEFI mode.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;linux-after-all&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#linux-after-all&quot; aria-label=&quot;Anchor link for: linux-after-all&quot;&gt;🔗&lt;&#x2F;a&gt;Linux After All&lt;&#x2F;h2&gt;
&lt;p&gt;And here we are, with Linux working out of the box^W^W^W^W. So yes, 2016 will
be another year of Linux on the desktop for me; but it still feels unlikely for
a newcomer to install Ubuntu and see it Just Work on any given machine, so
maybe 2017 will be the year of Linux on the desktop, but not 2016.&lt;&#x2F;p&gt;
&lt;p&gt;Many thanks to &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;etnbrd&quot;&gt;@etnbrd&lt;&#x2F;a&gt;,
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;martiusweb&#x2F;&quot;&gt;@martiusweb&lt;&#x2F;a&gt; and
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;padenot&quot;&gt;@padenot&lt;&#x2F;a&gt; for the detailed explanations about
some Linux specifics and for advices.&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>CozyCloud : migrer son instance 2, le retour</title>
        <published>2015-11-19T14:00:42+00:00</published>
        <updated>2015-11-19T14:00:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/cozycloud-migrer-son-instance-2-le-retour/"/>
        <id>https://bouvier.cc/tech/cozycloud-migrer-son-instance-2-le-retour/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/cozycloud-migrer-son-instance-2-le-retour/">&lt;p&gt;Si vous vous rappelez bien, j’ai &lt;a href=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;cozycloud-migrer-son-instance-2-le-retour&#x2F;%7Bfilename%7D2015-04-21_cozycloud-migrer-son-instance.md&quot;&gt;déjà écrit un
article&lt;&#x2F;a&gt;
pour expliquer comment migrer son instance &lt;a href=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;cozycloud-migrer-son-instance-2-le-retour&#x2F;CozyCloud&quot;&gt;https:&#x2F;&#x2F;cozy.io&lt;&#x2F;a&gt; d’un
endroit vers un autre. Ces derniers jours, je me suis rendu compte que mon
serveur personnel était sous-utilisé et qu’il me serait possible d’en prendre
un plus léger et moins cher, d’où la nécessité d’une nouvelle migration d’un
serveur à un autre.&lt;&#x2F;p&gt;
&lt;p&gt;Edit (10 février 2016) : j’ai ajouté la copie des répertoires permanents à
cette procédure, sans avoir pu tester. Vos retours sur cette partie sont les
bienvenus !&lt;&#x2F;p&gt;
&lt;p&gt;Les étapes sont les suivantes :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;Récupérer la base de l’ancien Cozy.&lt;&#x2F;li&gt;
&lt;li&gt;Recopier le répertoire de données permanentes des apps.&lt;&#x2F;li&gt;
&lt;li&gt;La mettre en place dans le nouveau Cozy.&lt;&#x2F;li&gt;
&lt;li&gt;Réinstaller les applications.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h1 id=&quot;recuperer-base-ancien-cozy&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#recuperer-base-ancien-cozy&quot; aria-label=&quot;Anchor link for: recuperer-base-ancien-cozy&quot;&gt;🔗&lt;&#x2F;a&gt;Récupérer la base de l’ancien Cozy&lt;&#x2F;h1&gt;
&lt;p&gt;Il s’agit d’un fichier qui contient toutes les données relatives à votre Cozy,
que ce soit des données internes (par exemple, quelles applications
installées) ou propres aux applications (par exemple, vos comptes en banques).&lt;&#x2F;p&gt;
&lt;p&gt;Si votre Cozy est hébergé par CozyCloud (sur
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;cozycloud.cc&quot;&gt;cozycloud.cc&lt;&#x2F;a&gt; donc), il est possible de demander
l’export de la base de données CouchDB à l’équipe Cozy. Problème réglé.&lt;&#x2F;p&gt;
&lt;p&gt;Si vous êtes auto-hébergé, il va falloir mettre les mains dans le cambouis :&lt;&#x2F;p&gt;
&lt;p&gt;D’abord, réduire la taille de la base de données, c’est-à-dire effectuer un
compactage de la base de données. C’est parti, depuis l’ancien serveur :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # On éteint Cozy pour éviter l&amp;#39;apparition de nouvelles données&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo service supervisor stop&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # On fait une sauvegarde de la base, en cas de pépin&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    cd &#x2F;var&#x2F;lib&#x2F;couchdb&#x2F;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo cp cozy.couch cozy.couch.backup&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # On compacte la base&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo cozy-monitor compact&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Ensuite, récupérer la base elle-même. Celle-ci est contenue dans le répertoire
&lt;code&gt;&#x2F;var&#x2F;lib&#x2F;couchdb&#x2F;&lt;&#x2F;code&gt; et porte le nom de &lt;code&gt;cozy.couch&lt;&#x2F;code&gt;, sauf si vous l’avez
modifié vous-même.&lt;&#x2F;p&gt;
&lt;h4 id=&quot;l-ancien-cozy-etait-installe-par-le-package-debian&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#l-ancien-cozy-etait-installe-par-le-package-debian&quot; aria-label=&quot;Anchor link for: l-ancien-cozy-etait-installe-par-le-package-debian&quot;&gt;🔗&lt;&#x2F;a&gt;L’ancien Cozy était installé par le package Debian&lt;&#x2F;h4&gt;
&lt;p&gt;Si vous avez installé votre serveur avec le package Debian ou Ubuntu, il est
possible d’utiliser l’outil &lt;code&gt;scp&lt;&#x2F;code&gt; pour effectuer cette tâche, en vous
connectant depuis le nouveau serveur vers l’ancien. Si vous utilisez cette
méthode, prenez soin de nommer la version sur le nouveau serveur
&lt;code&gt;cozy.couch.new&lt;&#x2F;code&gt;, c’est important pour la suite.&lt;&#x2F;p&gt;
&lt;h4 id=&quot;l-ancien-cozy-etait-installe-avec-docker&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#l-ancien-cozy-etait-installe-avec-docker&quot; aria-label=&quot;Anchor link for: l-ancien-cozy-etait-installe-avec-docker&quot;&gt;🔗&lt;&#x2F;a&gt;L’ancien Cozy était installé avec Docker&lt;&#x2F;h4&gt;
&lt;p&gt;Si comme moi vous avez préféré jouer avec le feu et utiliser Docker pour
installer votre cozy, il va falloir ruser un peu. Personnellement, j’ai choisi
la méthode &lt;em&gt;malpropre&lt;&#x2F;em&gt;, à savoir exposer la base de données sur le serveur web
pendant un court instant et utiliser &lt;code&gt;wget&lt;&#x2F;code&gt; sur le nouveau cozy. Il est
sûrement possible d’extraire le fichier depuis le Docker vers l’hôte, mais
j’avoue ne pas avoir cherché ; si quelqu’un connaît une méthode, je suis
preneur ! Pour ma méthode « malpropre », il faut modifier le fichier de
configuration de nginx pour donner accès au fichier :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Copier la base vers &#x2F;var&#x2F;www&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo cp &#x2F;var&#x2F;lib&#x2F;couchdb&#x2F;cozy.couch &#x2F;var&#x2F;www&#x2F;cozy.couch.new&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Donner les droits en lecture à nginx&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo chown www-data:www-data -R &#x2F;var&#x2F;www&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Editer le fichier de config de nginx avec le meilleur éditeur du monde&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo vim &#x2F;etc&#x2F;nginx&#x2F;sites-available&#x2F;cozy.conf&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;J’ai remplacé le bloc &lt;code&gt;location&#x2F; { proxy_set_header ...&lt;&#x2F;code&gt; par le suivant :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    location &#x2F; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        root &#x2F;var&#x2F;www;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        try_files $uri $uri&#x2F; &#x2F;index.html;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Toujours depuis l’ancien serveur, bien penser à relancer nginx :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo service nginx restart&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Ensuite, depuis le nouveau serveur, sur lequel je suppose que vous avez
déjà installé le paquet &lt;code&gt;cozy&lt;&#x2F;code&gt; et qu’il tourne correctement :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    cd &#x2F;var&#x2F;lib&#x2F;couchdb&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Stopper supervisor va arrêter tout cozy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo service supervisor stop&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo service couchdb stop&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo wget https:&#x2F;&#x2F;monanciencozy.tld&#x2F;cozy.couch.new&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h1 id=&quot;recopier-les-donnees-permanentes-des-applications&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#recopier-les-donnees-permanentes-des-applications&quot; aria-label=&quot;Anchor link for: recopier-les-donnees-permanentes-des-applications&quot;&gt;🔗&lt;&#x2F;a&gt;Recopier les données permanentes des applications&lt;&#x2F;h1&gt;
&lt;p&gt;Cela a été rajouté depuis l’écriture initiale de ce blog post, il est donc
probable que cela ne fonctionne pas, dans quel cas contactez-moi svp !&lt;&#x2F;p&gt;
&lt;p&gt;Depuis des versions récentes de la plateforme, Cozy autorise les applications à
avoir un répertoire de données permanentes, qui sont conservées même si
l’application a été désinstallée. C’est très pratique pour porter facilement
des applications qui utilisent des fichiers comme mémoire vers Cozy !&lt;&#x2F;p&gt;
&lt;p&gt;Pour porter ce répertoire, voici la procédure à effectuer :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;copier le répertoire sur la machine précédente et le mettre dans un zip, par
exemple:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  # depuis la machine qui héberge l&amp;#39;ancien cozy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  cd &#x2F;usr&#x2F;local&#x2F;var&#x2F;cozy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  zip &#x2F;tmp&#x2F;usr-local-var-cozy.zip -r .&#x2F;*&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;li&gt;
&lt;li&gt;
&lt;p&gt;transférer le zip vers le nouveau serveur (avec votre méthode préférée : FTP,
Web, scp, etc.). Par exemple, avec &lt;code&gt;scp&lt;&#x2F;code&gt;:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  # depuis la nouvelle machine&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  scp user@ancienne-machine:&#x2F;tmp&#x2F;usr-local-var-cozy.zip &#x2F;tmp&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;li&gt;
&lt;li&gt;
&lt;p&gt;remplacer le répertoire sur la nouvelle machine:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  # depuis la nouvelle machine&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  mkdir -p &#x2F;usr&#x2F;local&#x2F;var&#x2F;cozy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  cd &#x2F;usr&#x2F;local&#x2F;var&#x2F;cozy &amp;amp;&amp;amp; zip backup.zip -r .&#x2F;*&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;  unzip &#x2F;tmp&#x2F;usr-local-var-cozy.zip&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h1 id=&quot;mettre-en-place-base-nouveau-cozy&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#mettre-en-place-base-nouveau-cozy&quot; aria-label=&quot;Anchor link for: mettre-en-place-base-nouveau-cozy&quot;&gt;🔗&lt;&#x2F;a&gt;Mettre en place la base dans le nouveau cozy&lt;&#x2F;h1&gt;
&lt;p&gt;Je suppose que vous avez déjà récupéré la base d’une manière ou d’une autre, et
que celle-ci est déjà présente dans &lt;code&gt;&#x2F;var&#x2F;lib&#x2F;couchdb&lt;&#x2F;code&gt;, sous le nom
&lt;code&gt;cozy.couch.new&lt;&#x2F;code&gt;. Depuis le nouveau serveur, effectuez les commandes suivantes :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        cd &#x2F;var&#x2F;lib&#x2F;couchdb&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        # On arrête Cozy (via supervisor) et couchdb&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        sudo service supervisor stop&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        sudo service couchdb stop&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        # Backup de l&amp;#39;ancienne base&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        sudo mv cozy.couch cozy.couch.old&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        sudo mv cozy.couch.new cozy.couch&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        sudo chown -R couchdb:couchdb .&#x2F;cozy.couch&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        # On relance le tout&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        sudo service couchdb start&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;        sudo service supervisor start&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Et voilà ! Après cela, il va falloir attendre quelques minutes que le
contrôleur Cozy relance les applications.&lt;&#x2F;p&gt;
&lt;h1 id=&quot;recuperer-applications-manquantes&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#recuperer-applications-manquantes&quot; aria-label=&quot;Anchor link for: recuperer-applications-manquantes&quot;&gt;🔗&lt;&#x2F;a&gt;Réinstaller les applications manquantes&lt;&#x2F;h1&gt;
&lt;p&gt;L’import de la base comprend les données internes à Cozy, notamment les
informations sur les applications installées. Comme on vient d’importer une
base qui vient d’un autre cozy où des applications sont installées, le nouveau
cozy va penser que certaines applications sont présentes sur le disque, alors
qu’elles ne le sont pas. Il est nécessaire d’effectuer une petite réparation
ici, au niveau de la pile logicielle cozy et des applications installées.
Heureusement, l’équipe a pensé à ça et nous a fourni une commande qui permet de
réinstaller les applications utilisateurs. C’est parti, depuis le nouveau
serveur :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Mettre à jour cozy-monitor (utile pour les image préinstallées)&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo npm install -g cozy-monitor&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Mettre à jour l&amp;#39;ensemble de la pile Cozy pour commencer&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo cozy-monitor update-all-cozy-stack&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    # Réinstaller les applications manquantes&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo cozy-monitor reinstall-missing-app&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;J’ai eu plusieurs erreurs à ce moment-là, souvent liées à des applications qui
n’étaient plus trouvées sur github, ou des erreurs réseaux. En général,
relancer la commande fonctionne, sinon j’ai purement et simplement désinstallé
l’application en question. Par exemple, si l’application plantée était
&lt;a href=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;cozycloud-migrer-son-instance-2-le-retour&#x2F;%7Bfilename%7D2015-11-16_kresus-version-060.md&quot;&gt;Kresus&lt;&#x2F;a&gt;, j’ai simplement effectué&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;    sudo cozy-monitor uninstall kresus&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Ce n’est pas un problème, car la désinstallation d’une application n’implique
pas la désinstallation des données. Ensuite, il est possible de réinstaller les
applications depuis l’interface du site web.&lt;&#x2F;p&gt;
&lt;p&gt;Pour terminer, il faut mettre à jour les permissions des dossiers contenant les
données persistantes :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;   cd &#x2F;usr&#x2F;local&#x2F;var&#x2F;cozy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;   # On change les permissions pour chaque dossier. Note, penser à changer&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;   # le nom de l&amp;#39;application à chaque fois&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;   sudo chown -R cozy-nomApp nomApp&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h1 id=&quot;conclusion&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#conclusion&quot; aria-label=&quot;Anchor link for: conclusion&quot;&gt;🔗&lt;&#x2F;a&gt;Conclusion&lt;&#x2F;h1&gt;
&lt;p&gt;Notons que la migration d’un serveur à l’autre est désormais beaucoup plus
facile qu’auparavant ! Pas besoin de rafistoler les données directement dans la
base, et il existe une commande pour réinstaller les applications
automatiquement. Bien joué, l’équipe Cozy !&lt;&#x2F;p&gt;
&lt;p&gt;J’espère que cette procédure aura marché pour vous, j’ai écrit ces notes
rapidement et de mémoire. Si j’ai oublié quoi que ce soit, n’hésitez pas à me
contacter sur &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&quot;&gt;twitter&lt;&#x2F;a&gt;,
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;framasphere.org&#x2F;u&#x2F;bnjbvr&quot;&gt;diaspora&lt;&#x2F;a&gt; ou sur irc (mon nick est
&lt;code&gt;bnjbvr&lt;&#x2F;code&gt;). Si ça a marché et que vous avez apprécié, n’hésitez pas à me le dire
également. ;-)&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Des idées folles pour faciliter le développement sous Cozy</title>
        <published>2015-07-07T18:37:42+00:00</published>
        <updated>2015-07-07T18:37:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/des-idees-folles-pour-faciliter-le-developpement-sous-cozy/"/>
        <id>https://bouvier.cc/tech/des-idees-folles-pour-faciliter-le-developpement-sous-cozy/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/des-idees-folles-pour-faciliter-le-developpement-sous-cozy/">&lt;p&gt;Récemment, sur le forum de CozyCloud,
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;Clochix&quot;&gt;Clochix&lt;&#x2F;a&gt;
(employé de l’entreprise CC), demandait comment l’on pourrait faciliter
le développement d’applications pour Cozy. Bonne initiative ! J’ai
essayé de structurer un peu mes idées dans le forum d’abord, mais comme
ça devenait beaucoup trop long, j’ai préféré écrire ce post, vu que j’ai
beaucoup de commentaires :)&lt;&#x2F;p&gt;
&lt;p&gt;Parmi ces commentaires, il y a beaucoup d’idées saugrenues, sûrement
difficiles à implémenter, mais je fais le pari que si je les écris ici,
elles pourront inspirer les lecteurs, qui crééeront leurs propres idées
à partir de celles-ci, ou en les fusionnant, etc. Les idées
appartiennent à tout le monde !&lt;&#x2F;p&gt;
&lt;h3 id=&quot;dustatique&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#dustatique&quot; aria-label=&quot;Anchor link for: dustatique&quot;&gt;🔗&lt;&#x2F;a&gt;Du statique&lt;&#x2F;h3&gt;
&lt;p&gt;Tout d’abord, il me semble primordial de permettre de développeur sur
son cozy depuis son cozy. Pour commencer, on pourrait partir avec &lt;strong&gt;un
site statique et un éditeur de texte&lt;&#x2F;strong&gt;. Imaginons un instant un
répertoire spécial &lt;code&gt;www&lt;&#x2F;code&gt; par exemple, accessible depuis Files. Une
nouvelle application ferait office de serveur statique, et pourrait
servir tous les fichiers présents dans ce répertoire &lt;code&gt;www&lt;&#x2F;code&gt; sur une
adresse publique. Ensuite, une autre application saurait ouvrir tous les
fichiers textes importés dans Files et les éditer directement. Avec ces
deux choses, on a la possibilité de &lt;strong&gt;créer son site statique et le
modifier directement depuis son cozy&lt;&#x2F;strong&gt;. Cela me paraît une bonne
première étape !&lt;&#x2F;p&gt;
&lt;p&gt;L’intégration d’outils comme &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;webmaker.org&quot;&gt;WebMaker&lt;&#x2F;a&gt; pourrait
même permettre de créer des sites facilement pour les débutants, et de
propager l’apprentissage des technologies web auprès du grand public.&lt;&#x2F;p&gt;
&lt;h3 id=&quot;desoutilsdedveloppement&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#desoutilsdedveloppement&quot; aria-label=&quot;Anchor link for: desoutilsdedveloppement&quot;&gt;🔗&lt;&#x2F;a&gt;Des outils de développement&lt;&#x2F;h3&gt;
&lt;p&gt;Ensuite, &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;forum.cozy.io&#x2F;t&#x2F;comment-pouvons-nous-vous-aider-a-bidouiller-cozy&#x2F;880&#x2F;2?u=ben&quot;&gt;comme le dit
@Clochix&lt;&#x2F;a&gt;,
&lt;strong&gt;réduire le nombre d’outils à utiliser pour le développement&lt;&#x2F;strong&gt;. A
l’heure actuelle, quand je veux modifier
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;kresus&quot;&gt;Kresus&lt;&#x2F;a&gt; (mon application de finances
personnelles hébergée sur cozy) depuis ma machine de tous les jours, je
dois :&lt;&#x2F;p&gt;
&lt;ol&gt;
&lt;li&gt;lancer la VM de développement dans le bon répertoire, ce qui prend
facilement 2 minutes (vérification de l’image, timeouts
malheureusement réguliers sur le SSH, etc.)&lt;&#x2F;li&gt;
&lt;li&gt;aller dans le répertoire de Kresus depuis la machine hôte&lt;&#x2F;li&gt;
&lt;li&gt;lancer &lt;code&gt;cozy-dev deploy 9876&lt;&#x2F;code&gt; (parce que le cozy de dev ne se
souvient pas que j’ai pu faire un deploy auparavant… peut-être que
ça vaut un bug ?). C’est instantané, mais c’est redondant de le
faire à chaque fois !&lt;&#x2F;li&gt;
&lt;li&gt;lancer kresus&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;p&gt;Si jamais je veux modifier kresus depuis une nouvelle machine, je dois
en plus de ça télécharger et installer la VM (ce qui peut facilement
prendre 10 minutes, en fonction du débit et de la puissance de la
machine).&lt;&#x2F;p&gt;
&lt;p&gt;Dans un monde idéal de développement d’applications pour Cozy,
j’aimerais avoir à faire simplement :&lt;&#x2F;p&gt;
&lt;ol&gt;
&lt;li&gt;lancer kresus&lt;&#x2F;li&gt;
&lt;&#x2F;ol&gt;
&lt;p&gt;C’est-à-dire qu’il y ait une machine toute prête pour le développement.
Peu m’importe l’implémentation :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;que ce soit via le serveur de production, qui utiliserait une
deuxième base couchdb (si cette notion a du sens ?) pour ne pas
écraser les données de l’instance de production;&lt;&#x2F;li&gt;
&lt;li&gt;que ce soit via &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;c9.io&#x2F;&quot;&gt;cloud9&lt;&#x2F;a&gt; ou tout autre système de
dév avec VM intégrée en ligne;&lt;&#x2F;li&gt;
&lt;li&gt;que ce soit via une app de VM sur le cozy (qemu.js, quelqu’un ?);&lt;&#x2F;li&gt;
&lt;li&gt;[insérez d’autres idées folles ici];&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;Allons plus loin : si j’ai la possibilité d’avoir un éditeur de code
directement dans cozy (coucou
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;coolwanglu&#x2F;vim.js&quot;&gt;vim.js&lt;&#x2F;a&gt;), pourquoi ne pas
modifier à la volée mes apps, côté client comme côté serveur, et
recharger directement depuis le cozy ? Cela permettrait d’avoir des
retours encore plus rapides sur le processus de développement (et de
trouver plein de bugs de cozy-controller, si jamais il en y avait
certain de bien planqués !).&lt;&#x2F;p&gt;
&lt;h3 id=&quot;duportage&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#duportage&quot; aria-label=&quot;Anchor link for: duportage&quot;&gt;🔗&lt;&#x2F;a&gt;Du portage&lt;&#x2F;h3&gt;
&lt;p&gt;Je désigne par portage le fait de prendre du code d’une app déjà
existante et d’essayer de le porter sous Cozy.&lt;&#x2F;p&gt;
&lt;p&gt;Pour moi, le plus gros problème du portage est sûrement le fait qu’il
faille &lt;strong&gt;réécrire toutes les URLs&lt;&#x2F;strong&gt;, côté client. De nos jours, assez
souvent, les applications web supposent qu’elles ont accès à un domaine
ou un sous-domaine entier, et pas qu’elles vont vivre dans un
sous-répertoire particulier (sauf en PHP, sigh). Par exemple, le système
de blog Ghost suppose qu’il vit dans &lt;code&gt;http:&#x2F;&#x2F;ghost.example.com&lt;&#x2F;code&gt;, pas
dans &lt;code&gt;http:&#x2F;&#x2F;example.com&#x2F;ghost&lt;&#x2F;code&gt;, sauf contre-indication. Du coup, la
plupart des URLs doivent être réécrites, et c’est souvent assez
difficile ou obscur de trouver où sont ces dernières.&lt;&#x2F;p&gt;
&lt;p&gt;Bien sûr, il y a des solutions :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;un sous-domaine par app, mais ça pose des problèmes pour les
certificats SSL vu qu’il faut des certificats avec &lt;em&gt;wildcard&lt;&#x2F;em&gt; sur
les sous-domaines, mais heureusement &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;letsencrypt.org&#x2F;2015&#x2F;06&#x2F;16&#x2F;lets-encrypt-launch-schedule.html&quot;&gt;LetsEncrypt devrait sortir en
septembre&lt;&#x2F;a&gt;;&lt;&#x2F;li&gt;
&lt;li&gt;utilisation de &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;developer.mozilla.org&#x2F;en-US&#x2F;docs&#x2F;Web&#x2F;API&#x2F;ServiceWorker&quot;&gt;service
workers&lt;&#x2F;a&gt;
pour réécrire toutes les URLs commandées par le client, côté client
: beurk, mais efficace;&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;L’autre problème du portage est &lt;strong&gt;l’intégration des données&lt;&#x2F;strong&gt; : même
s’il est &lt;em&gt;faisable&lt;&#x2F;em&gt; d’intégrer les données d’apps existantes pour
qu’elles vivent dans le couchdb du cozy (voir dernière partie), cela
n’assure pas de créer facilement des interactions entre les données des
apps existantes et des apps portées. Quand bien même l’on voudrait
rajouter ces interactions, il faut modifier l’interface utilisateur de
l’app portée (difficile), ou bien le faire depuis une autre app (étrange
en termes d’interaction utilisateur). Le portage n’est pas un problème
simple (et ce n’est vraiment pas la faute de Cozy, c’est inhérent à tous
les clouds personnels, à mon avis).&lt;&#x2F;p&gt;
&lt;h3 id=&quot;deschoixtechnologiques&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#deschoixtechnologiques&quot; aria-label=&quot;Anchor link for: deschoixtechnologiques&quot;&gt;🔗&lt;&#x2F;a&gt;Des choix technologiques&lt;&#x2F;h3&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;blog.menfin.info&#x2F;posts&#x2F;2015-07-01-sur-un-petit-nuage-cozy&#x2F;&quot;&gt;Le blog de
N`&lt;&#x2F;a&gt;
lit : “[…] je regrette certains choix techniques et les conséquences
associées. CouchDB n’est plus très populaire ces derniers temps.”. A
vrai dire, quelle que fût la technologie utilisée comme backend, on
aurait pu trouver ces remarques, avec “postgre”, ou “mongo”, ou
n’importe quel autre produit à la place de “couchdb”. Et ce n’est pas
que la dite technologie est mauvaise, loin de là ; c’est simplement que
les différents auteurs d’app voudront utiliser des technologies
différentes.&lt;&#x2F;p&gt;
&lt;p&gt;Personnellement, l’utilisation de CouchDB m’a demandé un effort initial
(à cause du système de vues) que ne m’aurait pas demandé une bête base
de données SQL. Ce n’est pas pour autant quelque chose de difficile,
mais c’est quelque chose de nouveau, et l’amas de nouveaux concepts liés
au développement dans Cozy peut être un frein.&lt;&#x2F;p&gt;
&lt;p&gt;Et qu’il s’agisse de la base de données ou du langage, le problème dans
les deux cas est la dispersion : plus d’outils, ça signifie plus de
maintenance nécessaire, plus de vecteurs d’attaques, etc. Pourtant,
combien de personnes adoreraient développer une app pour Cozy, mais ne
le font pas parce qu’il n’est pas possible d’écrire sa web app en
Python, Erlang, Haxe, allez soyons fous, PHP ? Combien de projets ne
sont pas portés, pour les mêmes raisons ? Avec la mode de la
containerisation, sandboxer un interpréteur ou une VM est quasiment
gratuit et pourrait permettre cette utilisation d’autres langages dans
Cozy.&lt;&#x2F;p&gt;
&lt;p&gt;Pour le problème de la base de données, il y a plusieurs approches :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;juste installer plusieurs systèmes de gestion de base de données
dans la plateforme (mongo, cassandra, whatever), mais ce n’est
vraiment pas idéal, pour les raisons exposées ci-dessus;&lt;&#x2F;li&gt;
&lt;li&gt;utiliser un protocole de stockage de données indépendant du langage,
l’implémenter dans cozy puis écrire quelques bibliothèques de code
réutilisables pour pouvoir utiliser ce protocole. C’est l’approche
prise par ces bons gars de Mozilla avec
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;kinto.readthedocs.org&#x2F;&quot;&gt;Kinto&lt;&#x2F;a&gt;, comme expliqué
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;www.servicedenuages.fr&#x2F;stockage-generique-ecosysteme&quot;&gt;ici&lt;&#x2F;a&gt;.
En y réfléchissant bien, c’est &lt;em&gt;presque&lt;&#x2F;em&gt; l’approche prise par le
système de données dans Cozy, vu qu’il utilise des appels à un point
d’accès HTTP : la seule différence réside dans le fait que ces
appels sentent et rappellent trop Couch;&lt;&#x2F;li&gt;
&lt;li&gt;écrire un meta-adaptateur : une application qui comprenne les
protocoles de plusieurs bases de données (de manière à ce que
l’utilisateur puisse avoir l’impression de communiquer avec par
exemple mongo, ou mariadb, indifféremment), et qui retranscrive cela
en structures de données et requêtes couchdb. Je pense que c’est
illusoire : vu que les différents systèmes de stockage peuvent
assurer différentes parties du théorème CAP, beaucoup de
fonctionnalités pourraient ne pas être équivalentes (ou ne pas être
imitées de manière efficace).&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h3 id=&quot;decequimarche&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#decequimarche&quot; aria-label=&quot;Anchor link for: decequimarche&quot;&gt;🔗&lt;&#x2F;a&gt;De ce qui marche&lt;&#x2F;h3&gt;
&lt;p&gt;Les mêmes problèmes se posent pour d’autres plateformes de cloud
personnel comme &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;owncloud.com&#x2F;&quot;&gt;OwnCloud&lt;&#x2F;a&gt; et ne vont pas se
résoudre du jour au lendemain. Chacune de ces idées demande un temps
d’implémentation considérable (sauf peut-être le serveur statique et
l’éditeur de texte – à croire que c’en est une bonne idée !), du temps
pour tester de la part des utilisateurs et beaucoup de retours entre
développeurs et utilisateurs. Pourtant, certains ont déjà été confrontés
à ces problématiques et les ont résolues avec panache.&lt;&#x2F;p&gt;
&lt;p&gt;Un modèle en la matière est &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;app.net&#x2F;&quot;&gt;App.net&lt;&#x2F;a&gt; (ADN pour les
intimes), un réseau social &lt;em&gt;centralisé&lt;&#x2F;em&gt; mais qui assure de ne pas
espionner les données des utilisateurs, en contrepartie d’un abonnement
de quelques dollars par mois. Des apps pour ADN peuvent être créées par
les utilisateurs très facilement, et il en existe de très bonnes.
Qu’est-ce qui a attiré les développeurs pour créer des apps pour ADN ?&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;une &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;developers.app.net&#x2F;docs&#x2F;&quot;&gt;documentation exemplaire&lt;&#x2F;a&gt;,
avec de nombreux tutoriels, des présentations de l’API, plein de
liens vers différents canaux de communications réservés aux
développeurs, des articles de blog juste pour présenter des
nouvelles APIs aux développeurs, des podcasts, etc. La documentation
est open-source et peut être modifiée par n’importe qui, comme pour
Cozy.&lt;&#x2F;li&gt;
&lt;li&gt;une &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;blog.app.net&#x2F;2013&#x2F;11&#x2F;18&#x2F;adnpy-a-new-python-library-from-app-net&#x2F;&quot;&gt;diversité au niveau des
technologies&lt;&#x2F;a&gt;
utilisées pour écrire des apps clientes pour ADN : python, objective
c, android, etc.;&lt;&#x2F;li&gt;
&lt;li&gt;l’&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;blog.app.net&#x2F;2014&#x2F;01&#x2F;27&#x2F;app-net-hackathon-recap-3&#x2F;&quot;&gt;organisation de
hackathons&lt;&#x2F;a&gt;
pour lancer des projets en commun, faire se rencontrer des
développeurs et provoquer le démarrage de nouvelles apps;&lt;&#x2F;li&gt;
&lt;li&gt;des articles de blog &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;blog.app.net&#x2F;2014&#x2F;01&#x2F;17&#x2F;sunlit&#x2F;&quot;&gt;présentant des nouvelles apps pour
ADN&lt;&#x2F;a&gt; (n.b.: kresus n’a pas
encore atteint ce niveau de maturité à mon avis);&lt;&#x2F;li&gt;
&lt;li&gt;encore plus fou, &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;blog.app.net&#x2F;2013&#x2F;03&#x2F;01&#x2F;updates-to-the-app-net-developer-incentive-program&#x2F;&quot;&gt;un programme d’incentives pour le développement
d’app&lt;&#x2F;a&gt;.
En gros, si votre application a été sélectionnée comme étant une des
plus utiles à la communauté, vous recevez une part d’un budget
alloué à la promotion du développement des nouvelles apps. C’est un
argument pécunier, mais c’est sûrement ce qui a fait la différence
entre ADN et &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;tent.io&quot;&gt;tent&lt;&#x2F;a&gt;, par exemple (tent est mort,
vite tent).&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;A mon avis, les trois premiers points demandent un investissement
relativement mineur et permettraient de donner un coup de pied au
développement d’apps sous Cozy. Par ailleurs,
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;kresus&quot;&gt;Kresus&lt;&#x2F;a&gt; cherche toujours de nouveaux
contributeurs, si vous êtes intéressés, n’hésitez-pas à me contacter !&lt;&#x2F;p&gt;
&lt;p&gt;Si vous avez des remarques par rapport à cet article, je vous invite à
m’en faire part sur &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&quot;&gt;twitter&lt;&#x2F;a&gt; pour les
remarques courtes ou sur &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;forum.cozy.io&#x2F;t&#x2F;comment-pouvons-nous-vous-aider-a-bidouiller-cozy&#x2F;&quot;&gt;le
forum&lt;&#x2F;a&gt;
pour les remarques plus longues. Cheers !&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>Wallabag, mot de passes, sel et empreintes</title>
        <published>2015-05-06T19:53:42+00:00</published>
        <updated>2015-05-06T19:53:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/wallabag-changer-mot-de-passe-perdu/"/>
        <id>https://bouvier.cc/tech/wallabag-changer-mot-de-passe-perdu/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/wallabag-changer-mot-de-passe-perdu/">&lt;p&gt;TL;DR:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;$&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; cd &#x2F;var&#x2F;www&#x2F;wallabag&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;$&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; printf&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;motdepasseLogin&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;`&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cat&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; inc&#x2F;poche&#x2F;config.inc.php&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; grep&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;SALT&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; cut&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; -d&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; \&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt; -f&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 4&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;` |&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; openssl&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; sha1&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;span&gt; c87c3d3f823c166602a0ddfc2d96c82aa7b27d9e&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;$&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; sqlite3 db&#x2F;poche.sqlite&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;sqlite&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; update users set password=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;c87c3d3f823c166602a0ddfc2d96c82aa7b27d9e&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; where id=&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;sqlite&lt;&#x2F;span&gt;&lt;span&gt;&amp;gt;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; ^D&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h1 id=&quot;commentchangersonmotdepassewallabagladure&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#commentchangersonmotdepassewallabagladure&quot; aria-label=&quot;Anchor link for: commentchangersonmotdepassewallabagladure&quot;&gt;🔗&lt;&#x2F;a&gt;Comment changer son mot de passe Wallabag, à la dure&lt;&#x2F;h1&gt;
&lt;h5 id=&quot;akacommentperdresonmotdepassewallabagenpremierlieu&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#akacommentperdresonmotdepassewallabagenpremierlieu&quot; aria-label=&quot;Anchor link for: akacommentperdresonmotdepassewallabagenpremierlieu&quot;&gt;🔗&lt;&#x2F;a&gt;aka, comment perdre son mot de passe Wallabag en premier lieu&lt;&#x2F;h5&gt;
&lt;p&gt;&lt;img src=&quot;https:&#x2F;&#x2F;bouvier.cc&#x2F;tech&#x2F;wallabag-changer-mot-de-passe-perdu&#x2F;2015-05-06_kangourou.jpeg&quot; alt=&quot;Wallabag, mot de passes, sel et empreintes&quot; &#x2F;&gt;&lt;&#x2F;p&gt;
&lt;p&gt;J’utilise &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;wallabag.org&#x2F;&quot;&gt;Wallabag&lt;&#x2F;a&gt;, un service de liste de
lecture libre et décentralisé, donc un concurrent de
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;getpocket.com&#x2F;&quot;&gt;Pocket&lt;&#x2F;a&gt; et &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;alternativeto.net&#x2F;software&#x2F;read-it-later&#x2F;&quot;&gt;autres
alternatives&lt;&#x2F;a&gt;, que je
recommande particulièrement pour son efficacité et sa simplicité. En
particulier, j’utilise une &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;registry.hub.docker.com&#x2F;u&#x2F;bobmaerten&#x2F;docker-wallabag&#x2F;&quot;&gt;image
Docker&lt;&#x2F;a&gt;
prête à l’usage, pour éviter d’avoir à installer &lt;code&gt;php&lt;&#x2F;code&gt; directement sur
mon serveur. Tout allait bien, jusqu’à que je redémarre le container :
il m’était alors impossible de me reconnecter. En fouillant un peu, je
me rends compte qu’il y a un sel utilisé pour le hash des mots de passe,
et qu’il est possible que ce sel ait changé suite au redémarrage du
docker (si vous avez cliqué sur le lien du docker registry ci-dessus,
vous verrez qu’il est recommandé de définir ce sel comme une variable
d’environnement, ce que j’avais bien sûr omis). Dans ce cas-là, je me
suis trouvé bloqué hors de mon Wallabag, avec aucune possibilité de
changer le mot de passe (ou de m’envoyer un email de récupération de
compte), ce qui revient à perdre ses données.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;duselpourlehashmaisbiensr&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#duselpourlehashmaisbiensr&quot; aria-label=&quot;Anchor link for: duselpourlehashmaisbiensr&quot;&gt;🔗&lt;&#x2F;a&gt;Du sel pour le hash ? Mais bien sûr !&lt;&#x2F;h2&gt;
&lt;p&gt;Petite explication technique pour ceux qui veulent savoir de quoi on
parle ici. Si sel, hash et SHA1 n’ont pas de secrets pour vous, vous
pouvez passer à la suite.&lt;&#x2F;p&gt;
&lt;p&gt;Il est dangereux de sauvegarder des mots de passe utilisateur en base de
données, en clair : sinon, cela signifie que les administrateurs du site
ou d’éventuels pirates peuvent s’authentifier sur le compte de
l’utilisateur et accéder à toutes ses données. De telles fuites de mots
de passe ont lieu assez couramment. Pour éviter cela, plutôt que stocker
le mot de passe en clair dans une base de données, on utilise une
fonction mathématique à sens unique qui transforme le mot de passe à
l’aide d’une formule mathématique compliquée en une “empreinte”
numérique. Cela signifie qu’il est facile de calculer l’empreinte à
partir du mot de passe, mais qu’il est (théoriquement) compliqué de
retrouver le mot de passe à partir de l’empreinte.&lt;&#x2F;p&gt;
&lt;p&gt;Du coup, plutôt que stocker le mot de passe en clair dans la base de
données, on stocke l’empreinte générée dans la base. Ainsi, pour
vérifier si un mot de passe entré par l’utilisateur sur le site est le
bon, il suffit de générer l’empreinte du mot de passe entré sur le site,
et de comparer avec l’empreinte en base de données. Jusque là, tout va
bien.&lt;&#x2F;p&gt;
&lt;p&gt;Tout va bien, sauf si la fonction mathématiquement théoriquement
compliquée ne l’est pas tant que ça. Pour Wallabag, il s’agit de SHA1,
qui a été prouvée comme étant facilement cassable il y a quelques années
de ça déjà. Il y a une très bonne
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;2772014&#x2F;is-sha-1-secure-for-password-storage&quot;&gt;explication&lt;&#x2F;a&gt;
ici de la méthode utilisée pour retrouver le mot de passe à partir de
l’empreinte. L’idée est de prendre beaucoup de temps à générer des
tables avec toutes les entrées possibles, de manière à avoir une table
de correspondances entrées &#x2F; empreintes, et ensuite quand on a une
empreinte donnée, de comparer à celles qui étaient présentes dans la
table avec toutes les correspondances.&lt;&#x2F;p&gt;
&lt;p&gt;Pour contrer cette attaque, il suffit de rajouter un “sel” après le mot
de passe, c’est-à-dire une chaîne de caractères compliquée, connue
uniquement par l’application elle-même. Ainsi, les attaques par table de
correspondance sont (encore une fois, théoriquement) impossibles, car
l’attaquant ne connaît pas le “sel” rajouté.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;etlelienavecleschmilblick&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#etlelienavecleschmilblick&quot; aria-label=&quot;Anchor link for: etlelienavecleschmilblick&quot;&gt;🔗&lt;&#x2F;a&gt;Et le lien avec le Schmilblick ?&lt;&#x2F;h2&gt;
&lt;p&gt;Dans mon instance de Wallabag, le sel rajouté au mot de passe a été
modifié en redémarrant le container docker, ce qui fait que l’empreinte
que je générais en essayant de me connecter ne correspondait plus à
celle en base de données. Du coup, la solution pour s’en sortir est de
regénérer cette empreinte et la réécrire directement dans la base de
données. Comment faire ?&lt;&#x2F;p&gt;
&lt;p&gt;Heureusement, le code de Wallabag étant open-source, j’ai pu le
consulter sur &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;wallabag&#x2F;wallabag&quot;&gt;github&lt;&#x2F;a&gt;. En
particulier, en faisant une &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;wallabag&#x2F;wallabag&#x2F;search?utf8=%E2%9C%93&amp;amp;q=SALT&quot;&gt;recherche sur
SALT&lt;&#x2F;a&gt;,
j’ai trouvé &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;wallabag&#x2F;wallabag&#x2F;blob&#x2F;master&#x2F;inc&#x2F;poche&#x2F;config.inc.default.php#L11&quot;&gt;cette
ligne&lt;&#x2F;a&gt;
qui définit le sel rajouté pour calculer l’empreinte :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;php&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;@&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;define&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;SALT&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;, &amp;#39;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt; # put a strong string here&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Il suffit donc de regarder la valeur écrite en dur dans ce fichier.
Valeur qui est par ailleurs
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;wallabag&#x2F;wallabag&#x2F;blob&#x2F;4b1fa4c2febc7abbc6da3d65e4e760949a55843c&#x2F;install&#x2F;index.php#L66&quot;&gt;utilisée&lt;&#x2F;a&gt;
au moment de l’installation pour générer l’empreinte du mot de passe en
base de données :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;php&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;$salted_password&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt; sha1&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;$password&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; .&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; $username&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; .&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; $salt&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Donc l’empreinte est générée à partir de la chaîne de caractères
suivante : mot de passe, nom d’utilisateur, sel trouvé dans le fichier
&lt;code&gt;inc&#x2F;poche&#x2F;config.inc.default.php&lt;&#x2F;code&gt;, le tout sans espaces. Il suffit donc
maintenant de retrouver le sel, regénérer l’empreinte et mettre à jour
la base de données, ce qui se fait en 5 lignes, comme le montre la
section TL:DR ci-dessus.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;conclusiongometa&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#conclusiongometa&quot; aria-label=&quot;Anchor link for: conclusiongometa&quot;&gt;🔗&lt;&#x2F;a&gt;Conclusion : go meta&lt;&#x2F;h2&gt;
&lt;p&gt;Cette méthode de génération d’empreintes pour stocker des mots de passe
en base de données est assez récurrente, du coup il est possible de
l’adapter à d’autres algorithmes (SHA256) et d’autres constructions de
chaînes.&lt;&#x2F;p&gt;
&lt;p&gt;Bien sûr, pour ce qui est du Docker Wallabag, il est plus simple de
définir le sel comme une variable d’environnement, comme indiqué dans la
documentation du container.&lt;&#x2F;p&gt;
&lt;p&gt;Par ailleurs, s’il existe une autre méthode pour récupérer son mot de
passe sous Wallabag, je serai ravi de la connaître ! N’hésitez pas à me
faire part de vos remarques via &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&quot;&gt;twitter&lt;&#x2F;a&gt;.&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>CozyCloud : migrer son instance d&#x27;un serveur vers un autre</title>
        <published>2015-04-21T21:12:42+00:00</published>
        <updated>2015-04-21T21:12:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/cozycloud-migrer-son-instance-dun-serveur-vers-un-autre/"/>
        <id>https://bouvier.cc/tech/cozycloud-migrer-son-instance-dun-serveur-vers-un-autre/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/cozycloud-migrer-son-instance-dun-serveur-vers-un-autre/">&lt;p&gt;Il arrive de temps en temps de devoir changer de serveur, et puisque le
credo de &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;cozy.io&#x2F;&quot;&gt;CozyCloud&lt;&#x2F;a&gt; est de pouvoir partir à n’importe
quel moment, encore faut-il pouvoir le faire facilement et savoir
comment le faire ! En attendant que l’équipe mette en place un moyen
officiel d’effectuer cette tâche, voici une méthode que j’ai utilisée
pour migrer mon instance cozy d’un serveur vers un cozy situé sur un
autre serveur distant. C’est dangereux, vous allez mettre les mains dans
le cambouis et vous salir, mais au final vous ne perdrez aucune donnée
après la migration !&lt;&#x2F;p&gt;
&lt;h2 id=&quot;postscriptum&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#postscriptum&quot; aria-label=&quot;Anchor link for: postscriptum&quot;&gt;🔗&lt;&#x2F;a&gt;Post scriptum&lt;&#x2F;h2&gt;
&lt;p&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;nledez&quot;&gt;Nicolas&lt;&#x2F;a&gt;, de l’équipe Cozy, m’a dit qu’il
est plus rapide et plus efficace de faire une simple copie du fichier de
la base de données cozy &lt;code&gt;cozy.couch&lt;&#x2F;code&gt; depuis la machine source et de le
remplacer dans la machine distante. Dans ce cas, pas besoin de vous
embêter avec toutes les péripéties de ce tutoriel ! Des gens en parlent
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;forum.cozy.io&#x2F;t&#x2F;comment-faire-un-backup-de-la-base-de-donnees-couchdb&#x2F;531&quot;&gt;ici&lt;&#x2F;a&gt;.
Si quelqu’un a plus de détails sur la méthode et confirme que celle-ci
fonctionne, je serai ravi de mettre à jour cet article, n’hésitez-pas à
me contacter ! (cf fin du message)&lt;&#x2F;p&gt;
&lt;h2 id=&quot;lamthode&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#lamthode&quot; aria-label=&quot;Anchor link for: lamthode&quot;&gt;🔗&lt;&#x2F;a&gt;La méthode&lt;&#x2F;h2&gt;
&lt;p&gt;Dans ce tutoriel, j’utiliserai le vocabulaire suivant :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;la machine &lt;strong&gt;source&lt;&#x2F;strong&gt; sera la machine depuis laquelle on fait la
copie du cozy. C’est elle qui contient toutes les données et que
l’on cherche à répliquer.&lt;&#x2F;li&gt;
&lt;li&gt;la machine &lt;strong&gt;cible&lt;&#x2F;strong&gt; sera la machine vers laquelle on migre son
cozy. Je recommande que cette instance dispose d’un cozy pré
installé, &lt;strong&gt;à jour&lt;&#x2F;strong&gt; (la plateforme devrait être à jour), mais sans
aucune donnée à l’intérieur (c’est-à-dire un cozy tel qu’il serait
juste après l’installation par les scripts). Idéalement, il faudrait
que le nom d’utilisateur (email) de ce cozy cible soit quelque chose
de facilement reconnaissable, vu que l’on devra identifier cet
utilisateur plus tard… Par exemple, utiliser &lt;code&gt;demo@domain.tld&lt;&#x2F;code&gt;
comme email de login.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h2 id=&quot;cibledsactiverlascuritdesappareilsauthentifis&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#cibledsactiverlascuritdesappareilsauthentifis&quot; aria-label=&quot;Anchor link for: cibledsactiverlascuritdesappareilsauthentifis&quot;&gt;🔗&lt;&#x2F;a&gt;Cible : Désactiver la sécurité des appareils authentifiés&lt;&#x2F;h2&gt;
&lt;p&gt;Pour cette procédure, nous allons en fait nous faire passer pour
n’importe quel appareil qui voudrait se synchroniser avec le cozy,
c’est-à-dire que l’on va se faire passer pour l’application mobile
&lt;code&gt;cozy-mobile&lt;&#x2F;code&gt; ou pour l’application bureau &lt;code&gt;cozy-desktop&lt;&#x2F;code&gt;. Cependant, la
synchronisation avec ces appareils marche (grossièrement) de la manière
suivante :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;l’appareil qui veut se synchroniser avec le cozy s’authentifie la
première fois, avec le mot de passe du cozy.&lt;&#x2F;li&gt;
&lt;li&gt;l’appareil est inscrit dans la liste des appareils autorisés.&lt;&#x2F;li&gt;
&lt;li&gt;lors des synchronisations, le serveur cozy vérifie que l’appareil
est autorisé avant de transmettre la requête de réplication à
couchdb, qui lui gérera l’authentification.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;Seulement dans notre cas, on n’a pas la possibilité de s’inscrire en
tant qu’appareil autorisé, on va donc devoir contourner le code qui fait
cette vérification ! Bien sûr, ce ne sera que temporaire et on remettra
le code tel qu’il était à l’origine, dans la dernière étape…&lt;&#x2F;p&gt;
&lt;p&gt;On se place dans le bon répertoire, tout d’abord :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #0DB9D7;&quot;&gt;cd&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; &#x2F;usr&#x2F;local&#x2F;cozy&#x2F;apps&#x2F;proxy&#x2F;build&#x2F;server&#x2F;controller&#x2F;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Il faut ensuite ouvrir le fichier &lt;code&gt;devices.js&lt;&#x2F;code&gt; avec les droits de
modification (je suis passé par &lt;code&gt;sudo vim devices.js&lt;&#x2F;code&gt; pour le faire,
mais vous avez le droit à d’autres éditeurs de texte, inférieurs
soient-ils :D).&lt;&#x2F;p&gt;
&lt;p&gt;On identifie la fonction &lt;code&gt;replication&lt;&#x2F;code&gt;, et plus particulièrement la
partie qui vérifie l’authentification :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; deviceManager&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;isAuthenticated&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;username&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; password&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; function&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt;auth&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;    var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;auth&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;process&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;env&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;NODE_ENV&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; ===&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;production&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            req&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;headers&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;authorization&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; getCredentialsHeader&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;()&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; else&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            req&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;headers&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;authorization&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; null&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;        return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; getProxy&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;()&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;web&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;req&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; res&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;            target&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;http:&#x2F;&#x2F;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; couchdbHost&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; + &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; couchdbPort&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        })&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; else&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; = new&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; Error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Request unauthorized&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;status&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 401&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;        return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; next&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;})&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;On va juste remplacer &lt;code&gt;auth&lt;&#x2F;code&gt; par &lt;code&gt;true&lt;&#x2F;code&gt; dans la première condition, pour
faire croire au proxy que l’appareil est toujours authentifié :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;javascript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; deviceManager&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;isAuthenticated&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;username&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; password&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; function&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #E0AF68;&quot;&gt;auth&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9D7CD8;font-style: italic;&quot;&gt;    var&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;    if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;true&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;span style=&quot;color: #51597D;font-style: italic;&quot;&gt;   &#x2F;&#x2F; &amp;lt;============= ICI, j&amp;#39;ai remplacé auth par true&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt;        if&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; (&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;process&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;env&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt;NODE_ENV&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; ===&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;production&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;) {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            req&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;headers&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;authorization&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; getCredentialsHeader&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;()&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; else&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;            req&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;headers&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;[&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;authorization&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;#39;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;]&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; null&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;        return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; getProxy&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;()&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt;web&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;req&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; res&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;,&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #73DACA;&quot;&gt;            target&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;: &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;http:&#x2F;&#x2F;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; couchdbHost&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; + &amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;:&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot; +&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt; couchdbPort&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;        })&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;span style=&quot;color: #BB9AF7;&quot;&gt; else&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt; {&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; = new&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; Error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt;Request unauthorized&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;&amp;quot;&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;        error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;.&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7DCFFF;&quot;&gt;status&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt; =&lt;&#x2F;span&gt;&lt;span style=&quot;color: #FF9E64;&quot;&gt; 401&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #BB9AF7;font-style: italic;&quot;&gt;        return&lt;&#x2F;span&gt;&lt;span style=&quot;color: #7AA2F7;&quot;&gt; next&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;(&lt;&#x2F;span&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;error&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;)&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;    }&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #9ABDF5;&quot;&gt;})&lt;&#x2F;span&gt;&lt;span style=&quot;color: #89DDFF;&quot;&gt;;&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;On sauvegarde et on quitte, puis on relance le proxy pour être sûr que
les modifications sont prises en compte :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cozy-monitor&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; restart proxy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h2 id=&quot;ciblercuprerlemotdepassedecouchdb&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#ciblercuprerlemotdepassedecouchdb&quot; aria-label=&quot;Anchor link for: ciblercuprerlemotdepassedecouchdb&quot;&gt;🔗&lt;&#x2F;a&gt;Cible : Récupérer le mot de passe de couchdb&lt;&#x2F;h2&gt;
&lt;p&gt;On a maintenant besoin de l’identifiant et du mot de passe pour accéder
à la base de données couchdb de la machine cible. Pour cela, il suffit
de regarder dans &lt;code&gt;&#x2F;etc&#x2F;cozy&#x2F;couchdb.login&lt;&#x2F;code&gt;:&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cat&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; &#x2F;etc&#x2F;cozy&#x2F;couchdb.login&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;Chez moi, ça affiche :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;plain&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;bonjour&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;
&lt;span class=&quot;giallo-l&quot;&gt;&lt;span&gt;motdepasse&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;La première ligne correspond au login, la seconde au mot de passe. On
note ça quelque part, et c’est parti pour lancer la copie.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;sourcelancerlacopie&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#sourcelancerlacopie&quot; aria-label=&quot;Anchor link for: sourcelancerlacopie&quot;&gt;🔗&lt;&#x2F;a&gt;Source : lancer la copie&lt;&#x2F;h2&gt;
&lt;p&gt;Depuis la machine source maintenant, on va lancer la copie. Imaginons
que ma machine cible soit située sur le nom de domaine
&lt;code&gt;cozy.example.tld&lt;&#x2F;code&gt;, alors la commande à lancer est la suivante :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cozy-monitor&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; backup https:&#x2F;&#x2F;bonjour:motdepasse@cozy.example.tld&#x2F;cozy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;
&lt;p&gt;où &lt;code&gt;bonjour&lt;&#x2F;code&gt; et &lt;code&gt;motdepasse&lt;&#x2F;code&gt; sont respectivement le nom d’utilisateur et
le mot de passe couchdb de la machine cible.&lt;&#x2F;p&gt;
&lt;p&gt;La manipulation va prendre un peu de temps, et il arrive qu’elle échoue
même si vous avez beaucoup de données. Personnellement, il m’a fallu la
relancer plusieurs fois avant que ça finisse sans encombre, mais au bout
d’un moment, ça s’est arrêté de planter et ça a finalement marché.&lt;&#x2F;p&gt;
&lt;h2 id=&quot;cibleremettrelauthentificationdesappareilsdanssontatinitial&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#cibleremettrelauthentificationdesappareilsdanssontatinitial&quot; aria-label=&quot;Anchor link for: cibleremettrelauthentificationdesappareilsdanssontatinitial&quot;&gt;🔗&lt;&#x2F;a&gt;Cible : Remettre l’authentification des appareils dans son état initial&lt;&#x2F;h2&gt;
&lt;p&gt;La sécurité étant cruciale pour éviter que n’importe qui ne puisse
récupérer vos données personnelles, pensez bien à remettre le fichier de
code modifié &lt;code&gt;devices.js&lt;&#x2F;code&gt; dans son état initial (c’est-à-dire changer la
condition de &lt;code&gt;if (true)&lt;&#x2F;code&gt; à &lt;code&gt;if (auth)&lt;&#x2F;code&gt;), et à redémarrer le proxy sur la
machine cible :&lt;&#x2F;p&gt;
&lt;pre class=&quot;giallo&quot; style=&quot;color: #A9B1D6; background-color: #1A1B26;&quot;&gt;&lt;code data-lang=&quot;shellscript&quot;&gt;&lt;span class=&quot;giallo-l&quot;&gt;&lt;span style=&quot;color: #C0CAF5;&quot;&gt;cozy-monitor&lt;&#x2F;span&gt;&lt;span style=&quot;color: #9ECE6A;&quot;&gt; restart proxy&lt;&#x2F;span&gt;&lt;&#x2F;span&gt;&lt;&#x2F;code&gt;&lt;&#x2F;pre&gt;&lt;h2 id=&quot;ciblerafistolerlesdonnes&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#ciblerafistolerlesdonnes&quot; aria-label=&quot;Anchor link for: ciblerafistolerlesdonnes&quot;&gt;🔗&lt;&#x2F;a&gt;Cible : Rafistoler les données&lt;&#x2F;h2&gt;
&lt;p&gt;Maintenant, on va se connecter via l’interface web à la machine cible,
comme on se connecterait à n’importe quel moment pour aller voir son
cozy. A ce point-là, je me suis connecté avec mon login et mot de passe
utilisés lors de la création du cozy cible, mais il se pourrait bien que
ça marche avec le login et mot de passe de la machine source également
(à confirmer).&lt;&#x2F;p&gt;
&lt;p&gt;Le cozy cible est dans un état inconsistent. Pourquoi ?&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;Il y a plusieurs données qui sont censées être présentes de manière
unique (informations sur l’instance, l’utilisateur, etc.), et qui
sont présentes en doublons. En effet, ces données sont créées une
fois lors de la création du cozy, et on les a en double à cause de
la copie du cozy source. Il va falloir nettoyer ces données.&lt;&#x2F;li&gt;
&lt;li&gt;Les mots de passe stockés dans la base sont chiffrés avec des clés
qui dépendent du mot de passe de l’utilisateur et d’un sel
cryptographique, uniques par utilisateur. Du coup, comme il y a
plusieurs utilisateurs (rappelez-vous, les données sont en double),
le mot de passe pourrait ne pas être déchiffré correctement, ce qui
fait que les applications de récupération de données (comme
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;kresus&quot;&gt;kresus&lt;&#x2F;a&gt; ou
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;frankrousseau&#x2F;konnectors&quot;&gt;konnectors&lt;&#x2F;a&gt;) ne vont
plus marcher correctement.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;p&gt;La bonne nouvelle, c’est que ces deux problèmes se résolvent tous les
deux en supprimant les données présentes en doublons. Pour faire cela,
voici la liste des étapes :&lt;&#x2F;p&gt;
&lt;ul&gt;
&lt;li&gt;depuis l’interface web du cozy cible, on désinstalle l’appli
&lt;code&gt;data browser&lt;&#x2F;code&gt; (toutes mes données) si elle était présente, puis on
la réinstalle. Cela est nécessaire parce que le cozy pourrait croire
que l’application est installée en double (le cozy a des
enregistrements en base de données sur les applications actuellement
installées).&lt;&#x2F;li&gt;
&lt;li&gt;on lance l’application &lt;code&gt;data-browser&lt;&#x2F;code&gt;&lt;&#x2F;li&gt;
&lt;li&gt;dans la table &lt;code&gt;CozyInstance&lt;&#x2F;code&gt;, on supprime l’instance qui ne
correspond pas à celle du cozy cible (a priori, c’est celle qui n’a
pas de nom de domaine, sauf si vous en aviez défini un sur le cozy
cible).&lt;&#x2F;li&gt;
&lt;li&gt;dans la table &lt;code&gt;StackApplication&lt;&#x2F;code&gt;, on supprime toutes les lignes en
double. Comme il s’agit des applications de la plateforme, si le
cozy cible était bien mis à jour avant de faire ces opérations, il
suffit de supprimer les lignes avec les numéros de version
inférieurs ou égaux.&lt;&#x2F;li&gt;
&lt;li&gt;dans la table &lt;code&gt;Applications&lt;&#x2F;code&gt;, on supprime toutes les lignes (en
cliquant sur l’icône de corbeille), &lt;strong&gt;sauf data-browser&lt;&#x2F;strong&gt; qu’on a
remis dans un état consistent à la main.&lt;&#x2F;li&gt;
&lt;li&gt;dans la table &lt;code&gt;User&lt;&#x2F;code&gt;, on supprime la ligne qui correspond à
l’utilisateur du cozy cible (si vous avez suivi mon exemple, c’est
le &lt;code&gt;demo@domain.tld&lt;&#x2F;code&gt; dont je parlais au tout début).&lt;&#x2F;li&gt;
&lt;li&gt;on se déconnecte et on se reconnecte sur le cozy.&lt;&#x2F;li&gt;
&lt;li&gt;laisser reposer, mettre au four thermostat 6 pendant 1 minute.&lt;&#x2F;li&gt;
&lt;li&gt;c’est prêt, vous pouvez déguster !&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h2 id=&quot;conclusion&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#conclusion&quot; aria-label=&quot;Anchor link for: conclusion&quot;&gt;🔗&lt;&#x2F;a&gt;Conclusion&lt;&#x2F;h2&gt;
&lt;p&gt;J’espère que cette procédure aura marché pour vous, je n’ai pas pu la
réessayer, du coup cette procédure est extraite de mes notes
personnelles et il se pourrait que j’ai oublié quelques détails. Si
c’est le cas, n’hésitez pas à me contacter sur
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;twitter.com&#x2F;bnjbvr&quot;&gt;twitter&lt;&#x2F;a&gt; ou sur irc (mon nick est
&lt;code&gt;bnjbvr&lt;&#x2F;code&gt;). Si ça a marché et que vous avez apprécié, n’hésitez pas à me
le dire également ;-)&lt;&#x2F;p&gt;
</content>
    </entry>
    <entry xml:lang="en">
        <title>What I&#x27;ve Made On Year 2014</title>
        <published>2015-03-16T13:22:42+00:00</published>
        <updated>2015-03-16T13:22:42+00:00</updated>
        
        <author>
          <name>
            
              Benjamin Bouvier
            
          </name>
        </author>
        
        <link rel="alternate" type="text/html" href="https://bouvier.cc/tech/what-ive-made-on-year-2014/"/>
        <id>https://bouvier.cc/tech/what-ive-made-on-year-2014/</id>
        <content type="html" xml:base="https://bouvier.cc/tech/what-ive-made-on-year-2014/">&lt;p&gt;I’ve stumbled across &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;isawsomecode.tumblr.com&#x2F;post&#x2F;106997844921&#x2F;what-i-made-on-year-2014&quot;&gt;espadrine’s blog
post&lt;&#x2F;a&gt;
about his accomplishments in 2014 and found it was a really good idea.
It matters a lot to acknowledge your own accomplishments, without being
overly proud about them, just to realize &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Impostor_syndrome&quot;&gt;you’re not an
impostor&lt;&#x2F;a&gt;, and that you
bring value to your company and deserve your current position (yes,
there probably will be more about the so-called impostor syndrome on
this blog).&lt;&#x2F;p&gt;
&lt;span id=&quot;continue-reading&quot;&gt;&lt;&#x2F;span&gt;&lt;h3 id=&quot;opensource&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#opensource&quot; aria-label=&quot;Anchor link for: opensource&quot;&gt;🔗&lt;&#x2F;a&gt;open-source&lt;&#x2F;h3&gt;
&lt;ul&gt;
&lt;li&gt;I’ve started &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;kresus&quot;&gt;kresus&lt;&#x2F;a&gt;, which is
probably my biggest personal open-source project so far. It is a
personal finance manager, a la Linxo &#x2F; Mint &#x2F; Bankin, et al. The
only difference is that you can self-host it (if you have the chance
to own a &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;cozy.io&quot;&gt;CozyCloud&lt;&#x2F;a&gt; instance, which you can also
self-host). It is a fork of another open-source project,
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;seeker89&#x2F;cozy-pfm&#x2F;&quot;&gt;cozy-pfm&lt;&#x2F;a&gt;, which hasn’t been
maintained for long and whose owner made a few technical choices I
didn’t personally like (using backbone was the biggest one).&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;minstrel&quot;&gt;minstrel&lt;&#x2F;a&gt; is a non official
Spotify for Firefox OS app (a.k.a web app). In particular, it spawns
a proxy server that can read your Spotify playlists, stream tracks
in real-time, encode them in real-time before streaming them back to
a simple HTML5 &lt;code&gt;audio&lt;&#x2F;code&gt; tag. The web frontend can also synchronize
your tracks locally, so that you can still listen to your tracks
even when you’re offline, using the awesome
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;mozilla.github.io&#x2F;localForage&#x2F;&quot;&gt;localForage&lt;&#x2F;a&gt; library.&lt;&#x2F;li&gt;
&lt;li&gt;A few small contributions to different cozycloud projects, among
which I’d like to highlight the &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;cozy&#x2F;cozy-home&#x2F;pull&#x2F;178&quot;&gt;app update
notification&lt;&#x2F;a&gt; in
cozy-home (“hey, this app has a new version, go update it if you
want to!”).&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;diary&quot;&gt;diary&lt;&#x2F;a&gt;, a &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;tent.io&quot;&gt;tent&lt;&#x2F;a&gt;
blogging application which I started earlier in 2013. I’ve stopped
working on it as I don’t have a lot of traction in the tent
community.&lt;&#x2F;li&gt;
&lt;li&gt;&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;gsd&quot;&gt;gsd&lt;&#x2F;a&gt;, a todo list app started in
December 2013, to experiment with angularjs. Stopped development as
I found out about &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;jsilvestre&#x2F;tasky&quot;&gt;tasky&lt;&#x2F;a&gt; and
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;bulletjournal.com&#x2F;&quot;&gt;bullet journaling&lt;&#x2F;a&gt;.&lt;&#x2F;li&gt;
&lt;li&gt;extended &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;bnjbvr&#x2F;lebonsite&quot;&gt;lebonsite&lt;&#x2F;a&gt; a few by
adding a new frontend that includes realtime search. This app works
along with &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;ClementNotin&#x2F;lebonscrap&quot;&gt;lebonscrap&lt;&#x2F;a&gt;
and allows you to aggregate flat search results (it’s deeply
localized to France’s flat search websites) and quickly go through
them. Update: if you’re interested in the changes I’ve made, they
are in the &lt;em&gt;angular&lt;&#x2F;em&gt; branch of my repository.&lt;&#x2F;li&gt;
&lt;li&gt;as my job is about working in open-source, the next paragraph also
applies here.&lt;&#x2F;li&gt;
&lt;&#x2F;ul&gt;
&lt;h3 id=&quot;school-job&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#school-job&quot; aria-label=&quot;Anchor link for: school-job&quot;&gt;🔗&lt;&#x2F;a&gt;school&#x2F;job&lt;&#x2F;h3&gt;
&lt;p&gt;In March 2014, I’ve got an engineering diploma in computer science from
INSA Lyon, with the highest possible honors (in French, “félicitations
du jury”). Again, not being cocky: graduating with honors is pretty
classic and doesn’t mean anything regarding your actual knowledge &#x2F;
level in anything.&lt;&#x2F;p&gt;
&lt;p&gt;I’ve been hired as a full-time employee by Mozilla at the end of
January, following my internship during Summer 2013. I work in the
Platform team, in the JavaScript Engine sub-team. My main focuses have
been extending &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;http:&#x2F;&#x2F;asmjs.org&quot;&gt;asm.js&lt;&#x2F;a&gt;, to help proving that the web
platform is ready for gaming. I mostly implemented
&lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;github.com&#x2F;johnmccutchan&#x2F;ecmascript_simd&quot;&gt;SIMD.js&lt;&#x2F;a&gt; in Firefox
this year (if you’re interested in reading more about this, feel free to
read the official Mozilla &lt;a rel=&quot;noopener noreferrer external&quot; target=&quot;_blank&quot; href=&quot;https:&#x2F;&#x2F;blog.mozilla.org&#x2F;javascript&#x2F;2015&#x2F;03&#x2F;10&#x2F;state-of-simd-js-performance-in-firefox&#x2F;&quot;&gt;blog
post&lt;&#x2F;a&gt;
I’ve co-written).&lt;&#x2F;p&gt;
&lt;h3 id=&quot;whatabout2015&quot;&gt;&lt;a class=&quot;zola-anchor&quot; href=&quot;#whatabout2015&quot; aria-label=&quot;Anchor link for: whatabout2015&quot;&gt;🔗&lt;&#x2F;a&gt;what about 2015&lt;&#x2F;h3&gt;
&lt;p&gt;I’d like to get more focused in cloud services that you own and have
control on, as I think this is the thing that matters the most,
nowadays.&lt;&#x2F;p&gt;
&lt;p&gt;Also, I wish I can keep on learning new things, by myself or on massive
online open courses. As a newcomer in adulthood, I would also like to
learn how to take advantage of every single hour of every single day,
that is, not getting stuck in a routine and waking up one day thinking
“man, I’m already that old?”. Oh boy, there are only 24 hours a day.&lt;&#x2F;p&gt;
</content>
    </entry>
</feed>

