Final? post about gcc bootstrap: RISCV support for bootstrappable TinyCC

1 files changed, 371 insertions, 0 deletions

diff --git a/content/bootstrapGcc/06_tcc_mes.md b/content/bootstrapGcc/06_tcc_mes.md
new file mode 100644
index 0000000..e2a4a0e
--- /dev/null
+++ b/content/bootstrapGcc/06_tcc_mes.md
@@ -0,0 +1,371 @@
+Title: Milestone – RISC-V support in Mes's bootstrappable TinyCC
+Date: 2022-09-22
+Category:
+Tags: Bootstrapping GCC in RISC-V
+Slug: bootstrapGcc6
+Lang: en
+Summary:
+    Bringing RISC-V support to the bootstrappable TinyCC Mes forked. Some
+    problems and a look into the future.
+
+In the [series]({tag}Bootstrapping GCC in RISC-V) we already introduced GCC,
+TinyCC, Mes and Mes's TinyCC fork that is designed to be bootstrappable. In
+this post we are going to deal with the latter, explain how we made it work for
+RISC-V and the challenges we encountered.
+
+### The non-bootstrappable nature of TinyCC
+
+As we introduced in the previous post TinyCC is not compilable from very simple
+compilers like Mes's `mescc`. So the Mes project decided to make a [fork that
+`mescc` was able to compile](https://gitlab.com/janneke/tinycc). Mes calls it a
+*bootstrappable tinycc*.
+
+> There's a in uninteresting philosophical debate about what does
+> *bootstrappable* mean, which leads to many errors and
+> misunderstandings[^misunderstandings]. Many compilers call themselves
+> bootstrappable if they can be compiled with themselves. When **we** talk
+> about this, we are looking for a *full-source bootstrappability*, that is,
+> that the compilers can be compiled from *source*, or from a *full-source
+> bootstrappable* compiler.
+
+TinyCC is supposed to be compilable by itself, but who compiles the version
+that compiles TinyCC? Another TinyCC? And who compiles that?
+
+The yogurt problem we always get: how do you make yogurt? Take yogurt, mix with
+milk and in some hours you'll get yogurt. See the problem?
+
+If you are a culinary maniac, as I am, you can stretch this metaphor further.
+If you know what you are doing, you can obtain yogurt from raw milk[^kefir].
+
+That's what our project is doing: make yogurt from raw milk at some point.
+
+So the compilers normally only care about the latest yogurt, but, we, the
+saviors of the ancient milk, those who can acidify the raw pureness, can make
+yogurt starter with raw milk.
+
+That's the kind of magic nobody cares about, not in the compiler world nor in
+the real life.
+
+The yogurt starter does not make the best yogurt, by the way, it needs
+generations and generations of yogurts to make the best. That's what our
+project does: start simple (stage-0 and Mes) and go enriching the product
+(TinyCC) until reaching a mature yogurt (GCC).
+
+TinyCC does not really care about this bootstrappability concept. They only
+want to be compilable with themselves. Nothing else.
+
+That's why [Jan](http://joyofsource.com/), the inventor of this metaphor I just
+stretched to the infinite, had to fork the project. He had another choice:
+simplify TinyCC's code upstream to be able to be compiled from a simpler step,
+but his ideas were rejected and some weird animosity I don't understand
+started. More on that later.
+
+[^misunderstandings]: I've reached many misunderstandings about my project too.
+Some people have told me all this work is worthless because you can always
+bootstrap from an x86_64 machine and then continue the bootstrapping effort in
+your RISC-V. And so on. That's why this blog doesn't have a comment section.
+People insist to believe that other people's work is worthless or they are able
+to do it simpler with no effort. I won't claim that my explanations are the
+best, but I can claim to be the laziest person I know, and I'd never spent time
+in something that doesn't worth the effort.
+
+[^kefir]: With kefir you are fucked. We don't know where it comes from. Luckily
+  we harvested a lot and it's easy to grow.
+
+### The RISC-V support
+
+When the previous blogpost was written, TinyCC had a RV64 backend, but the
+TinyCC fork did not have RISC-V support.
+
+My job here was to take the backend from the official TinyCC and bring it to
+the bootstrappable one, Jan's fork. I can say that is done. Good for me.
+
+#### The process
+
+I followed the cross-compiler trick again, in order to make this process easier
+in my computer and because Mes doesn't support RISC-V output yet. Making a
+TinyCC for my x86_64 machine that had RISC-V output sounded more than
+reasonable to me. Later I could always move to a full RISC-V machine making
+sure that the backend was working.
+
+So first I made a guix package for upstream [TinyCC cross-compiler (for
+RISC-V)](https://github.com/ekaitz-zarraga/tcc/blob/guix_package/guix.scm#L85)
+with GCC. This wasn't really obvious, because there were some variables to set
+correctly. Tested everything compiled and worked like expected. Apart from a
+couple of issues later corrected upstream, it did.
+
+Next, I made a guix package for [the forked TinyCC with
+GCC](https://github.com/ekaitz-zarraga/tcc/blob/riscv-mes/guix.scm#L83). This
+also needed some changes, as the forked one is a quite old version of TinyCC.
+The process needs here a `libtcc1.a` that can be empty if the process is
+compiled with GCC (`libgcc` provides that functionality) but the compilation
+process doesn't mention anything about this, and coming up with that by
+yourself is hard.
+
+Now the project was compilable, it was time to code. You can see this part in
+the `riscv-mes` branch:
+
+<https://github.com/ekaitz-zarraga/tcc/commits/riscv-mes>
+
+I took the backend from the upstream and inserted it in the fork. Of course, it
+didn't compile. Many internal structures and APIs changed, so after trying to
+stitch all together myself, I headed to the Mailing List. At the beginning I
+wanted to think the answers I was getting were because I wasn't explaining my
+doubts properly or something but what it was happening was that the animosity
+towards our fork (decision I didn't take) appeared and someone tried to
+ridicule me in the mailing list for no reason at all.
+
+The funny thing is I'd never needed to contact the mailing list if the project
+was as well written as they claim it to be. It's full of functions and
+variables with one character, the code is mixed together in a very aggressive
+way... It's supersmall, tiny even, but really hard to read. Also, the commits
+are not very descriptive for anyone that is not the main maintainer, who,
+surprise! Is the same person that gives aggressive answers in the mailing
+list... I hope it's only my perception and they are nice with his friends and
+family, but the interaction made me feel uncomfortable and I don't want to
+touch this code again.
+
+It was a sad moment, I must admit. But I decided I was going to do this with
+help or without it. And I think I did it. Removed references here and there and
+finally it looks like I reached somewhere.
+
+There are some differences to point out, one of the commits that made me ask in
+the mailing list was a huge change on the way that conditionals are handled in
+TinyCC. Our fork didn't have that so I needed to split the code in several
+pieces and the benefits from that commit (some instruction optimization) are
+lost in the backport. Still the branching and jumping is correct, but less
+optimal. Not bad.
+
+Code added and compiled, it was time for testing. I made a little script (I
+didn't share that, but it's not really relevant either) and a small test case
+of simple C files and compiled (not linked) them with the upstream version of
+the compiler and the forked one. Disassembled them and compared differences.
+
+You can try it building the upstream TinyCC and the fork and make them compile
+(`-c`) a some files. Use `objdump --dissassemble` and see the results. It's not
+really hard to test. Here you have an example of a program you can build:
+
+``` clike
+// Example file to build
+int main (int argc, char *argv[]){
+    int a = 19, b = 90;
+    if (a && b){
+        return 1;
+    } else {
+        return 45 + 90 << 8;
+    }
+}
+```
+
+And the result it should give in both versions, optimized (upstream) and
+unoptimized (our fork):
+
+``` text
+OPTIMIZED VERSION                              || UNOPTIMIZED VERSION
+===============================================||==================================================
+0000000000000000 <main>:                       || 0000000000000000 <main>:
+   0:	fd010113	addi	sp,sp,-48          ||    0:	fd010113	addi	sp,sp,-48
+   4:	02113423	sd	ra,40(sp)              ||    4:	02113423	sd	ra,40(sp)
+   8:	02813023	sd	s0,32(sp)              ||    8:	02813023	sd	s0,32(sp)
+   c:	03010413	addi	s0,sp,48           ||    c:	03010413	addi	s0,sp,48
+  10:	00000013	nop                        ||   10:	00000013	nop
+  14:	fea43423	sd	a0,-24(s0)             ||   14:	fea43423	sd	a0,-24(s0)
+  18:	feb43023	sd	a1,-32(s0)             ||   18:	feb43023	sd	a1,-32(s0)
+  1c:	0130051b	addiw	a0,zero,19         ||   1c:	0130051b	addiw	a0,zero,19
+  20:	fca42e23	sw	a0,-36(s0)             ||   20:	fca42e23	sw	a0,-36(s0)
+  24:	05a0051b	addiw	a0,zero,90         ||   24:	05a0051b	addiw	a0,zero,90
+  28:	fca42c23	sw	a0,-40(s0)             ||   28:	fca42c23	sw	a0,-40(s0)
+  2c:	fdc42503	lw	a0,-36(s0)             ||   2c:	fdc42503	lw	a0,-36(s0)
+  30:	00051463	bnez	a0,38 <main+0x38>  ||   30:	00051463	bnez	a0,38 <main+0x38>
+  34:	0180006f	j	4c <main+0x4c>         ||   34:	01c0006f	j	50 <main+0x50>
+  38:	fd842503	lw	a0,-40(s0)             ||   38:	fd842503	lw	a0,-40(s0)
+  3c:	00051463	bnez	a0,44 <main+0x44>  ||   3c:	00051463	bnez	a0,44 <main+0x44>
+  40:	00c0006f	j	4c <main+0x4c>         ||   40:	0100006f	j	50 <main+0x50>
+  44:	0010051b	addiw	a0,zero,1          ||   44:	0010051b	addiw	a0,zero,1
+  48:	0100006f	j	58 <main+0x58>         ||   48:	0140006f	j	5c <main+0x5c>
+  4c:	00008537	lui	a0,0x8                 ||   4c:	0100006f	j	5c <main+0x5c>
+  50:	7005051b	addiw	a0,a0,1792         ||   50:	00008537	lui	a0,0x8
+  54:	00000033	add	zero,zero,zero         ||   54:	7005051b	addiw	a0,a0,1792
+  58:	02813083	ld	ra,40(sp)              ||   58:	00000033	add	zero,zero,zero
+  5c:	02013403	ld	s0,32(sp)              ||   5c:	02813083	ld	ra,40(sp)
+  60:	03010113	addi	sp,sp,48           ||   60:	02013403	ld	s0,32(sp)
+  64:	00008067	ret                        ||   64:	03010113	addi	sp,sp,48
+                                               ||   68:	00008067	ret
+
+```
+
+In the right you can see there are some `j` instructions duplicated, but it's
+not supposed to be a problem, as the rest of the addresses are calculated
+properly, and they are never going to be reached.
+
+
+#### Last step
+
+So the code is added to the fork and it seems to work. That's what I promised
+to do, but I wanted to go a little bit further and test if Mes was able to
+handle the code I added to the TinyCC fork.
+
+In order to do that I made another branch in the project where I changed the
+package and some configuration in order to compile the forked TinyCC using Mes.
+
+You can see what I did here:
+
+<https://github.com/ekaitz-zarraga/tcc/commits/mes-package>
+
+Turns out that I managed to build the thing, using Mes for my x86_64 machine
+choosing RISC-V as the backend, but it doesn't work at all.
+
+The resulting compiler generates empty files that have no permissions and fails
+instantly.
+
+At least we tested that `mescc` is ok with the C constructs we used in the
+backport of the RISC-V support. But there are still many things to test and
+this isn't easy at all.
+
+Let me give you some examples on how tricky this process is.
+
+This line in the `guix.scm` file[^line]:
+
+``` clike
+    "--extra-cflags=-Dinline= -DONE_SOURCE=1"
+```
+
+Does two crazy preprocessor tricks, inserted as C flags. It's equivalent to
+adding these macros in the top level of the sources:
+
+``` clike
+#define inline 
+#define ONE_SOURCE 1
+```
+
+The first one removes the word `inline` from the source code, because `mescc`
+does not support that. The second, defines `ONE_SOURCE` to a value because if
+it's only defined, without a value, like the makefile does by default, it is
+not matched properly by de `#ifdef`s. Finding this is not obvious.
+
+[^line]: <https://github.com/ekaitz-zarraga/tcc/blob/mes-package/guix.scm#L196>
+
+That's of course not the only thing, we found out many others. I spent a couple
+of weeks making the building process work for `mescc` and when I thought it was
+working the result is a broken binary. Pretty fun.
+
+And why all this trouble, you might think?
+
+Jan's fork is not compiled using the `configure` and the `Makefile` the project
+comes with, he wrote some shell scripts to build everything. I wanted to try to
+build the project directly as it came for several reasons: the scripts are
+prepared for native compilers and not for the cross compiler I was building,
+they use Mes from source but I just needed to use the upstream one and I
+thought integrating all this in the normal building process would be an extra
+win.
+
+I lost this time though.
+
+The compilation process might be missing some libraries, or some stubs might be
+in use instead of the real code... Maybe the problem is I'm using the x86_64
+version of Mes, which is not thoroughly tested... But using the i386 version is
+not possible because I'm building for 64bit RISC-V and the i386 doesn't know
+how to deal with 64 bit words... Honestly, I don't know what to do.
+
+### Something cool to say
+
+Mes does not compile following the classic process. Mes is integrated with some
+tools from the stage-0 project so it uses the M1 macro system, hex0 and all
+that kind of things to build the programs.
+
+During the process I found that some of the M1 instructions Mes was generating
+were not available by M1, so I had to add a few extra instructions to the M1
+macro definitions for Mes. Here's the diff (a little bit simplified) I had to
+make:
+
+
+``` diff
+diff --git a/lib/x86_64-mes/x86_64.M1 b/lib/x86_64-mes/x86_64.M1
+index 9ffbbf15..64997c55 100644
+--- a/lib/x86_64-mes/x86_64.M1
++++ b/lib/x86_64-mes/x86_64.M1
+@@ -147,6 +148,10 @@ DEFINE mov____0x8(%rbp),%rsp 488b65
+ DEFINE mov____0x8(%rdi),%rax 488b47
+ DEFINE mov____0x8(%rdi),%rbp 488b6f
+ DEFINE mov____0x8(%rdi),%rsp 488b67
++DEFINE mov____(%rax),%si 668b30
++DEFINE mov____(%rax),%sil 408a30
++DEFINE mov____%si,(%rdi) 668937
++DEFINE mov____%sil,(%rdi) 448837
+ DEFINE movl___%eax,0x32 890425
+ DEFINE movl___%edi,0x32 893c25
+ DEFINE movl___%esi,(%rdi) 8937
+
+base-commit: aa5f1533e1736a89e60d2c34c2a0ab3b01f8d037
+```
+
+Now, with those instructions added, my package got a little bit more complex:
+I had to extend the Mes package with my patch until that change is accepted
+upstream. But this is great! Using software and improving it while you use it
+is the best feeling in life![^choco]
+
+[^choco]: Chocolate and hot coffee too.
+
+Let me use this point to show you a little bit how this macro system works. You
+can see this `x86_64.M1` file has three columns: `DEFINE`, some text, and some
+number in hex. This is kind of an assembler description. There's the M1 program
+that receives a file written with instructions that look like the text in the
+second column in the `.M1` file and converts them one by one to the numbers in
+the third. In short, the `.M1` file is a reference that tells the M1 program
+how to do the conversion.
+
+M1 is just a text replacement tool that makes the conversion based on the input
+file it gets from the `.M1` file. It helps us write instructions in a way that
+looks like they have a meaning (that's what an assembler is after all).
+
+Later, those numbers are converted to binary, using Hex0 or another a little
+bit more sophisticated tool.
+
+All these tools are written in a way that can be audited (Hex0 is written in
+Hex0...) and they are executed from source at their very beginning.
+
+This is how we make yogurt directly from milk. Cool huh?
+Props to <http://bootstrappable.org/>
+
+### Conclusions
+
+Back to the project, considering the fact that I didn't manage to build a fully
+working TinyCC with a RISC-V backend using Mes, is this a failure?
+
+I wouldn't say so.
+
+The new RISC-V backend is added and tested in the forked TinyCC, using GCC as a
+compiler. That's a big chunk of the work.
+
+On the other hand, I can compile the forked TinyCC with `mescc` even if the
+result didn't work, I can say the code I added was processed so it was
+technically acceptable for `mescc`. Not bad, but we'll still need to see how
+true is this.
+
+In the end, these kind of small steps make progress, and having everything
+documented here and in the commits on the git repositories help others continue
+with what I just did.
+
+Now, I'm going to leave this as finished, as the code is supposed to work. All
+the dots are more or less drawn. Now it's time for another project, one that
+connects all the dots of the RISC-V full source bootstrap: from `mescc`
+(already has some RISC-V support) to the forked TinyCC (I added the RISC-V
+support), next to the mainline TinyCC (has RISC-V support) or/and GCC 4.6.4 (I
+added RISC-V support) and from one of those to GCC 7.5 (the first one with
+RISC-V support) and then to the world.
+
+My work in this project left all the breadcrumbs in the forest, ready for
+anyone to follow[^breadcrumbs].
+
+[^breadcrumbs]: I hope someone follows them before the birds eat them.
+
+That person can be me, anyone else or even a group of people. All I can say is
+I won't forget this project, I'll always be reachable for advice and I'd try to
+help as much as I can. As I always do.
+
+These days I'll continue to give a couple of tries to this and I may reach
+something else, but I won't be as busy on it as I've been. I think I gave
+everything I could in this project. There's still a lot to do, but what it's
+left is not something I can do alone.
+
+Until next time.