Windows 2

1 files changed, 498 insertions, 0 deletions

diff --git a/content/windows2.md b/content/windows2.md
new file mode 100644
index 0000000..b151f04
--- /dev/null
+++ b/content/windows2.md
@@ -0,0 +1,498 @@
+Title: Guix + Zig + NSIS for the win...DOWS?
+Date: 2023-12-15
+Category:
+Tags:
+Slug: windows2
+Lang: en
+Summary:
+    How I made a program for Windows and GNU/Linux without touching any Windows
+    machine. The tools and the tricks to be effective (Zig and NSIS for the
+    win).
+
+Some months later, it's time to talk about [the post I made about writing
+software for windows, without windows][win] because I put some of those things
+in practice.
+
+I made a simple application with one external audio library (OpenAL) and simple
+networking. No GUI this time, but some complexity was there.
+
+[win]: https://ekaitz.elenq.tech/windows.html
+
+### The program
+
+The program I'll discuss here is Karkarkar, a tool to read a Twitch chat out
+loud in Basque. I made this for the Basque streaming community of gamers, which
+is really cool but they had to rely on Text-To-Speech systems for other
+languages as the main services for streamers don't include Basque in their
+service, and the closest one, Spanish, is not good at some words' pronunciation
+defer.
+
+Most of these people are gamers, and they use Windows, which I don't like, but
+in the end they are also people and they deserve Free Software regardless of
+the Operating System they decide to (or are forced to) use.
+
+I took this as a cool exercise to test all we talked about in the past, and as
+a learning experience for the times I need to do something for a client that
+requires Windows support or anything like that.
+
+#### The Text-To-Speech system
+
+I used a TTS library by Aholab, a research group from the Bilbao School of
+Engineering, the university where I studied. They released AhoTTS (*ahots*
+means *voice* in Basque) some years ago in Github, a working TTS library for
+Basque and Spanish, with all the extra data it needs to work.
+
+The *only* problem it has is the AhoTTS codebase is a mess, with tons of
+horrible decisions and undercover bugs. I had to fork the lib and make it work
+more or less before adding it to the project (I won't discuss the issues here),
+but I did.
+
+#### Connection to the Twitch chat
+
+This simply uses IRC to connect to Twitch chat. The protocol is simple, and I
+only implemented an embarrassingly minimal amount of it. Enough to make it kind
+of work. I hope to make more of this in the future.
+
+#### Playing the audio
+
+I started with `libao`, a simple audio library and then moved to OpenAL for
+reasons I'll mention next.
+
+#### All together
+
+The program listens to IRC, when a message arrives it sends it to AhoTTS,
+receives the samples to play and sends them to OpenAL which does its magic to
+make it play out loud.
+
+Everything is the simplest thing possible as I didn't have a lot of time to
+spend on this and also wanted to focus on the release process and being able to
+make a package for windows, and some linux distributions. **Adding more code on
+top of that is easy once the problem of the distribution is solved.**
+
+
+### The tooling
+
+My strongest dependency[^1], AhoTTS, is written in C++, so I decided to go for
+binary file distribution. And as [I discussed in aforementioned post][win-zig],
+I was looking for an excuse to make a project in Zig, and its cross-compilation
+capabilities could help me in this case, so I went for that. I made everything
+with Zig 0.10.1, as that is the latest version packaged for Guix, but I was
+forced to move to my Zig 0.11.0 package (see [Testing](#testing)).
+
+[^1]: There are not that many Basque TTS systems out there... You know?
+
+[win-zig]: https://ekaitz.elenq.tech/windows.html#zig
+
+For binary distribution, I didn't have many ideas when I wrote the post, but a
+person in the Lisp Game Jam of that time suggested me to use NSIS for the
+Windows installer. It was already packaged in Guix, my distro of choice, so I
+just went for that, as it looked the simplest way to solve this.
+
+For testing all this, I relied in `wine`, what else could I do?
+
+So tl;dr:
+
+- I'm coding on [Guix](https://guix.gnu.org/).
+- Programming in [Zig](https://ziglang.org/) 0.10.1 but then moved to
+  [Zig](https://ziglang.org/) 0.11.0.
+- Installer using [NSIS](https://nsis.sourceforge.io/Main_Page).
+- Testing done with [Wine64](https://www.winehq.org/).
+
+
+### Keeping it small
+
+First of all, publishing software for Windows from Linux is painful if you have
+to compile everything for it. Guix helps a little bit with that as you can use
+`--target=` with mingw and have some luck. Sadly, many packages (most of them)
+need `bash-minimal`, which [is not buildable for mingw at the
+moment][issue-mingw].
+
+[issue-mingw]: https://issues.guix.gnu.org/62226
+
+In many software projects cross-compilation is not even possible.
+
+Knowing that I decided to keep my project as small as possible, because I
+wanted to actually deliver something without losing my sanity.
+
+Each library you depend on you have to compile and deliver to your target, too.
+It's not that common to have Windows users to install things by themselves as a
+GNU/Linux person would.
+
+#### Audio library
+
+`libao` is the smallest audio library I could find, but I didn't manage to
+build it for Windows myself, so I had to rely on something else. The
+OpenAL-Soft maintainers have a binary distribution for windows, so I decided to
+go for that one instead. It's way harder to use, and I had to do some weird
+stuff to make it work, but it's easier for me: no need to build it myself.
+
+This is even more important in the case of having an audio library, interacting
+with the system is always a pain in the ass, and trying to cross-compile a
+library like this is not always easy, as their configure scripts are complex as
+they need to check for many things.
+
+This is why I also avoided a GUI for the moment. Too much.
+
+#### AhoTTS
+
+AhoTTS is written in C++, it has zero dependencies and uses CMake as a build
+system.
+
+At the beginning I compiled it using Guix to mingw, because it doesn't have
+dependencies, it just worked. Later I encountered some issues though:
+`libstdc++` and `libgcc` were not found at runtime and I had to statically link
+them (`-static-libgcc -static-libstdc++` ftw).
+
+Later I started digging in its code, fixing some **horrible** things inside and
+then I managed to add it as a submodule (yes, I hate that too) and compile it
+directly with Zig. This is the best option so far: you can statically link the
+library, it is built with the same process and it is cross-compiled by Zig. No
+more missing library problems.
+
+> Extra: In order to interact with AhoTTS from Zig, I had to make [a small C++
+> to C bridge][c++-c], that I never did before. It's easy stuff, just convert
+> the API a little bit, put some `extern "C"` on it and everything will go
+> fine. `void *` is your friend.
+
+[c++-c]: https://github.com/ekaitz-zarraga/karkarkar/blob/master/ahotts_c/htts.cpp
+
+#### The rest of it
+
+That's just writing some actual code and making the program run. That's the
+easiest part.
+
+
+### Bringing it to the users
+
+One thing is making something that builds and runs in your computer and a very
+different thing is making it work in other people's computers. And this is the
+main problem that I wanted to discuss here.
+
+I have some requirements:
+
+- I made a terminal application, but I don't expect my users to know how to run
+  it. I need to make something that is click and run.
+- I need it to have some icon in the desktop/startup-menu.
+- The AhoTTS library needs some extra data files. These are searched by the
+  library and need to have some specific structure. These need to be installed
+  properly, too.
+- I need to provide a simple way to uninstall the application.
+
+#### Windows
+
+The cool thing of this is I don't own a windows machine since ~2010 and I have
+no access to any (living the good life!). I don't have any plans to change that
+neither I have plans to really learn about Windows. So we have to be clever to
+solve this.
+
+First things first:
+
+``` bash
+zig build -Dcpu=baseline -Dtarget=x86_64-windows-gnu
+```
+
+Damn! That was actually very easy to do. Why isn't this the norm in other
+languages?
+
+Of course, in order to do this I needed to provide a proper `build.zig` file
+that was able to find the DLLs I was linking against and the header files.
+That's not that difficult after all[^dll-names]. But has to be done. Still,
+easy. Kudos for Zig.
+
+[^dll-names]: I still had weird issues with this. In Zig 0.10.1 DLLs where
+    found even if their name started with `lib`, but moving to Zig 0.11.0
+    didn't find the libraries starting with `lib`. But Wine did find them so I
+    didn't know what to do. When I added the AhoTTS as a submodule the problem
+    disappeared, but not because it was solved, it was just avoided.
+
+Now, knowing where Windows searches the DLLs we depend on is important as our
+installation process will depend on it. They have [some good documentation for
+that][win-dll] which has a very interesting point:
+
+> Standard search order for unpackaged apps:  
+> ...  
+> 7. The folder from which the application loaded.
+
+This seems good enough for my purpose, so I can just install my binary and the
+libraries in the very same folder. I thought I would need to install stuff
+mixed somewhere else and learn a lot about windows, but I didn't have to.
+Simple!
+
+[win-dll]: https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-unpackaged-apps
+
+The extra data can be stored anywhere I want, because I am the one that
+controls the search algorithm, but I still need to have easy access to it. I
+decided to go for `LOCALAPPDATA`, but I'm thinking on putting it in the same
+folder as the rest of the program. At the time of writing that's not done yet.
+
+##### NSIS
+
+Having clear where everything should be installed, it's time to make it
+actually install it.
+
+NSIS is a great tool. The look of the vanilla installer is old-school but I
+didn't even bother to use the modern interface because that required me to
+think, activity I like to reserve to special occasions (like when I'm paid
+extreme amounts of money for it or the time I'm in bed before falling asleep).
+
+NSIS in a nutshell: you write a script, run `makensis` with the script as an
+input and, boom! You have a `.exe` that installs your stuff.
+
+It took me a little while to understand the structure of the script but once
+you learn it is really easy (for the basic things, after that you can go as
+hard as you want). It has two concepts you need to understand: Pages and
+Sections.
+
+- Pages: define the different *pages* the user will navigate through during the
+  process. There are many pages pre-defined, and they cover all the basic
+  functionality: license agreement, component selection, installation directory
+  selection...
+- Sections: define the different parts of your installation process. You can
+  mark some as optional or include them in different installation profiles
+  (all, minimal, recommended... You've seen this before). Then, if you use
+  `components` page, the sections will be listed to the user to choose which
+  one they want to install. You can also add sections for the uninstaller,
+  which will only be run when the user uninstalls the program.
+
+> EXAMPLE: I decided to include the sources also in the installer, but they are
+> not required to run the program. It's as simple as adding a Section with the
+> `/o` flag and they won't be automatically checked in the components step.
+> Really cool stuff!
+
+The rest of the thing is just commands you can read in the documentation. You
+can do many-many things with it. It has environment variables for almost
+anything you'll need, so you don't need to hardcode things in the script.
+
+In my case I decided to ask the user for a configuration (the Twitch username)
+during the installation using a custom page (this requires some digging in the
+plugins' documentation, but it's not hard either), and created a launcher that
+automagically inserts the username in the call to the program (not great for
+later configuration, I know). This is done with a Batch file, which also makes
+the dirty job of opening the terminal when it's double clicked.
+
+Here's the installer script I did, if you want to read it:
+
+<https://github.com/ekaitz-zarraga/karkarkar/blob/master/windows/installer.nsi>
+
+
+#### GNU/Linux
+
+GNU/Linux world is really diverse, so it's not easy to know about every single
+system's requirements. At the moment I stayed with Guix and Debian because they
+are the only distros I use and I'm more familiar with them.
+
+In Windows I was asking to the user to add their username in the installation
+process, but in Linux I don't have any simple (for the user) way to do it, so
+for the moment I ask for the username when the program is called with no input
+arguments. Ugly, but works for the moment. The goal was to deliver this thing,
+not to make it perfect. I can do that later.
+
+The cool part is they provide different approaches for packaging: source vs
+binary distribution.
+
+##### XDG standard: Desktop file and icons
+
+We need, of course, to arrange the same things we arranged in Windows: the
+desktop icon and launching the terminal automatically. That's not hard to do
+using XDG specification!
+
+Just add the `Terminal=true` line and it should open a terminal emulator when
+clicked[^should]. The `Exec=` line in the desktop file has the program you want
+to run, and it has to point to it correctly.
+
+[^should]: *"Should"* is the best word choice here, because it doesn't work for
+    some people, like in GNOME, as the terminals are hardcoded:  
+    <https://gitlab.gnome.org/GNOME/glib/-/blob/main/gio/gdesktopappinfo.c#L2685>
+
+Once the `.desktop` file is done, we realize we need to deal with the icons. I
+went just for a SVG icon, but I could add the rest. The only thing I needed to
+do is put everything in the correct folder. Something like this:
+
+``` bash
+ usr
+ └── share
+     ├── applications
+     │   └── karkarkar.desktop
+     └── icons
+         └── hicolor
+             └── scalable
+                 └── apps
+                     ├── karkarkar.svg
+                     └── karkarkar-symbolic.svg
+```
+
+Both Guix and Debian detect them properly once they are installed in the folder
+where they expect them.
+
+##### Guix
+
+I wrote quite a few Guix packages lately so I'm pretty comfortable with this.
+
+I just need to tell Guix how it should build the program and put some files in
+the proper directory.
+
+Also, I added the `zig-build-system` myself to Guix not that long ago. It's
+pretty straightforward to use.
+
+The icons and the desktop file needs to put everything in the correct place,
+`#$output/share/...`, and patch the Desktop file to point to the correct
+binary, `#$output/bin/...` that is. For this, I kept a desktop file as a
+template, with some reasonable defaults and just patched it in the Guix
+package. That's easy.
+
+Not the best Guix package ever but it simply works, and that's everything I
+want at this point.
+
+##### Debian
+
+Debian packages can be exported from Guix package definitions, but it exports
+its file structure with every single dependency. In our case, that meant
+hundreds of MegaBytes. That's too much so I did it manually, and the final size
+was around 10 MegaBytes. Not bad.
+
+I never made a Debian package before, and I did the most minimalistic thing I
+could think of.
+
+First, we need to build the thing:
+
+``` bash
+zig build -Dcpu=baseline
+```
+
+And then just place everything in place in a folder, and call `dpkg-deb` on top
+of it. In order to do that, I just wrote a bash script that did this whole
+thing, it explains what I did way better than I can write in English:
+
+``` bash
+# Run me in the linux/ folder
+# I need the version in an argument, which should be Major.Minor-Revision
+# for example 0.1-3.
+version=$1
+outfolder="karkarkar-$version"
+mkdir "$outfolder"
+mkdir -p "$outfolder/usr/bin"
+mkdir -p "$outfolder/usr/share/applications"
+mkdir -p "$outfolder/usr/share/AhoTTS/"
+mkdir -p "$outfolder/usr/share/icons/hicolor/scalable/apps"
+
+cp -r "../AhoTTS/data_tts"       "$outfolder/usr/share/AhoTTS/"
+cp    "karkarkar.desktop"        "$outfolder/usr/share/applications"
+cp    "../icons/karkarkar.svg"   "$outfolder/usr/share/icons/hicolor/scalable/apps"
+cp    "../zig-out/bin/karkarkar" "$outfolder/usr/bin/"
+patchelf "$outfolder/usr/bin/karkarkar" --set-interpreter "/lib64/ld-linux-x86-64.so.2"
+
+mkdir -p "$outfolder/DEBIAN"
+cat > $outfolder/DEBIAN/control <<EOF
+Package: karkarkar
+Version: $version
+Section: base
+Priority: optional
+Architecture: amd64
+Depends: libopenal1 (>=1.19.1)
+Maintainer: Ekaitz Zarraga <blablablah>
+Description: Karkarkar
+ Listen to a Twitch chat in Basque.
+EOF
+
+dpkg-deb --root-owner-group --build "$outfolder"
+rm -rf "$outfolder"
+```
+
+I need to highlight here the `zig build` I did automatically adds the Guix
+dynamic linker to the binary, but that is not where the dynamic linker  is in
+Debian. I decided to patch the binary (`patchelf`) instead of trying to
+configure the compilation process, I thought this would be easier. I don't know
+if it was easier or not, but it was easy, so that's ok.
+
+Also note that the `DEBIAN/control` file has the bare minimum fields, but it's
+enough to work. In Debian, everything is installed in `/usr/whatever`, but
+that's the only detail I changed.
+
+Something I want to write down to remember later is the debian packages (`.deb`
+files) can be extracted with `ar -x` and they have a couple of `tar.xz` files
+inside. The `data.tar.xz` file has the file structure that will later installed
+in the system.
+
+
+### Testing {#testing}
+
+Yeah, you have to do it too. I tested it in Wine, but still needed to be tested
+in Windows. I had a working version done in Zig 0.10.0 that was running well in
+Wine, but it exploded in Windows because of [this issue][issue], that didn't
+happen in Wine. I needed to use Zig 0.11 because of this error, which wasn't a
+big deal anyway, because I already had it more or less packaged.
+
+So, yes, you have to test in Windows just in case, if you can. I have to thank
+my friend (you know who you are!) for testing this program in his computer and
+reporting the error.
+
+[issue]: https://github.com/ziglang/zig/issues/8943
+
+### Conclusions
+
+The program itself it's really underdeveloped, but I actually made a lot of
+progress understanding how to make a program reach users in different systems
+easily. In the end, **being a solo developer forces me to be clever, and do
+everything as simple as I can**.
+
+I don't normally have many dependencies because I believe software has to be
+simple and tailor-made. This makes me control every aspect of the projects I do
+but also greatly simplify the distribution. That's my context.
+
+I already talked about this in the [previous post on the subject][win], but in
+the end, being able to do this relieves me from the "Web is Cross-platform"
+mentality, which I don't think is a silver bullet, even though in some cases
+might be a simple solution.
+
+I think developers today tend to avoid making native applications and software
+quality suffers from that. There are many reasons for this (corporate control,
+subscription systems, easy deployments, controlled environments...) but one
+might be that the code people is used to write has too many dependencies, and
+it is hard to package and distribute. I don't usually have that problem. I
+already said I don't like having many dependencies.
+
+Now I have an easy way to do this, I can simply focus on writing the actual
+code, which is, in the end, the most important part.
+
+Key points:
+
+- **Zig** happened to make this process really simple, just because the
+  developers decided to make it simple and they had the engineering skill to
+  back the decision.
+
+- **NSIS** gave me all I needed to put my application in a windows machine
+  without learning almost anything about Windows. I have the enough information
+  to make the thing work, not more. **NSIS** was the key I was missing.
+
+- **Guix** lets me cross-compile some of the dependencies to mingw, and I did
+  that at the beginning (and it worked!) so it's a powerful tool even for that.
+
+- **Wine** is a good way to see everything was working well, but I found some
+  discrepancies between it and Windows.
+
+The full problem is not completely solved yet. Finding a **GUI toolkit** I can
+cross compile easily is still in the TODO list. But I'm pretty satisfied when
+the result until now.
+
+> My colleague Andrius Štikonas talked to me about
+> [MXE](https://github.com/mxe/mxe). I may try it in the future but I don't
+> like the fact that it downloads things by itself. I leave it here just in the
+> case I need it in the future.
+
+In fact, I think I can try other programming languages, even interpreted ones.
+**NSIS** is surely capable of dealing with it. This opens a whole world of
+possibilities for me.
+
+In the end, everything has been a very interesting process, making applications
+targeted to the non-programmer (and non-gnu/linux) user has always been in my
+mind. Now, I can say I almost solved the problem and I have the base to work
+more on this in the future.
+
+Finally, the program is released in a very alpha stage in
+[itch.io](https://ekaitz-zarraga.itch.io/karkarkar) where you can find the
+installers and packages and the code is hosted in
+[Github](https://github.com/ekaitz-zarraga/karkarkar/) until I get completely
+mad and I delete my account entirely (which I have no doubt it will happen one
+day or another).
+
+And that's mostly it, now I can focus on extending the behavior.