path: root/content/windows2.md

Title: Guix + Zig + NSIS for the win...DOWS?
Date: 2023-12-15
Category:
Tags:
Slug: windows2
Lang: en
Summary:
    How I made a program for Windows and GNU/Linux without touching any Windows
    machine. The tools and the tricks to be effective (Zig and NSIS for the
    win).

Some months later, it's time to talk about [the post I made about writing
software for windows, without windows][win] because I put some of those things
in practice.

I made a simple application with one external audio library (OpenAL) and simple
networking. No GUI this time, but some complexity was there.

[win]: https://ekaitz.elenq.tech/windows.html

### The program

The program I'll discuss here is Karkarkar, a tool to read a Twitch chat out
loud in Basque. I made this for the Basque streaming community of gamers, which
is really cool but they had to rely on Text-To-Speech systems for other
languages as the main services for streamers don't include Basque in their
service, and the closest one, Spanish, is not good at some words' pronunciation
defer.

Most of these people are gamers, and they use Windows, which I don't like, but
in the end they are also people and they deserve Free Software regardless of
the Operating System they decide to (or are forced to) use.

I took this as a cool exercise to test all we talked about in the past, and as
a learning experience for the times I need to do something for a client that
requires Windows support or anything like that.

#### The Text-To-Speech system

I used a TTS library by Aholab, a research group from the Bilbao School of
Engineering, the university where I studied. They released AhoTTS (*ahots*
means *voice* in Basque) some years ago in Github, a working TTS library for
Basque and Spanish, with all the extra data it needs to work.

The *only* problem it has is the AhoTTS codebase is a mess, with tons of
horrible decisions and undercover bugs. I had to fork the lib and make it work
more or less before adding it to the project (I won't discuss the issues here),
but I did.

#### Connection to the Twitch chat

This simply uses IRC to connect to Twitch chat. The protocol is simple, and I
only implemented an embarrassingly minimal amount of it. Enough to make it kind
of work. I hope to make more of this in the future.

#### Playing the audio

I started with `libao`, a simple audio library and then moved to OpenAL for
reasons I'll mention next.

#### All together

The program listens to IRC, when a message arrives it sends it to AhoTTS,
receives the samples to play and sends them to OpenAL which does its magic to
make it play out loud.

Everything is the simplest thing possible as I didn't have a lot of time to
spend on this and also wanted to focus on the release process and being able to
make a package for windows, and some linux distributions. **Adding more code on
top of that is easy once the problem of the distribution is solved.**


### The tooling

My strongest dependency[^1], AhoTTS, is written in C++, so I decided to go for
binary file distribution. And as [I discussed in aforementioned post][win-zig],
I was looking for an excuse to make a project in Zig, and its cross-compilation
capabilities could help me in this case, so I went for that. I made everything
with Zig 0.10.1, as that is the latest version packaged for Guix, but I was
forced to move to my Zig 0.11.0 package (see [Testing](#testing)).

[^1]: There are not that many Basque TTS systems out there... You know?

[win-zig]: https://ekaitz.elenq.tech/windows.html#zig

For binary distribution, I didn't have many ideas when I wrote the post, but a
person in the Lisp Game Jam of that time suggested me to use NSIS for the
Windows installer. It was already packaged in Guix, my distro of choice, so I
just went for that, as it looked the simplest way to solve this.

For testing all this, I relied in `wine`, what else could I do?

So tl;dr:

- I'm coding on [Guix](https://guix.gnu.org/).
- Programming in [Zig](https://ziglang.org/) 0.10.1 but then moved to
  [Zig](https://ziglang.org/) 0.11.0.
- Installer using [NSIS](https://nsis.sourceforge.io/Main_Page).
- Testing done with [Wine64](https://www.winehq.org/).


### Keeping it small

First of all, publishing software for Windows from Linux is painful if you have
to compile everything for it. Guix helps a little bit with that as you can use
`--target=` with mingw and have some luck. Sadly, many packages (most of them)
need `bash-minimal`, which [is not buildable for mingw at the
moment][issue-mingw].

[issue-mingw]: https://issues.guix.gnu.org/62226

In many software projects cross-compilation is not even possible.

Knowing that I decided to keep my project as small as possible, because I
wanted to actually deliver something without losing my sanity.

Each library you depend on you have to compile and deliver to your target, too.
It's not that common to have Windows users to install things by themselves as a
GNU/Linux person would.

#### Audio library

`libao` is the smallest audio library I could find, but I didn't manage to
build it for Windows myself, so I had to rely on something else. The
OpenAL-Soft maintainers have a binary distribution for windows, so I decided to
go for that one instead. It's way harder to use, and I had to do some weird
stuff to make it work, but it's easier for me: no need to build it myself.

This is even more important in the case of having an audio library, interacting
with the system is always a pain in the ass, and trying to cross-compile a
library like this is not always easy, as their configure scripts are complex as
they need to check for many things.

This is why I also avoided a GUI for the moment. Too much.

#### AhoTTS

AhoTTS is written in C++, it has zero dependencies and uses CMake as a build
system.

At the beginning I compiled it using Guix to mingw, because it doesn't have
dependencies, it just worked. Later I encountered some issues though:
`libstdc++` and `libgcc` were not found at runtime and I had to statically link
them (`-static-libgcc -static-libstdc++` ftw).

Later I started digging in its code, fixing some **horrible** things inside and
then I managed to add it as a submodule (yes, I hate that too) and compile it
directly with Zig. This is the best option so far: you can statically link the
library, it is built with the same process and it is cross-compiled by Zig. No
more missing library problems.

> Extra: In order to interact with AhoTTS from Zig, I had to make [a small C++
> to C bridge][c++-c], that I never did before. It's easy stuff, just convert
> the API a little bit, put some `extern "C"` on it and everything will go
> fine. `void *` is your friend.

[c++-c]: https://github.com/ekaitz-zarraga/karkarkar/blob/master/ahotts_c/htts.cpp

#### The rest of it

That's just writing some actual code and making the program run. That's the
easiest part.


### Bringing it to the users

One thing is making something that builds and runs in your computer and a very
different thing is making it work in other people's computers. And this is the
main problem that I wanted to discuss here.

I have some requirements:

- I made a terminal application, but I don't expect my users to know how to run
  it. I need to make something that is click and run.
- I need it to have some icon in the desktop/startup-menu.
- The AhoTTS library needs some extra data files. These are searched by the
  library and need to have some specific structure. These need to be installed
  properly, too.
- I need to provide a simple way to uninstall the application.

#### Windows

The cool thing of this is I don't own a windows machine since ~2010 and I have
no access to any (living the good life!). I don't have any plans to change that
neither I have plans to really learn about Windows. So we have to be clever to
solve this.

First things first:

``` bash
zig build -Dcpu=baseline -Dtarget=x86_64-windows-gnu
```

Damn! That was actually very easy to do. Why isn't this the norm in other
languages?

Of course, in order to do this I needed to provide a proper `build.zig` file
that was able to find the DLLs I was linking against and the header files.
That's not that difficult after all[^dll-names]. But has to be done. Still,
easy. Kudos for Zig.

[^dll-names]: I still had weird issues with this. In Zig 0.10.1 DLLs where
    found even if their name started with `lib`, but moving to Zig 0.11.0
    didn't find the libraries starting with `lib`. But Wine did find them so I
    didn't know what to do. When I added the AhoTTS as a submodule the problem
    disappeared, but not because it was solved, it was just avoided.

Now, knowing where Windows searches the DLLs we depend on is important as our
installation process will depend on it. They have [some good documentation for
that][win-dll] which has a very interesting point:

> Standard search order for unpackaged apps:  
> ...  
> 7. The folder from which the application loaded.

This seems good enough for my purpose, so I can just install my binary and the
libraries in the very same folder. I thought I would need to install stuff
mixed somewhere else and learn a lot about windows, but I didn't have to.
Simple!

[win-dll]: https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-unpackaged-apps

The extra data can be stored anywhere I want, because I am the one that
controls the search algorithm, but I still need to have easy access to it. I
decided to go for `LOCALAPPDATA`, but I'm thinking on putting it in the same
folder as the rest of the program. At the time of writing that's not done yet.

##### NSIS

Having clear where everything should be installed, it's time to make it
actually install it.

NSIS is a great tool. The look of the vanilla installer is old-school but I
didn't even bother to use the modern interface because that required me to
think, activity I like to reserve to special occasions (like when I'm paid
extreme amounts of money for it or the time I'm in bed before falling asleep).

NSIS in a nutshell: you write a script, run `makensis` with the script as an
input and, boom! You have a `.exe` that installs your stuff.

It took me a little while to understand the structure of the script but once
you learn it is really easy (for the basic things, after that you can go as
hard as you want). It has two concepts you need to understand: Pages and
Sections.

- Pages: define the different *pages* the user will navigate through during the
  process. There are many pages pre-defined, and they cover all the basic
  functionality: license agreement, component selection, installation directory
  selection...
- Sections: define the different parts of your installation process. You can
  mark some as optional or include them in different installation profiles
  (all, minimal, recommended... You've seen this before). Then, if you use
  `components` page, the sections will be listed to the user to choose which
  one they want to install. You can also add sections for the uninstaller,
  which will only be run when the user uninstalls the program.

> EXAMPLE: I decided to include the sources also in the installer, but they are
> not required to run the program. It's as simple as adding a Section with the
> `/o` flag and they won't be automatically checked in the components step.
> Really cool stuff!

The rest of the thing is just commands you can read in the documentation. You
can do many-many things with it. It has environment variables for almost
anything you'll need, so you don't need to hardcode things in the script.

In my case I decided to ask the user for a configuration (the Twitch username)
during the installation using a custom page (this requires some digging in the
plugins' documentation, but it's not hard either), and created a launcher that
automagically inserts the username in the call to the program (not great for
later configuration, I know). This is done with a Batch file, which also makes
the dirty job of opening the terminal when it's double clicked.

Here's the installer script I did, if you want to read it:

<https://github.com/ekaitz-zarraga/karkarkar/blob/master/windows/installer.nsi>


#### GNU/Linux

GNU/Linux world is really diverse, so it's not easy to know about every single
system's requirements. At the moment I stayed with Guix and Debian because they
are the only distros I use and I'm more familiar with them.

In Windows I was asking to the user to add their username in the installation
process, but in Linux I don't have any simple (for the user) way to do it, so
for the moment I ask for the username when the program is called with no input
arguments. Ugly, but works for the moment. The goal was to deliver this thing,
not to make it perfect. I can do that later.

The cool part is they provide different approaches for packaging: source vs
binary distribution.

##### XDG standard: Desktop file and icons

We need, of course, to arrange the same things we arranged in Windows: the
desktop icon and launching the terminal automatically. That's not hard to do
using XDG specification!

Just add the `Terminal=true` line and it should open a terminal emulator when
clicked[^should]. The `Exec=` line in the desktop file has the program you want
to run, and it has to point to it correctly.

[^should]: *"Should"* is the best word choice here, because it doesn't work for
    some people, like in GNOME, as the terminals are hardcoded:  
    <https://gitlab.gnome.org/GNOME/glib/-/blob/main/gio/gdesktopappinfo.c#L2685>

Once the `.desktop` file is done, we realize we need to deal with the icons. I
went just for a SVG icon, but I could add the rest. The only thing I needed to
do is put everything in the correct folder. Something like this:

``` bash
 usr
 └── share
     ├── applications
     │   └── karkarkar.desktop
     └── icons
         └── hicolor
             └── scalable
                 └── apps
                     ├── karkarkar.svg
                     └── karkarkar-symbolic.svg
```

Both Guix and Debian detect them properly once they are installed in the folder
where they expect them.

##### Guix

I wrote quite a few Guix packages lately so I'm pretty comfortable with this.

I just need to tell Guix how it should build the program and put some files in
the proper directory.

Also, I added the `zig-build-system` myself to Guix not that long ago. It's
pretty straightforward to use.

The icons and the desktop file needs to put everything in the correct place,
`#$output/share/...`, and patch the Desktop file to point to the correct
binary, `#$output/bin/...` that is. For this, I kept a desktop file as a
template, with some reasonable defaults and just patched it in the Guix
package. That's easy.

Not the best Guix package ever but it simply works, and that's everything I
want at this point.

##### Debian

Debian packages can be exported from Guix package definitions, but it exports
its file structure with every single dependency. In our case, that meant
hundreds of MegaBytes. That's too much so I did it manually, and the final size
was around 10 MegaBytes. Not bad.

I never made a Debian package before, and I did the most minimalistic thing I
could think of.

First, we need to build the thing:

``` bash
zig build -Dcpu=baseline
```

And then just place everything in place in a folder, and call `dpkg-deb` on top
of it. In order to do that, I just wrote a bash script that did this whole
thing, it explains what I did way better than I can write in English:

``` bash
# Run me in the linux/ folder
# I need the version in an argument, which should be Major.Minor-Revision
# for example 0.1-3.
version=$1
outfolder="karkarkar-$version"
mkdir "$outfolder"
mkdir -p "$outfolder/usr/bin"
mkdir -p "$outfolder/usr/share/applications"
mkdir -p "$outfolder/usr/share/AhoTTS/"
mkdir -p "$outfolder/usr/share/icons/hicolor/scalable/apps"

cp -r "../AhoTTS/data_tts"       "$outfolder/usr/share/AhoTTS/"
cp    "karkarkar.desktop"        "$outfolder/usr/share/applications"
cp    "../icons/karkarkar.svg"   "$outfolder/usr/share/icons/hicolor/scalable/apps"
cp    "../zig-out/bin/karkarkar" "$outfolder/usr/bin/"
patchelf "$outfolder/usr/bin/karkarkar" --set-interpreter "/lib64/ld-linux-x86-64.so.2"

mkdir -p "$outfolder/DEBIAN"
cat > $outfolder/DEBIAN/control <<EOF
Package: karkarkar
Version: $version
Section: base
Priority: optional
Architecture: amd64
Depends: libopenal1 (>=1.19.1)
Maintainer: Ekaitz Zarraga <blablablah>
Description: Karkarkar
 Listen to a Twitch chat in Basque.
EOF

dpkg-deb --root-owner-group --build "$outfolder"
rm -rf "$outfolder"
```

I need to highlight here the `zig build` I did automatically adds the Guix
dynamic linker to the binary, but that is not where the dynamic linker  is in
Debian. I decided to patch the binary (`patchelf`) instead of trying to
configure the compilation process, I thought this would be easier. I don't know
if it was easier or not, but it was easy, so that's ok.

Also note that the `DEBIAN/control` file has the bare minimum fields, but it's
enough to work. In Debian, everything is installed in `/usr/whatever`, but
that's the only detail I changed.

Something I want to write down to remember later is the debian packages (`.deb`
files) can be extracted with `ar -x` and they have a couple of `tar.xz` files
inside. The `data.tar.xz` file has the file structure that will later installed
in the system.


### Testing {#testing}

Yeah, you have to do it too. I tested it in Wine, but still needed to be tested
in Windows. I had a working version done in Zig 0.10.0 that was running well in
Wine, but it exploded in Windows because of [this issue][issue], that didn't
happen in Wine. I needed to use Zig 0.11 because of this error, which wasn't a
big deal anyway, because I already had it more or less packaged.

So, yes, you have to test in Windows just in case, if you can. I have to thank
my friend (you know who you are!) for testing this program in his computer and
reporting the error.

[issue]: https://github.com/ziglang/zig/issues/8943

### Conclusions

The program itself it's really underdeveloped, but I actually made a lot of
progress understanding how to make a program reach users in different systems
easily. In the end, **being a solo developer forces me to be clever, and do
everything as simple as I can**.

I don't normally have many dependencies because I believe software has to be
simple and tailor-made. This makes me control every aspect of the projects I do
but also greatly simplify the distribution. That's my context.

I already talked about this in the [previous post on the subject][win], but in
the end, being able to do this relieves me from the "Web is Cross-platform"
mentality, which I don't think is a silver bullet, even though in some cases
might be a simple solution.

I think developers today tend to avoid making native applications and software
quality suffers from that. There are many reasons for this (corporate control,
subscription systems, easy deployments, controlled environments...) but one
might be that the code people is used to write has too many dependencies, and
it is hard to package and distribute. I don't usually have that problem. I
already said I don't like having many dependencies.

Now I have an easy way to do this, I can simply focus on writing the actual
code, which is, in the end, the most important part.

Key points:

- **Zig** happened to make this process really simple, just because the
  developers decided to make it simple and they had the engineering skill to
  back the decision.

- **NSIS** gave me all I needed to put my application in a windows machine
  without learning almost anything about Windows. I have the enough information
  to make the thing work, not more. **NSIS** was the key I was missing.

- **Guix** lets me cross-compile some of the dependencies to mingw, and I did
  that at the beginning (and it worked!) so it's a powerful tool even for that.

- **Wine** is a good way to see everything was working well, but I found some
  discrepancies between it and Windows.

The full problem is not completely solved yet. Finding a **GUI toolkit** I can
cross compile easily is still in the TODO list. But I'm pretty satisfied when
the result until now.

> My colleague Andrius Štikonas talked to me about
> [MXE](https://github.com/mxe/mxe). I may try it in the future but I don't
> like the fact that it downloads things by itself. I leave it here just in the
> case I need it in the future.

In fact, I think I can try other programming languages, even interpreted ones.
**NSIS** is surely capable of dealing with it. This opens a whole world of
possibilities for me.

In the end, everything has been a very interesting process, making applications
targeted to the non-programmer (and non-gnu/linux) user has always been in my
mind. Now, I can say I almost solved the problem and I have the base to work
more on this in the future.

Finally, the program is released in a very alpha stage in
[itch.io](https://ekaitz-zarraga.itch.io/karkarkar) where you can find the
installers and packages and the code is hosted in
[Github](https://github.com/ekaitz-zarraga/karkarkar/) until I get completely
mad and I delete my account entirely (which I have no doubt it will happen one
day or another).

And that's mostly it, now I can focus on extending the behavior.