path: root/content/clopher/04-Native_interface.md

Title: TUI Slang: Speak like the natives
Date: 2019-06-15
Categories: Series
Tags: Clopher - TUI Gopher client
Slug: clopher04
Lang: en
Summary: Interfacing between Clojure, Java and Native code.


The previous post introduced `termios` as a native interface to configure the
terminal input processing. With termios we managed to make our C programs get
input character by character processing them as they came with no buffering but
we didn't integrate that with our Clojure code. Now it's time to make it.


### Run before it's too late

Before we dig in the unknown, I have to tell you there are other alternatives
for the terminal configuration. The simplest one I can imagine is using
`stty`[^1] as an external command. I learned this from [Liquid][liquid], a
really interesting project I had as a reference. If you want to see this work
check the `adapters/tty.clj` file in the `src` directory of the project.[^2]

Of course, it has some drawbacks. `stty` is part of the GNU-Coreutils project
and you have to be sure your target has it installed if you want to rely on
that. I'm not sure about if it's supported in non-GNU operating systems[^3].

In my case, I decided to stay with `termios` interface to deal with all this
because I didn't really want to rely on external commands and it's supposed to
be implemented in any POSIX OS. The good (bad?) thing is it made me deal with
native libraries from Clojure and had to learn how to do it.


### The floor is Java

When dealing with low-level stuff we have to remember Clojure is just Java, and
most of the utilities we need to use come from it. This means the question we
have to answer is not really *"how to call native code from Clojure?"* because
if we are able to call native code from Java, we will be able to do it from
Clojure too (if we spread some magic on top).

#### So, how to call native code from Java?

First I checked the [Java Native Interface (aka JNI)][jni], but I thought it
was too much for me and I decided to check further. Remember there are only a
couple of calls to make to `termios` from our code, so we don't really want to
mess with a lot of boilerplate code, compilations and so on.

My research made me find [Java Native Access (aka JNA)][jna] library. If you
check the link there you'll find that the Wikipedia[^4] describes it as:

> JNA's design aims to provide native access in a natural way with a minimum of
> effort. No boilerplate or generated glue code is required.

Sounds like right for me. Doesn't it?

I encourage you to check the full Wikipedia entry and, if you have some free
time at the office or something, to check the implementation because it's
really interesting. But I'll leave that for you.

##### A lantern in the dark

JNA is quite easy to use for the case of Clopher, even easier if you realize
there is [lanterna][lanterna], the TUI library, out there, using it internally
so you can *steal*[^5] the implementation from it. Lanterna is a great piece
of software I took as a reference for many parts of the project. Digging in the
internals of large libraries is a great exercise and you can learn a lot from
it.

First of all, like many Java projects, the amount of abstractions it has is
crazy. It takes some time to find the actual implementation of what we want.
This isn't like this for no reason, the reality is they need to create this
amount of abstractions because the part of the library that handles the widgets
can work on top of many different terminal implementations, including a
[Swing][swing] based one that comes with Lanterna itself.

Clopher only targets POSIX compatible operating systems so we can go directly
to what we want and read the termios part directly discarding all the other
compatibility code. This code is quite easy to find if you see the directory
tree of Lanterna: there's a `native-integration` folder in the root directory.
If you follow that you'll arrive to [`PosixLibC.java`][posix-lanterna] that
uses JNA to interact with termios.

The implementation provided by Lanterna is quite complete, they declare a
library with the functions they need and the data structure introduced in the
previous chapter. Once the library interface and the necessary data structures
are defined from Java they can be called with JNA, like they do in the file:
[`NativeGNULinuxTerminal.java`][nativegnu-lanterna].

#### How to call JNA from Clojure, then?

Calling Java code from Clojure is quite simple because Clojure have been
designed with that in mind, but this is not only that. Thanks to the Internet,
there's a great [blogpost by Nurullah Akkaya][nakkaya] describing a simple way
to use JNA from Clojure. From that, we can move to our specific case.

`termios` has its own data structure so we need to define it so the JNA knows
how to interact with it. The problem is that Clojure doesn't have enough OOP
tools to do it directly so we need to make it in plain Java. The good thing is
that we don't really need to create anything else.

If we remove some unneeded code from Lanterna's termios structure
implementation it will look like the implementation I made at
`src/java/clopher/Termios.java`:

    ::clike
    package clopher.termios;
    import com.sun.jna.Structure;

    import java.util.Arrays;
    import java.util.List;


    /**
     * Interface to Posix libc
     */
    public class Termios extends Structure {
        private int NCCS = 32;

        public int c_iflag;           // input mode flags
        public int c_oflag;           // output mode flags
        public int c_cflag;           // control mode flags
        public int c_lflag;           // local mode flags
        public byte c_line;           // line discipline
        public byte c_cc[];           // control characters
        public int c_ispeed;          // input speed
        public int c_ospeed;          // output speed

        public Termios() {
            c_cc = new byte[NCCS];
        }

        // This function is important for JNA, because it needs to know the
        // order of the fields of the struct in order to make a correct Java
        // class to C struct translation
        protected List<String> getFieldOrder() {
            return Arrays.asList(
                    "c_iflag",
                    "c_oflag",
                    "c_cflag",
                    "c_lflag",
                    "c_line",
                    "c_cc",
                    "c_ispeed",
                    "c_ospeed"
                    );
        }
    }

Once the struct is defined, it's time to use it from Clojure. `clopher.term`
namespace has the code to solve this. Summarized here:

    ::clojure
    (ns clopher.term
      (:import [clopher.termios Termios]
               [com.sun.jna Function]))

    (def ^:private ICANON 02)
    (def ^:private ECHO   010)
    (def ^:private ISIG   01)
    (def ^:private ECHONL 0100)
    (def ^:private IEXTEN 0100000)

    (def ^:private VTIME 5)
    (def ^:private VMIN  6)

    ; The macro we saw at the blogpost by Nurulla Akkaya
    (defmacro jna-call [lib func ret & args]
      `(let [library#  (name ~lib)
             function# (Function/getFunction library# ~func)]
         (.invoke function# ~ret (to-array [~@args]))))

    ; Wrapper for the tcgetattr function
    (defn get-config!
      []
      (let [term-conf (Termios.)]
        (if (= 0 (jna-call :c "tcgetattr" Integer 0 term-conf))
          term-conf
          (throw (UnsupportedOperationException.
                   "Impossible to get terminal configuration")))))

    ; Wrapper for the tcsetattr function
    (defn set-config!
      [term-conf]
      (when (not= 0 (jna-call :c "tcsetattr" Integer 0 0 term-conf))
        (throw (UnsupportedOperationException.
                 "Impossible to set terminal configuration"))))

    ; Example to set the non-canonical mode using the flags at the top of the
    ; file
    ; Yeah, binary operations.
    (defn set-non-canonical!
      ([]
       (set-non-canonical! true))
      ([blocking]
      (let [term-conf (get-config!)]
        (set! (.-c_lflag term-conf)
              (bit-and (.-c_lflag term-conf)
                       (bit-not (bit-or ICANON ECHO ISIG ECHONL IEXTEN))))
        (aset-byte (.-c_cc term-conf) VMIN  (if blocking 1 0))
        (aset-byte (.-c_cc term-conf) VTIME 0)
        (set-config! term-conf))))


Pay attention to all the mutable code here!
`aset-byte` function helps a lot when dealing with all that.

Be also sure to check termios' documentation because the calls act in a very
C-like way, returning a non-zero answer when they fail.

We need an extra point in our code to solve the Java-Clojure interoperability:
we have to tell our project manager that we included some Java code in there.
If our project manager is Leiningen, we can just tell it where do we store our
Java code. Be careful because Leiningen [doesn't like if you mix Java and
Clojure in the same folder][leiningen].

    ::clojure
    (defproject
      ; There's more blablabla in here but these are the keys I want you to
      ; take in account
      :source-paths      ["src/clojure"]
      :java-source-paths ["src/java"]
      :javac-options     ["-Xlint:unchecked"])


### Look back!

Now you can configure your terminal to act non-canonically and serve you the
characters one by one as they come. It's cool but you'll see there are some
problems to come for the next chapters. Don't worry! They'll come.
 
> This is like a heroic novel where the character (in this case you) fights
> monsters one by one leaving their dead corpses in the dungeon floor. Looking
> back will let you remember how many monsters did you slaughter in your way to
> the deep where the treasure awaits. Remember to take rest and sharpen your
> sword. This is a long travel.
 
> Prepare yourself for the next monster. Let the voice of the narrator guide you
> to the unknown.

Why don't you mix what you learned on the previous chapter with what you
learned from this one and try to make an interactive terminal program yourself?

I'll solve that in the next chapter, but there's some code of that part already
implemented in the repository. You can check it while I keep writing and
coding. Here's the link to the project:

[https://gitlab.com/ekaitz-zarraga/clopher][clopher]


See you in the next episode!


[^1]: Use the man pages, seriously: [`man stty`](https://linux.die.net/man/1/stty)
[^2]: I've also been in contact with Mogens, the author of the project, who is
  a really good guy and gave me a lot of good information.
[^3]: But who cares about them anyway?
[^4]: *the free encyclopedia*
[^5]: If it's free software it's not stealing and it's exactly what you are
  supposed to do with it.

[liquid]: http://salza.dk/
[jni]: https://en.wikipedia.org/wiki/Java_Native_Interface
[jna]: https://en.wikipedia.org/wiki/Java_Native_Access
[lanterna]: https://github.com/mabe02/lanterna
[swing]: https://en.wikipedia.org/wiki/Swing_(Java)
[posix-lanterna]: https://github.com/mabe02/lanterna/blob/master/native-integration/src/main/java/com/googlecode/lanterna/terminal/PosixLibC.java
[nativegnu-lanterna]: https://github.com/mabe02/lanterna/blob/263f013a2ee1d522eb86b8f1d315423fb1f79711/native-integration/src/main/java/com/googlecode/lanterna/terminal/NativeGNULinuxTerminal.java#L123
[nakkaya]: https://nakkaya.com/2009/11/16/java-native-access-from-clojure/
[leiningen]: https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L302
[clopher]: https://gitlab.com/ekaitz-zarraga/clopher