Title: Down the <del>rabbit</del> gopher hole
Date: 2019-05-07
Categories: Series
Tags: Clopher - TUI Gopher client
Slug: clopher02
Lang: en
Summary: Working on the Gopher protocol implementation and opening the door to
    the future problems.

> Read all the posts in [Clopher series here]({tag}clopher-tui-gopher-client).


As the project's goal was to create a Gopher client, it was time to understand
something about the protocol and read the [RFC][gopher]. No need for you to
know the protocol to understand what I'm going to say here. I think I already
did the difficult part for you.

### Understand some Gopher

Gopher is a really simple protocol (this doesn't mean I implemented it
correctly anyway). It's assumed to work on top of TCP (default port is 70) and
it's as simple as creating a socket, sending the *selector string* to it, and
reading everything from it until it closes. That's in most of the cases how it
works.

It has two major ways to access the data:

1. **Text mode**, which is used in most of the queries, needs the client to
   read from the socket until a line with a single dot (`.`) appears[^1].

2. **Binary mode**, expects the client to read from the socket until the server
   closes it.

Easy-peasy.

Gopher is a stateless protocol and that helps a lot during the creation of the
client. There's no need to retain data or anything related.

*Selector strings* are what client wants to see. In order to know what
selections are possible, Gopher defines an specific text format that works as a
menu, and it's called, unsurprisingly, *Menu*.

Menus have a description of the content, the address or hostname, the port, the
selector string, and a number that indicates the type of each of its elements
separated by a TAB (aka "\t" character). Each element in one line[^1].

Pay attention to the fact that each menu entry contains an address and a port,
that means it can be pointing to a different server!

The *type* further than making the client choose between *binary* and *text*
mode also gives the client information about what kind of response it's going
to get from it: if it's a menu, an image, an audio file... It also says if the
element is a *search endpoint*[^2].

Yes, Gopher supports searches!

Well, Gopher supports tons of things because the only rule is that all the
logic is on the server side. You can do whatever you want, if you do it on the
server.

Searching is as simple as asking for a text document, but it adds also the
search query to the equation. During a search, the client needs to send the
*selector string* to select the endpoint and then the *search string*
(separated by a `TAB` character).

There are some points more but this is more than enough for the moment.

Let's make something work.

### Make Gopher queries work

A simple text request can be understood like this piece of Clojure code here:

    ::clojure
    ; Define the function to make the queries
    (defn send-text-request
      [host port body]
      (with-open [sock (java.net.Socket. host port)
                  writer (clojure.java.io/writer sock)
                  reader (clojure.java.io/reader sock)
                  response (java.io.StringWriter.)]
        (.append writer body)
        (.flush writer)
        (clojure.java.io/copy reader response)
        (str response)))

    ; Make a query and print the result
    (println (send-text-request "tilde.team" 70 (str "~giacomo" "\r\n"))

As you see, it's not waiting to the dot at the end of the file and it's not
doing any kind of parsing, error checking or timeout handling, but it works.
This a minimal (and ugly, clean the namespaces!) implementation for you to be
able to run it in the REPL.

The binary is almost the same but the output must be handled in a different
way. As Clopher is a terminal based application I made it store the answer in a
file.

There's a simple and beautiful way to handle temporary files in Java that you
can access from Clojure. As I wasn't a Java user before I didn't know this:

    ::clojure
    (defn- ->tempfile
     "Creates a temporary file with the provided extension. If extension is nil
     it adds `.tmp`."
     [extension]
     (doto
      (. java.io.File createTempFile "clopher" extension)
      .deleteOnExit))

With this function is really simple to create a temporary file and copy the
download there. It's also easy to ask the user if they want to store the file
as a temporary file or in a specific path.

You probably know what `doto` does but it's interesting enough to talk about it
here. It returns the result of the first form with all the rest of the forms
applied (inserting the result of the first form as first argument), discarding
their return values. This sounds weird at the beginning but in cases like this
one where you are working with mutation it's really handy.  We are creating a
`File` instance and returning it after calling `.deleteOnExit` on it. Take in
consideration that `.deleteOnExit` returns nothing, so it's really cool for us
to have a `doto` macro.

Once we now how to deal with `doto` we can improve the caller to with this
function that creates sockets with some timeout applied and automatically
connects:

    ::clojure
    (defn- ->socket
      ([host port]
       (->socket host port 10000))
      ([host port timeout]
      (doto (java.net.Socket.)
            (.setSoTimeout timeout)
            (.connect (InetSocketAddress. host port) timeout))))


Replacing `java.net.Socket` from the example above with a call to this function
will make the call handle timeouts.

Whatever, right? Better check the code for that. Beware that it may change as I
keep going with the development. Maybe not, it depends on the time I spend on
this.

Here's the link to the code. Relevant part can be found in
`src/clojure/clopher` in a file called `net` or similar:

https://gitlab.com/ekaitz-zarraga/clopher

Time to move on.

### Hey! But what about the Menus?

Menus are just queried like any other text document so they can be queried with
this little code. The parsing, processing and so on is only needed for user
interaction so we'll deal with that later. Don't worry, we'll arrive soonish.

See you in the next step.


[^1]: Line terminator is CRLF (carriage-return + line-feed), aka `"\r\n"`.
[^2]:  Don't be afraid of the types because they are just a few of them.

[gopher]:     https://en.wikipedia.org/wiki/Gopher_%28protocol%29