Go on with clopher

2 files changed, 252 insertions, 0 deletions

diff --git a/content/clopher/01-Intro.md b/content/clopher/01-Intro.md
new file mode 100644
index 0000000..2a2e90d
--- /dev/null
+++ b/content/clopher/01-Intro.md
@@ -0,0 +1,88 @@
+Title: Introducing Clopher
+Date: 2019-05-06
+Categories: Series
+Tags: Clopher - TUI Gopher client
+Slug: clopher01
+Lang: en
+Summary: Introducing Clopher, the terminal based Gopher client I'm making.
+
+> Read all the posts in [Clopher series here]({tag}clopher-tui-gopher-client).
+
+When you do a hack (or even a dirty hack) you do it for some reason. You do it
+because you can understand the complexity of the problem and you see it's a
+complex problem to solve that needs a good enough solution for your case.
+
+You are facing the complexity. You are seeing it. You are seeing the deepness
+of the abyss.
+
+This project started a little bit like an exercise to do that. Take a simple
+problem: make a Gopher client, and try to solve it in a decent way collecting
+information during the process.
+
+It's just a learning project, but it went wild.
+
+The initial idea was to force myself to use Clojure's network API, which is
+Java's one, because I never used it in the past and I wanted to learn about it
+and the possible problems it can have. In order to do that I decided to write a
+[Gopher][gopher] client, because that way I'd also had to read the
+[RFC][gopher-rfc] and some resources more.
+
+I sketched the Gopher protocol exchange without many problems, because it's
+quite simple and the RFC is really well explained. The wild part came with the
+rest of the project, which still is under heavy development and *it doesn't
+work yet* (this sentence may be edited in the future, I hope it will).
+
+I wanted to make a terminal based client, and I had a cool library for this,
+called `clojure-lanterna` which is just an interface to `lanterna`, a java
+library for TUI (Terminal User Interfaces). When I wanted to use
+`clojure-lanterna` I realized the project was kind of abandoned and it didn't
+cover the UI elements, only the basic screen interface, and I decided to make
+it by myself.
+
+Further than that, I thought that if I focused on only POSIX compatible
+operating systems I wouldn't need to use `lanterna` neither. So I decided to
+implement everything by myself.
+
+That took me to some thoughts I've been having these days: When software has
+few dependencies or no dependencies at all you have more control over the
+process of making it. People who code in popular programming languages have
+even more libraries than we need and it's really hard to stop the temptation to
+use them (this explains some recent events with NPM repositories, for
+instance). This is not only about security –possible security breaches in
+libraries we use– or control –the fact that we included some software we don't
+know– it's also about remembering that libraries can't be software you just
+import: they should be read, analysed and, often, thrown away in favor of an
+ad-hoc solution. Many times ad-hoc solutions reduce the codebase size and they
+solve the problem more accurately, as they are specifically design to solve
+*our* problem.
+
+Also, it's good to tell yourself you can code everything from scratch and try
+to prove it true.
+
+In summary, I wanted a project that covered these points:
+
+- Be a simple Gopher client.
+- Written in Clojure.
+- Terminal User Interface (TUI).
+- No dependencies if possible.
+
+And all of them had some sense, at least in my mind, on the early beginning of
+the project.
+
+### So, here we are
+
+As I said before, the goal is not to create a good software. It's not even to
+create something that works. The idea is to learn during the process and this
+post series is a way to put what I learned in an ordered way.
+
+If you follow this post series, you'll follow me on my research and hacks. We
+are going to dive on all those weird concepts that will appear. I'll try to be
+as technically correct as I can but I'm not an expert and this is not a class.
+I'm just sharing my experiences.
+
+I'm looking at the abyss and telling you what I see from this view, pointing
+the interesting things I spot.
+
+
+[gopher]:     https://en.wikipedia.org/wiki/Gopher_%28protocol%29
+[gopher-rfc]: https://tools.ietf.org/html/rfc1436
diff --git a/content/clopher/02-Protocol.md b/content/clopher/02-Protocol.md
new file mode 100644
index 0000000..71f9020
--- /dev/null
+++ b/content/clopher/02-Protocol.md
@@ -0,0 +1,164 @@
+Title: Down the <del>rabbit</del> gopher hole
+Date: 2019-03-30
+Categories: Series
+Tags: Clopher - TUI Gopher client
+Slug: clopher02
+Lang: en
+Summary: Working on the Gopher protocol implementation and opening the door to
+    the future problems.
+
+> Read all the posts in [Clopher series here]({tag}clopher-tui-gopher-client).
+
+
+As the project's goal was to create a Gopher client, it was time to understand
+something about the protocol and read the [RFC][gopher]. No need for you to
+know the protocol to understand what I'm going to say here. I think I already
+did the difficult part for you.
+
+### Understand some Gopher
+
+Gopher is a really simple protocol (this doesn't mean I implemented it
+correctly anyway). It's assumed to work on top of TCP (default port is 70) and
+it's as simple as creating a socket, sending the *selector string* to it, and
+reading everything from it until it closes. That's in most of the cases how it
+works.
+
+It has two major ways to access the data:
+
+1. **Text mode**, which is used in most of the queries, needs the client to
+   read from the socket until a line with a single dot (`.`) appears[^1].
+
+2. **Binary mode**, expects the client to read from the socket until the server
+   closes it.
+
+Easy-peasy.
+
+Gopher is a stateless protocol and that helps a lot during the creation of the
+client. There's no need to retain data or anything related.
+
+*Selector strings* are what client wants to see. In order to know what
+selections are possible, Gopher defines an specific text format that works as a
+menu, and it's called, unsurprisingly, *Menu*.
+
+Menus have a description of the content, the address or hostname, the port, the
+selector string, and a number that indicates the type of each of its elements
+separated by a TAB (aka "\t" character). Each element in one line[^1].
+
+Pay attention to the fact that each menu entry contains an address and a port,
+that means it can be pointing to a different server!
+
+The *type* further than making the client choose between *binary* and *text*
+mode also gives the client information about what kind of response it's going
+to get from it: if it's a menu, an image, an audio file... It also says if the
+element is a *search endpoint*[^2].
+
+Yes, Gopher supports searches!
+
+Well, Gopher supports tons of things because the only rule is that all the
+logic is on the server side. You can do whatever you want, if you do it on the
+server.
+
+Searching is as simple as asking for a text document, but it adds also the
+search query to the equation. During a search, the client needs to send the
+*selector string* to select the endpoint and then the *search string*
+(separated by a `TAB` character).
+
+There are some points more but this is more than enough for the moment.
+
+Let's make something work.
+
+### Make Gopher queries work
+
+A simple text request can be understood like this piece of Clojure code here:
+
+    ::clojure
+    ; Define the function to make the queries
+    (defn send-text-request
+      [host port body]
+      (with-open [sock (java.net.Socket. host port)
+                  writer (clojure.java.io/writer sock)
+                  reader (clojure.java.io/reader sock)
+                  response (java.io.StringWriter.)]
+        (.append writer body)
+        (.flush writer)
+        (clojure.java.io/copy reader response)
+        (str response)))
+
+    ; Make a query and print the result
+    (println (send-text-request "tilde.team" 70 (str "~giacomo" "\r\n"))
+
+As you see, it's not waiting to the dot at the end of the file and it's not
+doing any kind of parsing, error checking or timeout handling, but it works.
+This a minimal (and ugly, clean the namespaces!) implementation for you to be
+able to run it in the REPL.
+
+The binary is almost the same but the output must be handled in a different
+way. As Clopher is a terminal based application I made it store the answer in a
+file.
+
+There's a simple and beautiful way to handle temporary files in Java that you
+can access from Clojure. As I wasn't a Java user before I didn't know this:
+
+    ::clojure
+    (defn- ->tempfile
+     "Creates a temporary file with the provided extension. If extension is nil
+     it adds `.tmp`."
+     [extension]
+     (doto
+      (. java.io.File createTempFile "clopher" extension)
+      .deleteOnExit))
+
+With this function is really simple to create a temporary file and copy the
+download there. It's also easy to ask the user if they want to store the file
+as a temporary file or in a specific path.
+
+You probably know what `doto` does but it's interesting enough to talk about it
+here. It returns the result of the first form with all the rest of the forms
+applied (inserting the result of the first form as first argument), discarding
+their return values. This sounds weird at the beginning but in cases like this
+one where you are working with mutation it's really handy.  We are creating a
+`File` instance and returning it after calling `.deleteOnExit` on it. Take in
+consideration that `.deleteOnExit` returns nothing, so it's really cool for us
+to have a `doto` macro.
+
+Once we now how to deal with `doto` we can improve the caller to with this
+function that creates sockets with some timeout applied and automatically
+connects:
+
+    ::clojure
+    (defn- ->socket
+      ([host port]
+       (->socket host port 10000))
+      ([host port timeout]
+      (doto (java.net.Socket.)
+            (.setSoTimeout timeout)
+            (.connect (InetSocketAddress. host port) timeout))))
+
+
+Replacing `java.net.Socket` from the example above with a call to this function
+will make the call handle timeouts.
+
+Whatever, right? Better check the code for that. Beware that it may change as I
+keep going with the development. Maybe not, it depends on the time I spend on
+this.
+
+Here's the link to the code. Relevant part can be found in
+`src/clojure/clopher` in a file called `net` or similar:
+
+https://gitlab.com/ekaitz-zarraga/clopher
+
+Time to move on.
+
+### Hey! But what about the Menus?
+
+Menus are just queried like any other text document so they can be queried with
+this little code. The parsing, processing and so on is only needed for user
+interaction so we'll deal with that later. Don't worry, we'll arrive soonish.
+
+See you in the next step.
+
+
+[^1]: Line terminator is CRLF (carriage-return + line-feed), aka `"\r\n"`.
+[^2]:  Don't be afraid of the types because they are just a few of them.
+
+[gopher]:     https://en.wikipedia.org/wiki/Gopher_%28protocol%29