diff options
| -rw-r--r-- | content/clopher/01-Intro.md | 88 | ||||
| -rw-r--r-- | content/clopher/02-Protocol.md | 164 |
2 files changed, 252 insertions, 0 deletions
diff --git a/content/clopher/01-Intro.md b/content/clopher/01-Intro.md new file mode 100644 index 0000000..2a2e90d --- /dev/null +++ b/content/clopher/01-Intro.md @@ -0,0 +1,88 @@ +Title: Introducing Clopher +Date: 2019-05-06 +Categories: Series +Tags: Clopher - TUI Gopher client +Slug: clopher01 +Lang: en +Summary: Introducing Clopher, the terminal based Gopher client I'm making. + +> Read all the posts in [Clopher series here]({tag}clopher-tui-gopher-client). + +When you do a hack (or even a dirty hack) you do it for some reason. You do it +because you can understand the complexity of the problem and you see it's a +complex problem to solve that needs a good enough solution for your case. + +You are facing the complexity. You are seeing it. You are seeing the deepness +of the abyss. + +This project started a little bit like an exercise to do that. Take a simple +problem: make a Gopher client, and try to solve it in a decent way collecting +information during the process. + +It's just a learning project, but it went wild. + +The initial idea was to force myself to use Clojure's network API, which is +Java's one, because I never used it in the past and I wanted to learn about it +and the possible problems it can have. In order to do that I decided to write a +[Gopher][gopher] client, because that way I'd also had to read the +[RFC][gopher-rfc] and some resources more. + +I sketched the Gopher protocol exchange without many problems, because it's +quite simple and the RFC is really well explained. The wild part came with the +rest of the project, which still is under heavy development and *it doesn't +work yet* (this sentence may be edited in the future, I hope it will). + +I wanted to make a terminal based client, and I had a cool library for this, +called `clojure-lanterna` which is just an interface to `lanterna`, a java +library for TUI (Terminal User Interfaces). When I wanted to use +`clojure-lanterna` I realized the project was kind of abandoned and it didn't +cover the UI elements, only the basic screen interface, and I decided to make +it by myself. + +Further than that, I thought that if I focused on only POSIX compatible +operating systems I wouldn't need to use `lanterna` neither. So I decided to +implement everything by myself. + +That took me to some thoughts I've been having these days: When software has +few dependencies or no dependencies at all you have more control over the +process of making it. People who code in popular programming languages have +even more libraries than we need and it's really hard to stop the temptation to +use them (this explains some recent events with NPM repositories, for +instance). This is not only about security –possible security breaches in +libraries we use– or control –the fact that we included some software we don't +know– it's also about remembering that libraries can't be software you just +import: they should be read, analysed and, often, thrown away in favor of an +ad-hoc solution. Many times ad-hoc solutions reduce the codebase size and they +solve the problem more accurately, as they are specifically design to solve +*our* problem. + +Also, it's good to tell yourself you can code everything from scratch and try +to prove it true. + +In summary, I wanted a project that covered these points: + +- Be a simple Gopher client. +- Written in Clojure. +- Terminal User Interface (TUI). +- No dependencies if possible. + +And all of them had some sense, at least in my mind, on the early beginning of +the project. + +### So, here we are + +As I said before, the goal is not to create a good software. It's not even to +create something that works. The idea is to learn during the process and this +post series is a way to put what I learned in an ordered way. + +If you follow this post series, you'll follow me on my research and hacks. We +are going to dive on all those weird concepts that will appear. I'll try to be +as technically correct as I can but I'm not an expert and this is not a class. +I'm just sharing my experiences. + +I'm looking at the abyss and telling you what I see from this view, pointing +the interesting things I spot. + + +[gopher]: https://en.wikipedia.org/wiki/Gopher_%28protocol%29 +[gopher-rfc]: https://tools.ietf.org/html/rfc1436 diff --git a/content/clopher/02-Protocol.md b/content/clopher/02-Protocol.md new file mode 100644 index 0000000..71f9020 --- /dev/null +++ b/content/clopher/02-Protocol.md @@ -0,0 +1,164 @@ +Title: Down the <del>rabbit</del> gopher hole +Date: 2019-03-30 +Categories: Series +Tags: Clopher - TUI Gopher client +Slug: clopher02 +Lang: en +Summary: Working on the Gopher protocol implementation and opening the door to + the future problems. + +> Read all the posts in [Clopher series here]({tag}clopher-tui-gopher-client). + + +As the project's goal was to create a Gopher client, it was time to understand +something about the protocol and read the [RFC][gopher]. No need for you to +know the protocol to understand what I'm going to say here. I think I already +did the difficult part for you. + +### Understand some Gopher + +Gopher is a really simple protocol (this doesn't mean I implemented it +correctly anyway). It's assumed to work on top of TCP (default port is 70) and +it's as simple as creating a socket, sending the *selector string* to it, and +reading everything from it until it closes. That's in most of the cases how it +works. + +It has two major ways to access the data: + +1. **Text mode**, which is used in most of the queries, needs the client to + read from the socket until a line with a single dot (`.`) appears[^1]. + +2. **Binary mode**, expects the client to read from the socket until the server + closes it. + +Easy-peasy. + +Gopher is a stateless protocol and that helps a lot during the creation of the +client. There's no need to retain data or anything related. + +*Selector strings* are what client wants to see. In order to know what +selections are possible, Gopher defines an specific text format that works as a +menu, and it's called, unsurprisingly, *Menu*. + +Menus have a description of the content, the address or hostname, the port, the +selector string, and a number that indicates the type of each of its elements +separated by a TAB (aka "\t" character). Each element in one line[^1]. + +Pay attention to the fact that each menu entry contains an address and a port, +that means it can be pointing to a different server! + +The *type* further than making the client choose between *binary* and *text* +mode also gives the client information about what kind of response it's going +to get from it: if it's a menu, an image, an audio file... It also says if the +element is a *search endpoint*[^2]. + +Yes, Gopher supports searches! + +Well, Gopher supports tons of things because the only rule is that all the +logic is on the server side. You can do whatever you want, if you do it on the +server. + +Searching is as simple as asking for a text document, but it adds also the +search query to the equation. During a search, the client needs to send the +*selector string* to select the endpoint and then the *search string* +(separated by a `TAB` character). + +There are some points more but this is more than enough for the moment. + +Let's make something work. + +### Make Gopher queries work + +A simple text request can be understood like this piece of Clojure code here: + + ::clojure + ; Define the function to make the queries + (defn send-text-request + [host port body] + (with-open [sock (java.net.Socket. host port) + writer (clojure.java.io/writer sock) + reader (clojure.java.io/reader sock) + response (java.io.StringWriter.)] + (.append writer body) + (.flush writer) + (clojure.java.io/copy reader response) + (str response))) + + ; Make a query and print the result + (println (send-text-request "tilde.team" 70 (str "~giacomo" "\r\n")) + +As you see, it's not waiting to the dot at the end of the file and it's not +doing any kind of parsing, error checking or timeout handling, but it works. +This a minimal (and ugly, clean the namespaces!) implementation for you to be +able to run it in the REPL. + +The binary is almost the same but the output must be handled in a different +way. As Clopher is a terminal based application I made it store the answer in a +file. + +There's a simple and beautiful way to handle temporary files in Java that you +can access from Clojure. As I wasn't a Java user before I didn't know this: + + ::clojure + (defn- ->tempfile + "Creates a temporary file with the provided extension. If extension is nil + it adds `.tmp`." + [extension] + (doto + (. java.io.File createTempFile "clopher" extension) + .deleteOnExit)) + +With this function is really simple to create a temporary file and copy the +download there. It's also easy to ask the user if they want to store the file +as a temporary file or in a specific path. + +You probably know what `doto` does but it's interesting enough to talk about it +here. It returns the result of the first form with all the rest of the forms +applied (inserting the result of the first form as first argument), discarding +their return values. This sounds weird at the beginning but in cases like this +one where you are working with mutation it's really handy. We are creating a +`File` instance and returning it after calling `.deleteOnExit` on it. Take in +consideration that `.deleteOnExit` returns nothing, so it's really cool for us +to have a `doto` macro. + +Once we now how to deal with `doto` we can improve the caller to with this +function that creates sockets with some timeout applied and automatically +connects: + + ::clojure + (defn- ->socket + ([host port] + (->socket host port 10000)) + ([host port timeout] + (doto (java.net.Socket.) + (.setSoTimeout timeout) + (.connect (InetSocketAddress. host port) timeout)))) + + +Replacing `java.net.Socket` from the example above with a call to this function +will make the call handle timeouts. + +Whatever, right? Better check the code for that. Beware that it may change as I +keep going with the development. Maybe not, it depends on the time I spend on +this. + +Here's the link to the code. Relevant part can be found in +`src/clojure/clopher` in a file called `net` or similar: + +https://gitlab.com/ekaitz-zarraga/clopher + +Time to move on. + +### Hey! But what about the Menus? + +Menus are just queried like any other text document so they can be queried with +this little code. The parsing, processing and so on is only needed for user +interaction so we'll deal with that later. Don't worry, we'll arrive soonish. + +See you in the next step. + + +[^1]: Line terminator is CRLF (carriage-return + line-feed), aka `"\r\n"`. +[^2]: Don't be afraid of the types because they are just a few of them. + +[gopher]: https://en.wikipedia.org/wiki/Gopher_%28protocol%29 |