From 19ed330f5067a393839d983252d4a8799e296ad2 Mon Sep 17 00:00:00 2001 From: Ekaitz Zarraga Date: Thu, 30 Jul 2020 16:59:13 +0200 Subject: Update contents --- content/git-setup.md | 350 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 350 insertions(+) create mode 100644 content/git-setup.md (limited to 'content/git-setup.md') diff --git a/content/git-setup.md b/content/git-setup.md new file mode 100644 index 0000000..e4f5b06 --- /dev/null +++ b/content/git-setup.md @@ -0,0 +1,350 @@ +Title: Our own git server +Date: 2020-07-09 +Category: +Tags: +Slug: git-repo +Lang: en +Summary: How to set up a git server with simple tools + +Having some rest these days after some really hardworking months... I decided I +wanted to solve something that was on my to-do list for a long time. Really +long time. + +I wanted to have my own git repository for ElenQ and my personal projects +(which are the same thing because I take ElenQ *very* personally) and so I did. + +You may think: so you installed Gitea or something and you are done, right? + +But the answer is no. + +That would be the normal approach if I didn't have the arbitrary constraints I +imposed. This text is about those weird constraints and random thoughts I have +about this, and also about what's my current setup and how to make it. Serving +the second part as a *tutorial* for myself if I screw up and I need to start +over and also as a way to consciously think about what I did. + +### Context: random thoughts + +For me code is not a social network and it shouldn't be. I understand why +github is the way it is but for me it's just code. I don't need to show how +much I code, I don't need to follow, like, star, fork, or even share my opinion +about other people's work publicly. That's completely unrelated to the job. + +Large projects like github are changing the way we collaborate. I'm not against +that, but looks like we start to forget that git doesn't need anything else to +function. There's no need for pull/merge requests, for web servers or anything. + +Web interfaces for code are cool, but nothing is better than your own editor. +I realized I just clone the repositories I want to dig in and search with my +own tools in my local clone so... Why bother to have a powerful[^powerful] web +interface? + +I don't like to be forced to register in a platform just for sending patches or +taking part in a project. Why do I **need** to have a github account? + +ElenQ Technology currently uses a free gitlab account, but recently I've +started to be concerned about gitlab's business practices so I prefer to start +migrating out of it. I've seen they always send you to login page when you hit +a 404, and all that kind of weird behaviours that don't look they have been +done by accident[^gitlab]. Of course, there's also the fact that they are a +startup and all that. I don't really trust them. But that's a different story. +I still like the fact that their community edition is free software. It's a +business model we should do more. + +Gitea and Gogs are easy to install, which is a must for me, and they are simple +and useful, but replicate the same model. It's much better to self-host your +code than relying in a third party, no doubt. But that makes the login problem +even harder: more gitea or gogs instances we create more separate places to +register in[^forgefed]. + +Those free software tools solve the problem of the centralization in different +scales, but they are small social networks, still. + +### Possible replacements + +I'm ok with sending an email and I'm ok with receiving emails from people. + +With git's email workflow (`git sendmail`, or `git format-patch` if you don't +want to configure your connection to your email account) you can send patches +via email that can be applied in the code directly. That's more than enough for +many projects. + +Issues, suggestions and questions can be sent via email with no hassle, too. + +The possibility to clone the repositories via git protocol gives people the +chance to check the code freely in their editor of choice, without being +tracked while they browse[^editor]. + +### Problems + +Issue management makes perfect sense to me and it's a process that is cool to +have in the open. It helps people take part in projects, check what's the +status of the project, collaborate more effectively, share the bugs they find +and so on. But, for the kind of projects I have, issue management is more of a +problem than a solution. I've been receiving spam in my gitlab issues for a +while. To be honest, there have been more spam than real issues in my gitlab +account. + +There's not any easy way to fully replace an issue management tool, though. +Maybe a good use of the `README.md` and some extra files in the repository can +help. People are still able to reach and share their bug reports via email +without being publicly exposed. + +That's also a thing: if you let people interact freely on an issue board you +need some moderation (which requires skills and effort). It is true that people +may come to very interesting ideas if working together, but it's also true that +only happens in very popular projects[^popularity]. Handling that privately +helps to avoid misunderstandings you can't control. + +Apart from that, we have to admit only sharing repositories via git protocol +has exactly `0` discoverability, so we have to share them in a website or +something. Maybe not for interacting with them, but at least to show them. + +Git is able to handle a website via gitweb too, but it's simple, a little bit +hard to configure and not too fast. Also, it can be more visually appealing by +default. + +On the owner's side, it's interesting to be able to decide which repositories +you want to share with the public. Being able to give permissions to specific +people without giving them permissions to the whole server is also nice. If +the permissions can be set in specific branches of the repositories better. + +### Other option + +[Fossil-scm](https://fossil-scm.org/home/doc/trunk/www/index.wiki) is really +interesting. It comes with support for issues and wikis, and devnotes are a +great idea I'm sure I could take advantage of. + +But the tool itself is not as good as git in my opinion. + +Fossil uses SQLite databases for its things (it's developed by SQLite's +developers) which is cool sometimes but in other times is not as good as it +sounds. I'm getting too used to plain text files maybe? + +I tried to configure a multi-repository fossil for the server and I gave up in +the past but it's probably my fault rather than theirs. + +If you are interested on trying something new, you should take a look to +fossil. If you do, please, [contact +me](https://ekaitz.elenq.tech/pages/about.html) and tell me your experience +with it. + +### My solution + +For the permissions I used Gitolite, which is an authorization control that +makes heavy use of ssh. It uses a management repository where the administrator +can add users' public keys and each project's permissions and metadata. + +It basically creates ssh pseudo-sessions that are locked in gitolite-shell, +which decides if the user has access to the repo or not. Interesting use of ssh +for this. [Read more in the website][gitolite], they explain it much better +than I can. + +For the website I chose `cgit`, which is famous for being fast (cached by +default) and reliable, and turned out to be easy to configure. + +Both projects are in the order of some thousands lines of code, which is an +amount I could manage to read and edit if I want to. + +### How to configure + +Well, this is the reminder for myself, but it can be useful for you too. + +I installed both of the projects using debian's package repository. + +#### Gitolite + +By default, debian package creates a `gitolite3` user so you have to take that +in account if you want to make gitolite work in a debian machine (other +machines will have other details to check). + +Gitolite's debian package also asks for administrator's public ssh-key so you +have to provide it sooner or later. Once that's done you'll get a fantastic +`/var/lib/gitolite3` folder with everything you need. You'll see that folder +contains a `projects.list` file, that lists the git repositories, a +`repositories` folder with the repositories, a `.gitolite` folder and a +`.gitolite.rc` file. The last one needs some changes in order to work correctly +with cgit: + +#### Enable cgit access to the repos + +Set `.gitolite.rc`'s `UMASK` to `0027` to give group access to new repositories, +that will let other users in the group (cgit and git-daemon) access the +repositories. + +You probably don't want to share the gitolite-admin repository so leave it with +the permissions it came with. If you screw up here or there don't be afraid to +`chmod` any repository later. + +You also need to make `GIT_CONFIG_KEYS` more permissive (`.*` if you are crazy +enough) if you want Gitolite to be able to load git configuration. That way +you'll be able to set gitweb description in the repository that cgit can read. + +#### Enable git unauthenticated clone + +There are a couple of ways to do this. The first is to set the HTTP mode, that +is something I didn't do but you can check how to do it in the docs. + +I used git-daemon for git based unauthenticated clones. It's simple but you +may need to create your own systemd service or something: + + ::systemd-service + # git.service file + + [Unit] + Description=Start Git Daemon + + [Service] + ExecStart=/usr/bin/git daemon --base-path=/var/lib/gitolite3/repositories --reuseaddr /var/lib/gitolite3/repositories + + Restart=always + RestartSec=500ms + + StandardOutput=syslog + StandardError=syslog + SyslogIdentifier=git-daemon + + User=gitdaemon + Group=gitolite3 + + [Install] + WantedBy=multi-user.target + +Once you do that you should add it to systemd with `systemctl enable +/path/to/git.service` or something like that. Once added you can start it. + +But that's not going to show any repository because you didn't export any. If +you want to export them, Gitolite has an specific configuration option you have +to set in the `gitolite-admin` repo. You have to give the user `daemon` read +access: + + ::perl + repo testing + # daemon is what adds the daemon-export + R = daemon + # You should add some extra people too... + + # This is for cgit: + config gitweb.description = For testing purpose. + config gitweb.owner = Ekaitz + +When you add the `daemon` access Gitolite adds a `git-daemon-export-ok` file to +the repository that says to git-daemon the project can be shared. It won't be +possible to push to it anyway because we didn't allow it in the git-daemon +configuration. + + +#### cgit + +Some cgit configuration does the rest. This is my example configuration on +cgit. I'll probably change it soon, but there it goes: + + ::bash + # cgit config + # see cgitrc(5) for details + + css=/cgit.css + logo=/cgit.png + footer=/usr/share/cgit/footer.html + + repository-sort=age + + # if cgit messes up links, use a virtual-root. + # For example, cgit.example.org/ has this value: + virtual-root=/ + + clone-url=git://$HTTP_HOST/$CGIT_REPO_URL + # gitolite3@$HTTP_HOST:$CGIT_REPO_URL + + enable-index-links=1 + enable-index-owner=1 + enable-git-config=1 + enable-gitweb-owner=1 + remove-suffix=1 + + # Readmes to use + # readme=README.md + # you can set more of them here like README.rst and stuff, but all of them + # require some rendering I didn't want to configure. + + # Set title and description + root-title=ElenQ Technology + root-desc=Software repository for ElenQ + root-readme=/usr/share/cgit/root-readme.html + + project-list=/var/lib/gitolite3/projects.list + scan-path=/var/lib/gitolite3/repositories + + # Mimetypes + mimetype.gif=image/gif + mimetype.html=text/html + mimetype.jpg=image/jpeg + mimetype.jpeg=image/jpeg + mimetype.pdf=application/pdf + mimetype.png=image/png + mimetype.svg=image/svg+xml + +But cgit is still unable to see the projects because it's not part of the +`gitolite3` group. Make it part of the `gitolite3` group with `usermod` or +something. + +Also, cgit is a web server you have to add to your stuff. I have an nginx based +config so I need to add cgit to it. Cgit can work with uWSGI or fcgiwrap. I +chose the latter for no real reason: + + ::nginx + server { + listen 80; + listen [::]:80; + server_name git.elenq.tech; + root /usr/share/cgit; + try_files $uri @cgit; + + location @cgit { + include fastcgi_params; + fastcgi_param SCRIPT_FILENAME /usr/lib/cgit/cgit.cgi; + fastcgi_param PATH_INFO $uri; + fastcgi_param QUERY_STRING $args; + fastcgi_param HTTP_HOST $server_name; + fastcgi_pass unix:/run/fcgiwrap.socket; + } + } + + +Also you may be interested on HTTPS support, but you know how to add that +(certbot does, and it's not hard to do). + +### Closing words + +Now it's live at [https://git.elenq.tech](https://git.elenq.tech). If you were +wondering, cloning and pushing from there crazy fast, and the server that hosts +it is the cheapest server possible. It's much faster than github, or that's at +least my impression. + +So yeah... That's most of it. + +I just wanted to share some thoughts about software development workflow and +find an excuse to write down my configuration since I had issues to find any +explanation that had all the points I needed together. + +And I think I did, didn't I? + +Stay safe. + + + +[gitolite]: https://gitolite.com/gitolite/overview.html + +[^powerful]: There's no power for free. Powerful also means resource-intensive. + +[^gitlab]: cHeCKiNg YOur brOWsER bEfOrE AcCeSsIng gitLAB.cOm. + +[^forgefed]: Maybe not. Forgefed is doing a good job: + + +[^editor]: It also depends on the editor you choose. Choose wisely. + +[^popularity]: If a project I make reaches that kind of popularity, I'll open + a tool for that kind of discussion or maintain a mirror somewhere else. + + -- cgit v1.2.3