lila-cruft/README-install.md

HOWTO Set up Your Own Chess Server

This document shows how to install a chess server. The free software application Lila, written by the fine developers at https://lichess.org, will be used.

Overview

System will be built from these main parts listed below. You don't need to know all of these, but knowing some system administration will help.

• OVH ISP.
• Debian stable (Bullseye/11).
• Lila.
• Lila-ws.
• Apache.
• MongoDB.
• Redis.
• Scala.
• Java.
• SBT.
• Yarn.
• Python.
• Git.
• Node.
• Certbot.
• DNS.
• All the way down to GRUB and below...

For a high volume service, some of these services can be broken out across multiple servers. For this example, we'll use just one "blank" virtual server with nothing else on it.

Upstream

The best current upstream document describing the process is here:

https://github.com/ornicar/lila/wiki/Lichess-Development-Onboarding

Main upstream repos:

• https://github.com/ornicar/lila
• https://github.com/ornicar/lila-ws

Donate

Be sure to donate to lichess for their great website and for making free software:

• https://lichess.org/patron

Pre-Installation Setup

First, you need to have a location to host the server. You will want a server with a minimum of 4 gigs of RAM. When the server is running, usage is low, but it takes awhile to compile, so more CPU/RAM will speed that up.

For this example, we'll set up at OVH, which is the same Internet company that lichess.org uses.

You will also need a domain and someone providing domain name service (DNS). OVH provides this service (presumably?) or I recommend Njalla.

Register DNS

Since it takes awhile to spread across the Internet, it is best to first register your domain so that process can happen in the background while you are setting up the server.

For this example, we'll use the domain mychestserver.org with the final example server URL being:

• https://www.mychestserver.org

Go to your registrar, and register your domain, such as:

• https://njal.la/

Register at ISP

Go to OVH (or ISP of your choice) and create an account to host your server. OVH may have regional websites as well:

• https://ovh.com/

Set up Workstation SSH Keys

To connect to the server, you will need SSH keys. They'll be needed at time of server creation, so we'll make them now. This is an example how to create keys on a Debian stable workstation, where the username is "debian" and the workstation name is "workstation". For OVH, we're creating ecdsa keys, which is inferior to ed25519 keys. Last I tested, OVH doesn't accept the latter.

# Run command to create keys.
# Note the location where you saved the key.
# Just hit "enter" for a passphrase.

debian@workstation:~$ ssh-keygen -t ecdsa
Generating public/private ecdsa key pair.
Enter file in which to save the key (/home/debian/.ssh/id_ecdsa): /home/debian/.ssh/id_ecdsa-chess
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/debian/.ssh/id_ecdsa-chess
Your public key has been saved in /home/debian/.ssh/id_ecdsa-chess.pub
The key fingerprint is:
SHA256:M2qUpyl31CCUcn3t2+vM6Cn4JaZIVvnFJICtTQiTQmY debian@workstation
The key's randomart image is:
+---[ECDSA 256]---+
| .E oo.*.  .     |
| o. oo= +.. .    |
|   . o.+.....    |
|      .o.+ +.    |
|      o S . oo   |
|     . B + .. .  |
|    . O ..+ .  . |
|     * o.o.o =.  |
|      . ...o+.+  |
+----[SHA256]-----+

Upload SSH key to ISP

Take SSH the key you just created above and upload it to OVH. Go to Public Cloud, then near the bottom left column, under Project Management click SSH Keys. Under the new SSH Keys window, click Add and SSH Key button. Paste the PUBLIC key created above, ending with .pub extension, into the Key section of the Add an SSH key popup window.

Take this output and paste into that form in the browser:

cat /home/debian/.ssh/id_ecdsa-chess.pub

It should look like a tangled mess like this (note, the debian@workstation field at the end is informational and can be something depending on your user/workstaion):

ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC16EdTLECoLqSnmM/aSKrskLYN5ygu2dVvSAfiu4SAHPElrY6wqgUq6kzzsbbnko+VqyGzZ4tTWMml/AlBrQaw= debian@workstation

In the Name field, enter mychestkey.

Click Add to save the key at OVH. You should now see it in the list.

Create Virtual Machine at ISP

OVH sells dedicated "bare metal" servers called the Bare Metal Cloud. They also sell virtual machine instances under the Public Cloud. The bare metal servers can be better, but they are generally more expensive, a bit more complex to set up and maintain. So for this example, we will set up a virtual machine in the Public Cloud.

In OVH Dashboard click on Public Cloud, then in left column near the top under Compute, click Instances. Then under the new Instances window, click Create an Instance.

• Select a Model: Discovery tab, then select D2-8. There are some options with 4 gigs of RAM and fewer CPUs, which could be used, but kind of slow. This option is ~$20USD/month.

• Select a Region: The https://lichess.org server is in various data centers around Northern France, such as Gravelines (GRA). If you want to be close to that for some reason, you can select that. Or you could select a server that is regionally close to you and your users in another part of the world. For this example, we'll select Gravelines GRA3. Click Next.

• Select an Image: Under Unix Distributions tab, select Debian 11.

• Select an Image: Under SSH key at the bottom of the section, select the mychestkey you created and uploaded above. Click Next

• Configure your instance: Just one instance. We'll use mychestserver for the name, use yours as appropriate. We won't do any of Post-installation script, Private Networks, or Backups, although they could be used. Click Next.

• Billing Period: As you like. This is just a test, so here just using Hourly at $0.03886/hour. Click Create an instance to create the virtual computer, which also starts billing.

• OVH will say Launching Instance and a few minutes later, your server should be ready and in Activated status when viewed under the Instances tab under Public Cloud.

Forward DNS Configuration

Set up forward DNS with the new IP address OVH gave you for your instance. Look at the Public IP of your new server Activated server instance. In this example, it is 147.135.193.212. That is the network address of your new server. We want to add it to DNS, so add it to OVH (?) or Njalla's records. For this example, this URL was used:

• https://njal.la/domains/mychestserver.org/

Click Manage for the domain, then + Add Record.

• Type: Use A record.

• Name: Use www.

• IPv4 Address: Use the Public IP OVH gave you for your instance. In this example, 147.135.193.212.

• TTL: Lets do something short for now, use 5m. Click Add.

That will take anywhere from a few seconds to an hour to be picked up by nameservers around the world. It is best if you *don't` query it for now (wait ~15+ minutes) or servers may cache a negative answer, which you'll have to outwait.

Reverse DNS Configuration

Set up reverse DNS with the new IP address OVH gave you for your instance. Look at the Public IP of your new server Activated server instance. In this example, it is 147.135.193.212.

In the OVH Dashboard under your Instances, click on your instance, such as the example mychestserver. On the right hand side under Networks in the IPv4 section there is a button with three dots. Click it and select Change reverse DNS. Find your Public IP address in the list, our example 147.135.193.212. In the Reverse DNS column, click the edit pencil box icon. Enter your full domain name, such as our example www.mychestserver.org and click the check mark to save it.

Set up SSH on Workstation

Back on your workstation, set up your SSH configuration with the key you created and the new Public IP. Edit the file ~/.ssh/config.

vim ~/.ssh/config

Add using your name and Public IP instead of this example. Also, use the path to the private workstation SSH key created earlier. Add to ~/.ssh/config:

Host mychestserver
	Hostname 147.135.193.212
	User debian
	Port 22
	Identityfile ~/.ssh/id_ecdsa-chess

Login

Now from your workstation, log into the server and check that all is ok:

ssh mychestserver

It should look something like this:

debian@workstation:~$ ssh mychestserver 
Host key fingerprint is SHA256:WgtWRY7N3POEhSqhhS6aq7Wac1sR7AQ+abQTpgXiQvU
+---[ECDSA 256]---+
|SSB.   . .S  ..  |
|+* *. . SB ..S   |
|o.B +E oo.=.+ .  |
|.. =..o.. .  +   |
|   bb.b S.    .  |
|  o  o + .       |
|  ... . .        |
|.ooo             |
|+=o.             |
+----[SHA256]-----+
Linux mychestserver 5.10.0-8-cloud-amd64 #1 SMP Debian 5.10.46-5 (2021-09-23) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
debian@mychestserver:~$ 

You can check all is happy with commands like:

free -h
df -h
cat /proc/cpuinfo
dpkg -l
uname -a
dmesg -T

Update Server

First, set new passwords for user debian and then root on the server, using sudo as root... Looks something like this:

debian@mychestserver:~$ sudo passwd debian
New password: 
Retype new password: 
passwd: password updated successfully
debian@mychestserver:~$ sudo passwd
New password: 
Retype new password: 
passwd: password updated successfully

Now, update to latest Debian packages.

sudo apt update
sudo apt upgrade
sudo apt clean

Reboot server to newly updated system. It should take less than a minute to reboot.

reboot

Install

Log back into the new server:

debian@workstation:~$ ssh mychestserver 

Install Debian Dependencies

Install the following dependencies from Debian's repos:

sudo apt update

sudo apt install			\
	apache2				\
	build-essential			\
	git				\
	openjdk-11-jre-headless		\
	python-is-python3		\
	python2				\
	python3-certbot-apache		\
	redis-server

sudo apt clean

Note: Docs say python2 is needed, but is that still correct?

Install External Dependencies

Lila has quite a few dependencies, many of which are outside of distributions' repositories. Sometimes the dependency exists in the repo, but it is the wrong version. So we'll need to install these dependencies from external repositories:

• mongodb
• node
• sbt
• yarn

Install MongoDB

Install MongoDB thusly.

Note, they don't have a Debian Bullseye repo, but the Debian Buster repo works.

# Get APT Key
wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -

# Add Repository
echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/5.0 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list

# Update Apt
sudo apt update

# Install MongoDB Server
sudo apt install mongodb-org

# Be clean
sudo apt clean

# Start mongodb server
sudo systemctl start mongod.service

# Enable mongodb server on boot
sudo systemctl enable mongod.service

# Logs are here:
sudo tail -f /var/log/mongodb/mongod.log

Install Node

Install Node thusly:

# Setup repos with their script
curl -fsSL https://deb.nodesource.com/setup_12.x | sudo bash -

# Install it
sudo apt install nodejs

sudo apt clean

Install SBT

Install SBT thusly:

# Set up repos
echo "deb https://repo.scala-sbt.org/scalasbt/debian all main" | sudo tee /etc/apt/sources.list.d/sbt.list

echo "deb https://repo.scala-sbt.org/scalasbt/debian /" | sudo tee /etc/apt/sources.list.d/sbt_old.list

# Add repo key
curl -sL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823" | sudo apt-key add

# Update repo, install, and clean.
sudo apt update

sudo apt install sbt

sudo apt clean

Install Yarn

Install Yarn thusly:

# Set up repos
echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

# Add repo key
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null

# Update repo, install, and clean.
sudo apt update

sudo apt install yarn

sudo apt clean

# Check OK:
debian@mychestserver:~$ yarn --version
1.22.17

Set up Webserver

It is a bit easier to set up the webserver and get its SSL certificates confirmed all working correctly before installing Lila, to lessen any complications.

The webserver directories will be owned by user debian.

# User debian owns webserver files
sudo chown -R debian:debian /var/www

# Quick words for the webserver for testing
echo "mychestserver web" > /var/www/html/index.html

# Start webserver
sudo systemctl start apache2

# Logs are:
sudo tail -f /var/log/apache2/*.log

In your browser, you should now be able to see your website in insecure plaintext on port 80. Go to your site with your workstation's browser to check. It should say like "mychestserver web".

• http://www.mychestserver.org

Note, your browser may try to send you to the https URL, but that is set up below with certbot.

# Set up SSL certificates
sudo certbot

It should look something like this:

debian@mychestserver:~$ sudo certbot
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): webmaster@mychestserver.org

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: n
Account registered.
No names were found in your configuration files. Please enter in your domain
name(s) (comma and/or space separated)  (Enter 'c' to cancel): www.mychestserver.org
Requesting a certificate for www.mychestserver.org
Performing the following challenges:
http-01 challenge for www.mychestserver.org
Enabled Apache rewrite module
Waiting for verification...
Cleaning up challenges
Created an SSL vhost at /etc/apache2/sites-available/000-default-le-ssl.conf
Enabled Apache socache_shmcb module
Enabled Apache ssl module
Deploying Certificate to VirtualHost /etc/apache2/sites-available/000-default-le-ssl.conf
Enabling available site: /etc/apache2/sites-available/000-default-le-ssl.conf
Enabled Apache rewrite module
Redirecting vhost in /etc/apache2/sites-enabled/000-default.conf to ssl vhost in /etc/apache2/sites-available/000-default-le-ssl.conf

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled https://www.mychestserver.org
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/www.mychestserver.org/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/www.mychestserver.org/privkey.pem
   Your certificate will expire on 2022-03-22. To obtain a new or
   tweaked version of this certificate in the future, simply run
   certbot again with the "certonly" option. To non-interactively
   renew *all* of your certificates, run "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

Then restart your web server:

sudo systemctl restart apache2

Now go to your website, and you should see that https encrypted SSL is now working and you can view the certificate in your workstation's web browser:

• https://www.mychestserver.org/

Lets also enable some Apache modules we'll need later.

# Enable Apache modules
sudo a2enmod headers http2 proxy proxy_http proxy_http2 proxy_wstunnel

# Restart Apache
sudo systemctl restart apache2

# Enable Apache to start on boot
sudo systemctl enable apache2

Install Lila

Now we can actually install lila! See here:

• https://github.com/ornicar/lila

We'll install it in the Apache web tree. Install thusly on server as debian user.

# Remove old directory
rm -rf /var/www/html

# Go to web directory
cd /var/www

# Clone the Lila git repository to the `html` directory
git clone --recursive https://github.com/ornicar/lila.git html

Create MongoDB

Create a new MongoDB database.

# Go to new cloned dir
cd /var/www/html

# Create MongoDB database indexes
mongo lichess bin/mongodb/indexes.js

Creating the MongoDB database should look something like this:

debian@mychestserver:/var/www/html$ mongo lichess bin/mongodb/indexes.js
MongoDB shell version v5.0.5
connecting to: mongodb://127.0.0.1:27017/lichess?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("7b35e8f2-528c-4136-8b8f-7e9e21200857") }
MongoDB server version: 5.0.5

Install lila-ws

• https://github.com/ornicar/lila-ws

vim ./src/main/resources/application.conf
# set
csrf.origin = "https://deepcrayon.fish"

Configure

Configure thusly...

Use

Use thusly...

Misc

Potentially include items such as:

• Local firewall.
• Securing ssh.
• Locking down system overall.
• Set locale.
• Set timezone.
• Disable IPv6.
• Lila secrets & salts.
• Turn off unneeded services.
• Forums.
• Irwin.
• Mail.
• Bots.