lila-cruft/README-install.md

# HOWTO Set up Your Own Chess Server
This document shows how to install a chess server.
The free software application Lila, written by the fine developers at
https://lichess.org, will be used.

# Overview
System will be built from these main parts listed below. You don't
need to know all of these, but knowing some system administration
will help.

* OVH ISP.
* Debian stable (Bullseye/11).
* Lila.
* Lila-ws.
* Apache.
* MongoDB.
* Redis.
* Scala.
* Java.
* SBT.
* Yarn.
* Python.
* Git.
* Node.
* Certbot.
* DNS.
* All the way down to GRUB and below...

For a high volume service, some of these services can be broken out
across multiple servers. For this example, we'll use just one "blank"
remote virtual server with nothing else on it.

# Upstream
The best current upstream document describing the process is here:

https://github.com/ornicar/lila/wiki/Lichess-Development-Onboarding

Main upstream repos:

* https://github.com/ornicar/lila
* https://github.com/ornicar/lila-ws


# Donate
Be sure to donate to lichess for their great website and for
making free software:

* https://lichess.org/patron


# Pre-Installation Setup
First, you need to have a location to host the server. You will want a server
with a minimum of 4 gigs of RAM. When the server is running, usage is low,
but it takes awhile to compile, so more CPU/RAM will speed that up.

For this example, we'll set up at OVH, which is the same Internet company
that lichess.org uses.

You will also need a domain and someone providing domain name service (DNS).
OVH provides this service (presumably?) or I recommend Njalla.


## Register DNS
Since it takes awhile to spread across the Internet, it is best to first
register your domain so that process can happen in the background while
you are setting up the server.

For this example, we'll use the domain `mychestserver.org` with the final
example server URL being:

* https://www.mychestserver.org

Go to your registrar, and register your domain, such as:

* https://njal.la/


## Register at ISP
Go to OVH (or ISP of your choice) and create an account to host your server.
OVH may have regional websites as well:

* https://ovh.com/

You'll probably need to set up billing of some sort at this point.


## Set up Workstation SSH Keys
To connect to the server, you will need SSH keys. They'll be needed at time
of server creation, so we'll make them now. This is an example how to create
keys on a Debian stable workstation, where the username is "debian" and
the workstation name is "workstation". For OVH, we're creating `ecdsa` keys,
which is inferior to `ed25519` keys. Last I tested, OVH doesn't accept the
latter.


```
# Run command to create keys.
# Note the location where you saved the key.
# Just hit "enter" for a passphrase.

debian@workstation:~$ ssh-keygen -t ecdsa
Generating public/private ecdsa key pair.
Enter file in which to save the key (/home/debian/.ssh/id_ecdsa): /home/debian/.ssh/id_ecdsa-chess
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/debian/.ssh/id_ecdsa-chess
Your public key has been saved in /home/debian/.ssh/id_ecdsa-chess.pub
The key fingerprint is:
SHA256:M2qUpyl31CCUcn3t2+vM6Cn4JaZIVvnFJICtTQiTQmY debian@workstation
The key's randomart image is:
+---[ECDSA 256]---+
| .E oo.*.  .     |
| o. oo= +.. .    |
|   . o.+.....    |
|      .o.+ +.    |
|      o S . oo   |
|     . B + .. .  |
|    . O ..+ .  . |
|     * o.o.o =.  |
|      . ...o+.+  |
+----[SHA256]-----+
```


## Upload SSH key to ISP
Take SSH the key you just created above and upload it to OVH.
Go to `Public Cloud`, then near the bottom left column, under
`Project Management` click `SSH Keys`.
Under the new `SSH Keys` window, click `Add an SSH Key` button.
Paste the PUBLIC key created above, ending with `.pub` extension,
into the `Key` section of the `Add an SSH key` popup window.

Take this output and paste into the OVH form in the browser:

```
cat /home/debian/.ssh/id_ecdsa-chess.pub
```

It should look like a tangled mess like this (note, the `debian@workstation` field
at the end is informational and can be something depending on your user/workstaion):

```
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC16EdTLECoLqSnmM/aSKrskLYN5ygu2dVvSAfiu4SAHPElrY6wqgUq6kzzsbbnko+VqyGzZ4tTWMml/AlBrQaw= debian@workstation
```

In the `Name` field, enter `mychestkey`.

Click `Add` to save the key at OVH. You should now see it in the list.


## Create Virtual Machine at ISP
OVH sells dedicated "bare metal" servers called the `Bare Metal Cloud`.
They also sell virtual machine instances under the `Public Cloud`.
The bare metal servers can be better, but they are generally more expensive,
a bit more complex to set up and maintain. So for this example, we
will set up a virtual machine in the `Public Cloud`.


In OVH `Dashboard` click on `Public Cloud`, then in left column near the top
under `Compute`, click `Instances`. Then under the new `Instances` window,
click `Create an Instance`.

* `Select a Model`: `Discovery` tab, then select D2-8. There are some options
with 4 gigs of RAM and fewer CPUs, which could be used, but kind of slow.
This option is ~$20USD/month.

* `Select a Region`: The https://lichess.org server is in various data centers around
Northern France, such as Gravelines (GRA). If you want to be close to that
for some reason, you can select that. Or you could select a server that
is regionally close to you and your users in another part of the world.
For this example, we'll select Gravelines GRA3. Click `Next`.

* `Select an Image`: Under `Unix Distributions` tab, select `Debian 11`.

* `Select an Image`:  Under `SSH key` at the bottom of the section, select the
`mychestkey` you created and uploaded above. Click `Next`

* `Configure your instance`: Just one instance. We'll use `mychestserver` for
the name, use yours as appropriate. We won't do any of `Post-installation
script`, `Private Networks`, or `Backups`, although they could be used.
Click `Next`.

* `Billing Period`: As you like. This is just a test, so here just using
`Hourly` at $0.03886/hour. Click `Create an instance` to create the
virtual computer, which also starts billing.

* OVH will say `Launching Instance` and a few minutes later, your server
should be ready and in `Activated` status when viewed under the `Instances`
tab under `Public Cloud`.


## Forward DNS Configuration
Set up forward DNS with the new IP address OVH gave you for your
instance. Look at the `Public IP` of your new server `Activated`
server instance. In this example, it is `147.135.193.212`. That is the
network address of your new server. We want to add it to DNS, so add
it to OVH (?) or Njalla's records. For this example, this URL was used
to manage the domain:

* https://njal.la/domains/mychestserver.org/

Click `Manage` for the domain, then `+ Add Record`.

* `Type`: Use `A` record.

* `Name`: Use `www`.

* `IPv4 Address`: Use the `Public IP` OVH gave you for your instance. In
this example, `147.135.193.212`.

* `TTL`: Lets do something short for now, use `5m`. Click `Add`.

That will take anywhere from a few seconds to an hour to be picked up by
nameservers around the world. It is best if you *don't* query it for now
(wait ~15+ minutes) or servers may cache a negative answer, which you'll
have to outwait.


## Reverse DNS Configuration
Set up reverse DNS with the new IP address OVH gave you for your
instance. Look at the `Public IP` of your new server `Activated`
server instance. In this example, it is `147.135.193.212`.

In the OVH `Dashboard` under your `Instances`, click on your instance,
such as the example `mychestserver`. On the right hand side under
`Networks` in the `IPv4` section there is a button with three dots.
Click it and select `Change reverse DNS`. Find your `Public IP` address
in the list, our example `147.135.193.212`. In the `Reverse DNS`
column, click the edit pencil box icon. Enter your full domain name,
such as our example `www.mychestserver.org` and click the check mark
to save it.


## Set up SSH on Workstation
Back on your workstation, set up your SSH configuration with the key
you created and the new `Public IP`. Edit the file `~/.ssh/config`.

```
vim ~/.ssh/config
```


Add using your name and `Public IP` instead of this example.
Also, use the path to the *private* workstation SSH key created earlier.
Add to `~/.ssh/config`:

```
Host mychestserver
	Hostname 147.135.193.212
	User debian
	Port 22
	Identityfile ~/.ssh/id_ecdsa-chess
```


# Login
Now from your workstation, log into the server and check that all
is ok:

```
ssh mychestserver
```

It should look something like this:

```
debian@workstation:~$ ssh mychestserver 
Host key fingerprint is SHA256:WgtWRY7N3POEhSqhhS6aq7Wac1sR7AQ+abQTpgXiQvU
+---[ECDSA 256]---+
|SSB.   . .S  ..  |
|+* *. . SB ..S   |
|o.B +E oo.=.+ .  |
|.. =..o.. .  +   |
|   bb.b S.    .  |
|  o  o + .       |
|  ... . .        |
|.ooo             |
|+=o.             |
+----[SHA256]-----+
Linux mychestserver 5.10.0-8-cloud-amd64 #1 SMP Debian 5.10.46-5 (2021-09-23) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
debian@mychestserver:~$ 
```

You can check all is happy with commands like:

```
free -h
df -h
cat /proc/cpuinfo
dpkg -l
uname -a
dmesg -T
```

# Update Server
First, set new passwords for user `debian` and then `root` on the server,
using sudo as root... Looks something like this:

```
debian@mychestserver:~$ sudo passwd debian
New password: 
Retype new password: 
passwd: password updated successfully
debian@mychestserver:~$ sudo passwd
New password: 
Retype new password: 
passwd: password updated successfully
```

Now, update to latest Debian packages.
This should take around ten minutes.

```
sudo apt update
sudo apt upgrade
sudo apt clean
```

Reboot server to newly updated system.
It should take less than a minute to reboot.

```
reboot
```


## View Server Console
It generally isn't needed unless something goes wrong,
but you can also see your server's console to make sure
reboots or whatever are working ok. In your workstation
browser, go to the OVH web page.
Under `Public Cloud`, `Compute`, `Instances`, click on
your instance, such as `mychestserver`.
Near the center top, there is a tab that reads
`VNC Console`. Click it. There you can see your server's
console.


# Install
Log back into the new server:

```
debian@workstation:~$ ssh mychestserver 
```


## Install Debian Dependencies
Install the following dependencies from Debian's repos:

```
sudo apt update

sudo apt install			\
	apache2				\
	build-essential			\
	git				\
	openjdk-11-jre-headless		\
	python-is-python3		\
	python2				\
	python3-certbot-apache		\
	redis-server

sudo apt clean
```

Note: Docs say `python2` is needed, but is that still correct?

## Install External Dependencies
Lila has quite a few dependencies, many of which are outside
of distributions' repositories. Sometimes the dependency exists
in the repo, but it is the wrong version. So we'll need to install
these dependencies from external repositories:

* `mongodb`
* `node`
* `sbt`
* `yarn`


### Install MongoDB
Install MongoDB thusly.

Note, they don't have a Debian Bullseye repo, but the Debian
Buster repo works.

```
# Get APT Key
wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -

# Add Repository
echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/5.0 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list

# Update Apt
sudo apt update

# Install MongoDB Server
sudo apt install mongodb-org

# Be clean
sudo apt clean

# Start mongodb server
sudo systemctl start mongod.service

# Enable mongodb server on boot
sudo systemctl enable mongod.service

# Logs are here:
sudo tail -f /var/log/mongodb/mongod.log
```

### Install Node
Install Node thusly:

```
# Setup repos with their script
curl -fsSL https://deb.nodesource.com/setup_12.x | sudo bash -

# Install it
sudo apt install nodejs

sudo apt clean
```


### Install SBT
Install SBT thusly:

```
# Set up repos
echo "deb https://repo.scala-sbt.org/scalasbt/debian all main" | sudo tee /etc/apt/sources.list.d/sbt.list

echo "deb https://repo.scala-sbt.org/scalasbt/debian /" | sudo tee /etc/apt/sources.list.d/sbt_old.list

# Add repo key
curl -sL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823" | sudo apt-key add

# Update repo, install, and clean.
sudo apt update

sudo apt install sbt

sudo apt clean
```


### Install Yarn
Install Yarn thusly:

```
# Set up repos
echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

# Add repo key
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null

# Update repo, install, and clean.
sudo apt update

sudo apt install yarn

sudo apt clean

# Check OK:
debian@mychestserver:~$ yarn --version
1.22.17
```


## Set up Webserver
It is a bit easier to set up the webserver and get its SSL certificates
confirmed all working correctly before installing Lila, to
lessen any complications.


The webserver directories will be owned by user `debian`.


```
# User debian owns webserver files
sudo chown -R debian:debian /var/www

# Quick words for the webserver for testing
echo "mychestserver web" > /var/www/html/index.html

# Start webserver
sudo systemctl start apache2

# Logs are:
sudo tail -f /var/log/apache2/*.log
```

In your browser, you should now be able to see your website
in insecure plaintext on port 80. Go to your site with your
workstation's browser to check.
It should say like "mychestserver web".

* http://www.mychestserver.org

Note, your browser may try to send you to the `https` URL,
but that is set up below with `certbot`.

```
# Set up SSL certificates
sudo certbot
```

It should look something like this:

```
debian@mychestserver:~$ sudo certbot
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): webmaster@mychestserver.org

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: n
Account registered.
No names were found in your configuration files. Please enter in your domain
name(s) (comma and/or space separated)  (Enter 'c' to cancel): www.mychestserver.org
Requesting a certificate for www.mychestserver.org
Performing the following challenges:
http-01 challenge for www.mychestserver.org
Enabled Apache rewrite module
Waiting for verification...
Cleaning up challenges
Created an SSL vhost at /etc/apache2/sites-available/000-default-le-ssl.conf
Enabled Apache socache_shmcb module
Enabled Apache ssl module
Deploying Certificate to VirtualHost /etc/apache2/sites-available/000-default-le-ssl.conf
Enabling available site: /etc/apache2/sites-available/000-default-le-ssl.conf
Enabled Apache rewrite module
Redirecting vhost in /etc/apache2/sites-enabled/000-default.conf to ssl vhost in /etc/apache2/sites-available/000-default-le-ssl.conf

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled https://www.mychestserver.org
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/www.mychestserver.org/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/www.mychestserver.org/privkey.pem
   Your certificate will expire on 2022-03-22. To obtain a new or
   tweaked version of this certificate in the future, simply run
   certbot again with the "certonly" option. To non-interactively
   renew *all* of your certificates, run "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le
```

Then restart your web server:

```
sudo systemctl restart apache2
```

Now go to your website, and you should see that `https` encrypted
SSL is now working and you can view the certificate in your
workstation's web browser:

* https://www.mychestserver.org/


Lets also enable some Apache modules we'll need later.

```
# Enable Apache modules
sudo a2enmod headers http2 proxy proxy_http proxy_http2 proxy_wstunnel

# Restart Apache
sudo systemctl restart apache2

# Enable Apache to start on boot
sudo systemctl enable apache2
```


## Install Lila
Now we can actually install lila! See here:

* https://github.com/ornicar/lila

We'll install it in the Apache web tree.
Install thusly on server as `debian` user.

```
# Remove old directory
rm -rf /var/www/html

# Go to web directory
cd /var/www

# Clone the Lila git repository to the `html` directory
git clone --recursive https://github.com/ornicar/lila.git html
```

## Create MongoDB
Create a new MongoDB database.

```
# Go to new cloned dir
cd /var/www/html

# Create MongoDB database indexes
mongo lichess bin/mongodb/indexes.js
```

Creating the MongoDB database should look something like this:

```
debian@mychestserver:/var/www/html$ mongo lichess bin/mongodb/indexes.js
MongoDB shell version v5.0.5
connecting to: mongodb://127.0.0.1:27017/lichess?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("7b35e8f2-528c-4136-8b8f-7e9e21200857") }
MongoDB server version: 5.0.5
```


## Build Lila CSS and JS
Build the Lila CSS and JS files. This will take around ten minutes.

```
cd /var/www/html
./ui/build
```

It should look something like this:

```
debian@mychestserver:/var/www/html$ ./ui/build
building ui modules with target=dev and mode=build
node: v12.22.8
yarn: 1.22.17
yarn install v1.22.17
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
[4/5] Linking dependencies...
[5/5] Building fresh packages...
Done in 37.56s.
For faster builds, install GNU parallel.

### ui/common ###
yarn run v1.22.17
$ $npm_execpath run compile
$ tsc
Done in 5.45s.

### ui/chess ###
yarn run v1.22.17
$ $npm_execpath run compile
$ tsc --incremental
Done in 4.38s.

...

### ui/puzzle plugin dashboard ###
yarn run v1.22.17
$ rollup --config --config-plugin dashboard

src/dashboard.ts → ../../public/compiled/puzzle.dashboard.js...
created ../../public/compiled/puzzle.dashboard.js in 6.2s
Done in 6.90s.
```

## Start Lila Console
With everything built in the UI ok above,
start the Lila console thusly:

```
cd /var/www/html
./lila
```

It should look something like this:

```
debian@mychestserver:/var/www/html$ ./lila 
   |\_    _ _      _
   /o \  | (_) ___| |__   ___  ___ ___   ___  _ __ __ _
 (_. ||  | | |/ __| '_ \ / _ \/ __/ __| / _ \| '__/ _` |
   /__\  | | | (__| | | |  __/\__ \__ \| (_) | | | (_| |
  )___(  |_|_|\___|_| |_|\___||___/___(_)___/|_|  \__, |
                                                   |___/
Java 11.0.13
sbt -Dreactivemongo.api.bson.document.strict=false
downloading sbt launcher 1.5.8
copying runtime jar...
[info] [launcher] getting org.scala-sbt sbt 1.5.8  (this may take some time)...
[info] [launcher] getting Scala 2.12.14 (for sbt)...
[info] welcome to sbt 1.5.8 (Debian Java 11.0.13)
[info] loading settings for project html-build from plugins.sbt ...
[info] loading project definition from /var/www/html/project
[info] Updating 
https://repo1.maven.org/maven2/ch/epfl/scala/sbt-bloop_2.12_1.0/1.4.11/sbt-bloop-1.4.11.pom
  100.0% [##########] 2.7 KiB (13.4 KiB / s)
https://repo1.maven.org/maven2/org/scalameta/sbt-scalafmt_2.12_1.0/2.4.5/sbt-scalafmt-2.4.5.pom
  100.0% [##########] 2.7 KiB (13.4 KiB / s)

...

https://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.typesafe.sbt/sbt-web/scala_2.12/sbt_1.0/1.4.4/docs/sbt-web-javadoc.jar
  100.0% [##########] 1.3 MiB (643.2 KiB / s)
[info] Fetched artifacts of 
[info] compiling 3 Scala sources to /var/www/html/project/target/scala-2.12/sbt-1.0/classes ...
[info] Non-compiled module 'compiler-bridge_2.12' for Scala 2.12.14. Compiling...
[info]   Compilation completed in 14.772s.
[warn] one feature warning; re-run with -feature for details
[warn] one warning found
[info] loading settings for project lila from build.sbt ...
[info] resolving key references (67100 settings) ...
[info] set current project to lila (in build file:/var/www/html/)
[info] sbt server started at local:///home/debian/.sbt/1.0/server/22a4d6cb6dbbe2874385/sock
[info] started sbt server
[lila] $ 
```

## Compile Lila
At the `lila` prompt created by running Lila in the above step,
compile Lila thusly:

```
compile
```

It should look like:

```
[lila] $ compile

...

# lots of output, running for around five minutes.

...


https://repo1.maven.org/maven2/org/specs2/specs2-common_2.13/4.8.1/specs2-common_2.13-4.8.1.jar
  100.0% [##########] 2.0 MiB (12.2 MiB / s)
https://raw.githubusercontent.com/ornicar/lila-maven/master/com/typesafe/play/play-logback_2.13/2.8.8-lila_1.8/play-logback_2.13-2.8.8-lila_1.8.jar
  100.0% [##########] 9.5 KiB (50.2 KiB / s)
[info] Fetched artifacts of 
[info] compiling 9 Scala sources and 4 Java sources to /var/www/html/modules/rating/target/scala-2.13/classes ...
[info] compiling 26 Scala sources and 1 Java source to /var/www/html/modules/user/target/scala-2.13/classes ...

...


[info] compiling 22 Scala sources to /var/www/html/modules/api/target/scala-2.13/classes ...
[info] compiling 342 Scala sources and 1 Java source to /var/www/html/target/scala-2.13/classes ...
[success] Total time: 335 s (05:35), completed Dec 22, 2021, 10:04:55 PM
[lila] $ 
```


## Run Lila
Run Lila thusly, from the `lila` prompt, after compiling above.

```
[lila] $ run
```

After a minute or so, you should have output similar to this:

```
[lila] $ run
[warn] Compile / run / javaOptions will be ignored, Compile / run / fork is set to false
[info] running play.core.server.ProdServerStart 
[info] boot - lila Dev   / java 11.0.13, memory: 2048MB
[info] reactivemongo.api.Driver - [Supervisor-1] Creating connection: main
[info] e.r.StaticListenerBinder - Starting connection listener ...
[info] r.c.actors.MongoDBSystem - [Supervisor-1/main] Starting the MongoDBSystem
[info] r.core.netty.Pack - Instantiated reactivemongo.core.netty.Pack
[info] db.main - MongoDB connected to mongodb://127.0.0.1:27017?appName=lila in 1089 ms
[info] boot - Loaded lila modules in 7217 ms
[info] play.api.Play - Application started (Dev) (no global state)
[info] p.c.server.AkkaHttpServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9663
[info] reactivemongo.api.Driver - [Supervisor-1] Creating connection: insight
[info] e.r.StaticListenerBinder - Starting connection listener ...
[info] r.c.actors.MongoDBSystem - [Supervisor-1/insight] Starting the MongoDBSystem
[info] r.core.netty.Pack - Instantiated reactivemongo.core.netty.Pack
```

If you are at this point, the Lila server is now running, but the
full setup isn't complete.


## Connect to Lila via SSH Tunnel
The Lila server is now running, but can only be tested via `localhost`
since it hasn't been configured yet. You can set up an SSH tunnel
on your workstation to connect to the server. For example, open a new
terminal on your workstation and run this, using the name of your
server in your `~/.ssh/config` file.

```
ssh -N -C -L 9663:localhost:9663 mychestserver
```

Note, this command will just "hang" and not necessarily return
any output. When you want to end the tunnel, just hit
`CTRL-c` and it will stop.


Now the tunnel is up, you can access the Lila server via SSH
in your web browser. Go to this URL on your workstation:

* http://localhost:9663/

You should get the world famous lichess.org front page! :)

Note, not everything is working yet, and you will get `Reconnecting`
in your browser until we set up the websockets proxy.


## Install lila-ws
Set up the `lila-ws` web sockets server thusly. We will run it from
users `debian` home directory...

Since Lila is currently running in one terminal, open another
terminal to install `lila-ws`.

* https://github.com/ornicar/lila-ws

```
# Log into server from your workstation:
ssh mychestserver

# You should be in your homedir
cd

# Clone lila-ws
git clone https://github.com/ornicar/lila-ws

# Go there
cd lila-ws/

# And run lila web sockets server
sbt run
```

Should look like this:

```
debian@workstation:~$ ssh mychestserver
Last login: Wed Dec 22 20:17:10 2021 from 143.131.13.195
debian@mychestserver:~$ cd
debian@mychestserver:~$ git clone https://github.com/ornicar/lila-ws
Cloning into 'lila-ws'...
remote: Enumerating objects: 5577, done.
remote: Counting objects: 100% (665/665), done.
remote: Compressing objects: 100% (400/400), done.
remote: Total 5577 (delta 314), reused 500 (delta 229), pack-reused 4912
Receiving objects: 100% (5577/5577), 1.04 MiB | 8.55 MiB/s, done.
Resolving deltas: 100% (3494/3494), done.
debian@mychestserver:~$ cd lila-ws/
debian@mychestserver:~/lila-ws$ sbt run
[info] [launcher] getting org.scala-sbt sbt 1.5.7  (this may take some time)...
[info] welcome to sbt 1.5.7 (Debian Java 11.0.13)
[info] loading settings for project lila-ws-build from plugins.sbt ...
[info] loading project definition from /home/debian/lila-ws/project
[info] Updating 
https://repo1.maven.org/maven2/com/github/sbt/sbt-native-packager_2.12_1.0/1.9.7/sbt-native-packager-1.9.7.pom
  100.0% [##########] 3.7 KiB (24.6 KiB / s)
[info] Resolved  dependencies
[info] Fetching artifacts of 
https://repo1.maven.org/maven2/com/github/sbt/sbt-native-packager_2.12_1.0/1.9.7/sbt-native-packager-1.9.7.jar
  100.0% [##########] 860.6 KiB (5.8 MiB / s)
https://repo1.maven.org/maven2/net/java/dev/jna/jna/4.5.0/jna-4.5.0.jar
  100.0% [##########] 1.4 MiB (7.4 MiB / s)
[info] Fetched artifacts of 
[info] loading settings for project lila-ws from build.sbt ...
[info] set current project to lila-ws (in build file:/home/debian/lila-ws/)
[info] Updating 
https://repo1.maven.org/maven2/com/typesafe/scala-logging/scala-logging_2.13/3.9.4/scala-logging_2.13-3.9.4.pom
  100.0% [##########] 2.6 KiB (34.3 KiB / s)
https://repo1.maven.org/maven2/io/kamon/kamon-influxdb_2.13/2.4.2/kamon-influxdb_2.13-2.4.2.pom
  100.0% [##########] 2.6 KiB (82.7 KiB / s)
https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.2.9/logback-classic-1.2.9.pom
  100.0% [##########] 9.5 KiB (71.9 KiB / s)
https://repo1.maven.org/maven2/com/typesafe/akka/akka-actor-typed_2.13/2.6.18/akka-actor-typed_2.13-2.6.18.pom

...

https://raw.githubusercontent.com/ornicar/lila-maven/master/org/lichess/scalachess_2.13/10.2.0/scalachess_2.13-10.2.0.jar
  100.0% [##########] 1.1 MiB (2.2 MiB / s)
[info] Fetched artifacts of 
[info] compiling 61 Scala sources to /home/debian/lila-ws/target/scala-2.13/classes ...
[info] Non-compiled module 'compiler-bridge_2.13' for Scala 2.13.7. Compiling...
[info]   Compilation completed in 11.272s.
[warn] /home/debian/lila-ws/src/main/scala/actor/ClientActor.scala:63:5: match may not be exhaustive.
[warn] It would fail on the following inputs: Unexpected(_), WrongHole
[warn]     msg match {
[warn]     ^
[warn] one warning found
[warn] Compile / run / javaOptions will be ignored, Compile / run / fork is set to false
[info] running lila.ws.Boot 
SLF4J: A number (1) of logging calls during the initialization phase have been intercepted and are
SLF4J: now being replayed. These are subject to the filtering rules of the underlying logging system.
SLF4J: See also http://www.slf4j.org/codes.html#replay
INFO r.api.Driver [clients-akka.actor.default-dispatcher-6] [Supervisor-1] Creating connection: Connection-2
INFO r.api.Driver [clients-akka.actor.default-dispatcher-3] [Supervisor-1] Creating connection: Connection-1
INFO r.c.a.MongoDBSystem [reactivemongo-akka.actor.default-dispatcher-6] [Supervisor-1/Connection-1] Starting the MongoDBSystem
INFO r.c.a.MongoDBSystem [reactivemongo-akka.actor.default-dispatcher-7] [Supervisor-1/Connection-2] Starting the MongoDBSystem
INFO r.core.netty.Pack [reactivemongo-akka.actor.default-dispatcher-6] Netty EPoll successfully loaded (shaded: true)
INFO r.core.netty.Pack [reactivemongo-akka.actor.default-dispatcher-7] Netty EPoll successfully loaded (shaded: true)
INFO r.core.netty.Pack [reactivemongo-akka.actor.default-dispatcher-7] Instantiated reactivemongo.core.netty.Pack
INFO r.core.netty.Pack [reactivemongo-akka.actor.default-dispatcher-6] Instantiated reactivemongo.core.netty.Pack
INFO lila.ws.Lila [clients-akka.actor.default-dispatcher-7] Redis connection took 1374 ms
INFO lila.ws.Monitor$ [run-main-0] lila-ws netty epoll=false kamon=false
INFO lila.ws.Monitor$ [run-main-0] Java version: 11.0.13, memory: 1024MB
INFO l.w.n.NettyServer [run-main-0] Start
INFO l.w.n.NettyServer [run-main-0] Listening to 9664
```


Note, if you go to your workstation's browser and view the site via
the SSH tunnel again, the "Reconnecting" alert in the bottom left
shouldn't appear anymore.

* http://localhost:9663/


## Set up Apache Web Reverse Proxy
Now that the site is nominally working, we can set up the Apache
web server as a reverse proxy so we can access the site at an
encrypted URL, such as:

* https://www.mychestserver.org


Note, setting this up will re-break the web sockets until we
update that lila-ws configuration below.

Open yet another terminal on your workstation and ssh into the
server again, ala:

```
ssh mychestserver
```

Using your favorite text editor, such as vim, edit the Apache configuration
file:

```
sudo vim /etc/apache2/sites-enabled/000-default-le-ssl.conf
```

The full configuration file should look like this:

```
<IfModule mod_ssl.c>
<VirtualHost *:443>
ProxyRequests On
ProxyVia On
ServerAdmin webmaster@localhost
DocumentRoot /var/www/html
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
RewriteEngine on
AllowEncodedSlashes NoDecode
ProxyPass / http://localhost:9663/ nocanon
ProxyPassReverse / http://localhost:9663/
RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteRule /(.*) ws://localhost:9664/$1 [P,L]
Header set "Access-Control-Allow-Origin" "https://www.mychestserver.org"
Header set "Access-Control-Allow-Methods" "POST, GET, OPTIONS"
ServerName www.mychestserver.org
SSLCertificateFile /etc/letsencrypt/live/www.mychestserver.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/www.mychestserver.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
ProxyPreserveHost On
SSLProtocol -All TLSv1.3 TLSv1.2 -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
SSLCipherSuite AES256+EECDH
SSLHonorCipherOrder on
SSLCompression off
SSLVerifyClient None
SSLSessionTickets Off
SSLOptions +StrictRequire
</VirtualHost>
</IfModule>
```

Then restart the webserver with the new config:

```
sudo systemctl restart apache2.service

# Logs are:
sudo tail -f /var/log/apache2/*.log
```

Now you should be able to go to this page, but note,
everything will be broken! We're going to have to
configure Lila below for the domain. But you can see that
the Apache proxy part is at least redirecting to the
Lila server.

In your workstation browser, check:

* https://www.mychestserver.org/


# Configure
Now that the dependencies are in place, and the site is sort of
working, it is time to configure Lila and the web sockets 
daemon to use the correct domain. This will require recompiling.


## Configure Web Sockets lila-ws
Configure `lila-ws` thusly:


```
# Go to Lila web sockets source dir
cd ~/lila-ws/

# Edit the configuration file
vim ./src/main/resources/application.conf

# In the configuration, set this line
csrf.origin = "https://www.mychestserver.org"
```

Now that you have changed the configuration, you need to
restart the `lila-ws` daemon. Later, we'll set up a
system service, but for now, we'll just hammer it with
`CTRL-c` and restart it. So if you still have the lila-ws
running from earlier, go to that window and hit `CTRL-c`.
This window probably has a last line like this:

```
INFO l.w.n.NettyServer [run-main-0] Listening to 9664
```

Stopping it looks like:

```
INFO l.w.n.NettyServer [run-main-0] Listening to 9664
^C
[warn] Canceling execution...
Cancelled: run
[error] Cancelled: run
[error] Use 'last' for the full log.
```

Then just re-run it like:

```
sbt run
```


## Configure Lila
To configure Lila to use your domain and the web sockets
server, do the following. Log into the server and run:

```
# Go to the lila git root dir
cd /var/www/html/

# Edit using your favorite editor
vim conf/base.conf

# Update the following section from this:

net {
  domain = "localhost:9663"
  socket.domains = [ "localhost:9664" ]
  asset.domain = ${net.domain}
  asset.base_url = "http://"${net.asset.domain}
  asset.minified = false
  base_url = "http://"${net.domain}
  email = ""
  crawlable = false
  ratelimit = true
  prodDomain = "lichess.org"
  http.log = true
  stage.banner = false
}

# To this:

net {
  domain = "www.mychestserver.org"
  socket.domains = [ "www.mychestserver.org" ]
  asset.domain = ${net.domain}
  asset.base_url = "https://"${net.asset.domain}
  asset.minified = false
  base_url = "https://"${net.domain}
  email = "gm@mychestserver.org"
  crawlable = false
  ratelimit = true
  prodDomain = "www.mychestserver.org"
  http.log = true
  stage.banner = false
}

# Also change this line ( XXX needed?)
cookieName = "lila2"

# To something unique
cookieName = "mychestserver"

# And change the User Agent ( XXX probably not needed)
# From:
useragent = "lichess.org"

# To:
useragent = "www.mychestserver.org"

```

Note, there are many other sections in `conf/base.conf` that should be
edited, but this is minimally viable.

Now restart Lila. If you still are running it in a window, you
can `CTRL-c` that, which leaves you at `lila` prompt. Then quit,
recompile, and run.

```
# Looks like this

[info] http - 200 browser GET / Lobby.home 206ms
^C
[warn] Canceling execution...
Cancelled: run
[error] Cancelled: run
[error] Use 'last' for the full log.
[lila] $ 
[lila] $ exit
[info] shutting down sbt server
...
[info] shutdown - <194ms> service-stop Closing mongodb driver
[info] p.c.server.AkkaHttpServer - Running provided shutdown stop hooks
```


## Rebuild Static Pages for Domain
I think you have to rebuild the static bits now...
XXX probably need to patch foo.scala with the base URL?

```
cd /var/www/html

# Then rebuild
./ui/build
```

## Re-Run Lila

Then re-run `lila` at the server prompt.
Looks like:


```
debian@mychestserver:/var/www/html$ ./lila 
   |\_    _ _      _
   /o \  | (_) ___| |__   ___  ___ ___   ___  _ __ __ _
 (_. ||  | | |/ __| '_ \ / _ \/ __/ __| / _ \| '__/ _` |
   /__\  | | | (__| | | |  __/\__ \__ \| (_) | | | (_| |
  )___(  |_|_|\___|_| |_|\___||___/___(_)___/|_|  \__, |
                                                   |___/
Java 11.0.13
sbt -Dreactivemongo.api.bson.document.strict=false
[info] welcome to sbt 1.5.8 (Debian Java 11.0.13)
[info] loading settings for project html-build from plugins.sbt ...
[info] loading project definition from /var/www/html/project
[info] loading settings for project lila from build.sbt ...
[info] resolving key references (67100 settings) ...
[info] set current project to lila (in build file:/var/www/html/)
[info] sbt server started at local:///home/debian/.sbt/1.0/server/22a4d6cb6dbbe2874385/sock
[info] started sbt server
[lila] $ 
```

Now recompile:

```
[lila] $ compile
[success] Total time: 22 s, completed Dec 23, 2021, 12:25:50 AM
```

And finally run it:

```
[lila] $ run
```

In less than a minute, should look like this:

```
[lila] $ run
[warn] Compile / run / javaOptions will be ignored, Compile / run / fork is set to false
[info] running play.core.server.ProdServerStart 
[info] boot - lila Dev   / java 11.0.13, memory: 2048MB
[info] reactivemongo.api.Driver - [Supervisor-1] Creating connection: main
[info] e.r.StaticListenerBinder - Starting connection listener ...
[info] r.c.actors.MongoDBSystem - [Supervisor-1/main] Starting the MongoDBSystem
[info] r.core.netty.Pack - Instantiated reactivemongo.core.netty.Pack
[info] db.main - MongoDB connected to mongodb://127.0.0.1:27017?appName=lila in 871 ms
[info] boot - <0ms> RoundSocket Done loading 0/0 round games
[info] boot - Loaded lila modules in 7350 ms
[info] play.api.Play - Application started (Dev) (no global state)
[info] p.c.server.AkkaHttpServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9663
```

Now go test in your workstation browser:

* https://www.mychestserver.org


# Use
Use thusly...


# Patch
Example Lila patch:

https://spacecruft.org/deepcrayon/lila/commit/d0fb110e218292921e35e8fe8181d7e7ea8883e4


# Misc
Potentially include items such as:

* Local firewall.
* Securing ssh.
* Locking down system overall.
* Set locale.
* Set timezone.
* Disable IPv6.
* Lila secrets & salts.
* Turn off unneeded services.
* Forums.
* Irwin.
* Mail.
* Bots.
* git branches
* Apache SSL tweaks.
* Apache redirects to only use parts of site.