Installing Ubuntu Server 20.04 – part 1

by Posted byAndré Jacobs

Overview

The plan for today:

  • Install the latest Ubuntu Server 20.04 LTS (the one without a GUI) on my server.
  • Use full disk encryption on the 240GB SSD that I use as the OS disk.
  • Setup and harden SSH so I can manage the server remotely.
  • Use a static IP address.
  • Manage all changes to config files in /etc using etckeeper, git and GitHub.
  • Setup a firewall using ufw.

NOTE: I prefix commands to be run on the server with a $ to indicate the prompt. Commands to be run from your other computer (like a laptop) will be indicated using local$.

I also prefer to use a public-private encryption key pair for each machine I connect to. In other words one key pair = one machine only.

Prerequisites

  • Create a USB bootable disk of the Ubuntu server installation.
  • Backup the existing OS disk.
# Here is an example of doing a full disk copy to a file
$ dd if=/dev/sda of=/media/some-other-disk/sda_backups/sda_20211001.img
  • Shutdown the machine and disconnect all drives except for the SDD that will be formatted and Ubuntu installed on.

Creating a bootable USB drive with the Ubuntu installer on

Download the latest Ubuntu live server image from here: https://ubuntu.com/download/server.

Choose "Option 2: Manual server installation" to get the live server image.

I will be installing the ubuntu-20.04.3-live-server-amd64.iso.

NOTE: Ubuntu has phased out the previous generations of installers and you can’t find it on the alternative images anymore. At first I was a bit worried because in the past I had issues with the live server image. However this time round I loved it!

Since I am on a Mac most of the time, I will be using balena Etcher to write the image to a USB disk.

Installing Ubuntu Server 20.04 LTS

  • Plug in the USB disk, power on the machine and ensure you are booting from the USB disk.
  • Select your language. English (UK)
  • Select your keyboard layout. I prefer English (US)
  • Configure the network connection. I have two NICs in this machine but for time being only the onboard motherboard one is connected. Take note of the IP address assigned by the DHCP server.
  • Proxy address was left blank.
  • Mirror address was left as default.
  • Configuring the SSD partitions. I chose to use the guided storage layout with the encryption option.

Use an entire disk: ✅ (Make sure it is the correct disk!) Set up this disk as an LVM group: ✅ Encrypt the LVM group with LUKS: ✅ (Specify the Passphrase you already generated and stored in your password manager)

On the Storage configuration screen the installer chose to only use 111GB out of 222GB to be mounted as / and thus leaving 50% free. To enlarge this partition select the ubuntu-lv item under Used Devices and select Edit. I chose to use the max size.

This will create the following layout (as seen by using lsblk after the installation):

sdb                           8:16   0 223.6G  0 disk
├─sdb1                        8:17   0     1M  0 part
├─sdb2                        8:18   0     1G  0 part  /boot
└─sdb3                        8:19   0 222.6G  0 part
  └─dm_crypt-0              253:0    0 222.6G  0 crypt
    └─ubuntu--vg-ubuntu--lv 253:1    0 222.6G  0 lvm   /
  • Confirm destructive action.
  • Setup Profile by specifying server’s name and your name, username and password.
  • Install OpenSSH server ✅
  • Select extra software. I chose none of them.
  • Sit back and enjoy a coffee.
  • Once the install is done, select Reboot Now. Remove the USB disk and press enter.
  • At boot you will be asked to enter the passphrase that used to encrypt the disk with.

Initial hardening

The first priority is to configure the network and secure the remote logging that was enabled by SSH.

  • Login with the user created during installation.

    Check the network is working:

$ ping google.com

# Get the IP that was assigned by the DHCP server
$ ip address show
or
$ ip a
  • Test that SSH is working from another computer.
local$ ssh user@192.168.x.x
# Ok boet now you can do the rest all from the sofa instead of hunched over in the "server room"
  • Update software.
$ sudo apt -y update && sudo apt -y upgrade
$ sudo apt autoremove
  • Change the IP address to be static (since this is a server and will eventually become the DHCP server and firewall for the local network). NOTE: Ubuntu has been using netplan for sometime now and you configure via a yaml file.
$ ls /etc/netplan
00-installer-config.yaml

$ sudo cp /etc/netplan/00-installer-config.yaml /etc/netplan/00-installer-config.bak
$ sudo vi /etc/netplan/00-installer-config.yaml
# Here is what my config looks like. The internet router is .y.y

# This is the network config written by 'subiquity'
network:
  ethernets:
    enp2s0:
      dhcp4: false
      addresses: [192.168.x.x/24]
      gateway4: 192.168.y.y
      nameservers:
        addresses: [192.168.y.y,8.8.8.8]
    enp5s0:
      dhcp4: true
  version: 2
  • Reboot to verify the change does work as expected. sudo reboot now.
  • Configure SSH to be on a different port and don’t allow root to login.
$ sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak
$ sudo vi /etc/ssh/sshd_config

Port 8204
PermitRootLogin no
  • Create a new public-private key pair on your laptop. UPDATE: There is a newer ed25519 standard that is recommended instead of RSA.
local$ ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa_descriptive_name -C "Server name - user etc."

# Add to macOS keychain
local$ ssh-add -K ~/.ssh/id_rsa_descriptive_name
  • Copy the public key to be used over to the server. This is important, make sure you are copying the file with the .pub extension over!
local$ scp -P 8204 ~/.ssh/id_rsa_descriptive_name.pub user@192.168.x.x:/home/user
  • Verify that this key pair can be used to SSH into the server.
local$ ssh -p 8204 -o PreferredAuthentications=publickey -i ~/.ssh/id_rsa_descriptive_name user@192.168.x.x
$ # In like flynn
  • Configure SSH to only use key pair authentication.
$ sudo vi /etc/ssh/sshd_config

PasswordAuthentication no

$ sudo systemctl restart ssh
$ sudo systemctl status ssh
  • Bind the server’s name to it’s IP address on your laptop (so you can stop having to remember the IP address). Later I will be installing mDNS so that Bonjour! can work on the Mac to see the Ubuntu server.
# Edit your laptop's /etc/hosts file
# Add an entry like this
192.168.x.x servername

# Verify this works
local$ ping servername
  • Configure the local SSH client to always use the correct SSH key pair.
local$ vi ~/.ssh/config

# Servername
Host servername
	HostName servername
	PreferredAuthentications publickey
	IdentityFile ~/.ssh/id_rsa_descriptive_name
	IdentitiesOnly yes
	Port 8204

# Now you should be able to connect using only this
local$ ssh -p 8204 user@servername

Track all changes made using etckeeper & git

It goes without saying that is it super important to keep a log of all changes made to a server. I have learned the hard way on Linux why this is important. Therefore I want to track all changes made to config files either by myself or by installing programs. This can be done using etckeeper. I will also be using git as the version control software and configure a remote private git repo to be used as origin.

See the official Ubuntu server documentation for more information about etckeeper.

  • Install etckeeper.
$ sudo apt install etckeeper
  • To check the git / etckeeper log of commits being made.
$ sudo etckeeper vcs log

commit 686b2f2a3297f9279952a741fe5ea5cfb96c636a
Author: user <user@servername>
Date:   Sun Oct 31 14:00:24 2021 +0000

    Initial commit

Setting up a private GitHub repo

  • For this example I will be using GitHub as a remote git hosting service. I assume you have already signed up etc.
  • Go to https://github.com/new to create a new repo. Specify the repository name, description and ensure you select Private. Skip all the "Initialize this repository" checkboxes since you will be pushing the etckeeper git repo to this repo.
  • Create the repository. On the next screen take notice of the section "…or push an existing repository from the command line" and copy the command that looks like this:
git remote add origin git@github.com:username/name-of-repo.git
  • Next we need to setup Deployment keys so that we can automate the process of pushing the git repo from the server to the GitHub repo without requiring a password to be entered every time.

See GitHub’s documentation for more information.

  • Generate a new SSH public-private key pair. Note GitHub recommends to use the newer ed25519 standard (up to now I always used -t rsa -b 4096 for a 4K RSA key). I have setup email privacy on GitHub and thus I will use the "@users.noreply.github.com" one as the comment.
# The key will be owned by root
$ sudo su
root$ ssh-keygen -t ed25519 -C "your_github_email@example.com"
  • Add the public key to the GitHub repo. Go to the repository’s Settings and select the "Deploy keys" section. Click on Add deploy key.

Paste in the Public key (the filename with .pub as extension). On Mac you can use cat id_ed25519_github.pub | pbcopy to copy the file to the clipboard.

Allow write access ✅

Add the key.

  • Verify the server can use SSH to connect to GitHub. You can find more info here.
root$ ssh -i /root/.ssh/id_ed25519_github -T git@github.com

# You should see this message
Hi ____! You've successfully authenticated, but GitHub does not provide shell access.

# If not, use ssh -vT to get more information
  • Create a SSH config file that will be used to map the host github-etckeeper to specify the key and other SSH settings to be used.
root$ vi /root/.ssh/config

# GitHub etckeeper
Host github-etckeeper
	HostName github.com
	PreferredAuthentications publickey
	IdentityFile /root/.ssh/id_ed25519_github
	IdentitiesOnly yes
	VisualHostKey=yes
	Port 22
  • Save and verify.
root$ ssh -T git@github-etckeeper
  • Add the GitHub repo as the remote destination known as origin for the etckeeper’s git repository
root$ cd /etc

# Recall previously you saved a command given by GitHub that looked liked this:
# git remote add origin git@github.com:username/name-of-repo.git
# This needs to be changed so that github.com becomes github-etckeeper (from the ssh config)
root$ git remote add origin git@github-etckeeper:username/name-of-repo.git

# Rename master to be main (so you are PC)
root$ git branch -M main
root$ git status
On branch main
nothing to commit, working tree clean

# Push all commits to the github repo
root$ git push -u origin main
  • Refresh your GitHub repo in the browser and you should see the files have been pushed to the remote git repo.

Configure etckeeper to push to the remote git repo

  • Allow etckeeper to automatically push to origin.
root$ vi /etc/etckeeper/etckeeper.conf

PUSH_REMOTE="origin"
  • Let us test this works.
# Create a test file
root$ echo "testing 123" > /etc/test.txt

# Commit all changes from /etc/ to the git repo
root$ etckeeper commit "Test 1"

# Check that the commit was also pushed to GitHub
# You should not have been asked for a passphrase to use the SSH key file

# Make another change and check the cron job should work
root$ echo "456" >> /etc/test.txt
root$ /etc/etckeeper/daily
# This should have also pushed the change to GitHub

# Finally
root$ rm /etc/test.txt
# Now wait until the cron job runs the next day and verify.
# I am also rebooting the server to verify everything works as I expect, and not find out 3 months from now when we had a power outage something doesn't work anymore
  • P.S. I came across keychain for Linux that will manage ssh-agent for key files with a passphrase. See cyberciti for more information.

Setting up a firewall

I like to use ufw (Uncomplicated Firewall) on Linux to manage the iptables.

It should be installed by default on Ubuntu 20.04 LTS, if not install using sudo apt install ufw.

  • Check current status.
$ sudo ufw status
Status: inactive
  • Deny all incoming connections by default.
$ sudo ufw default deny incoming
  • Allow SSH on the configured port and only from the specified NIC.
# Get the name of the NIC to use
$ ip a
...
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> ...

# port 8204 here is what the SSH server has been configured to use
$ sudo ufw allow in on enp2s0 proto tcp to 192.168.x.x port 8204

# Here is an example if you wanted to allow Samba (file sharing)
$ sudo ufw allow in on enp2s0 proto tcp to 192.168.x.x port 139,445
$ sudo ufw allow in on enp2s0 proto udp to 192.168.x.x port 137,138
  • Turn on logging. Log files will be written to /var/log/ufw.log 
$ sudo ufw logging on
  • Activate the firewall.
$ sudo ufw enable

Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup

$ sudo ufw status

Status: active

To                         Action      From
--                         ------      ----
192.168.x.x 8204/tcp on enp2s0 ALLOW       Anywhere
  • Test you can still SSH into the server.

In the next sessions

  • More hardening with tools like rkhunter, logwatch, tiger and zeek (Bro)
  • Setup email delivery (so we can receive notifications when things go wrong)
  • Monitor CPU temperature and tune power usage.
  • Add existing RAID array and non raided disks back to the system.
  • Setup Samba file sharing.
  • Setup Time machine for Mac backups.