Cache control using a map in Nginx

Google's delightful Lighthouse utility moaned at me about cache-control. For that particular site I use Nginx as a reverse proxy to NodeJS.

Nginx allows us to set a map for filetypes and apply that map to one or more server definitions. To define the map create the following near the top of your configuration file:

map $sent_http_content_type $expires {  
    default                    off;
    text/html                  epoch;
    text/css                   max;
    application/javascript     max;
    ~image/                    max;
}

In the above example, epoch sets the expiry time of all html to 1st January 1970 (which we all know was a Thursday). max on the other hand sets the expiry time to 31st December 2037 (a Friday). Other options off or a specific time.

The last entry, ~image will apply to rule to any content type beginning with image/, for example image/jpeg or image/png.

To apply the map add the following line inside the server block:

server {  
    listen...

    expires $expires;
}

Plex transcoding using a RAM disk

My home server is a server in software only. Hardware wise it's an old Dell Inspiron desktop which was my main machine for a good five years. It's a core 2 duo with 6gb of RAM and on-board graphics.

I run the Plex media server on it, as well as some other services, all on Ubuntu Server 18.04. Without a fancy GPU, Plex does all the transcoding of media files using software, something which can tax an old system like mine given I encode to HEVC to save space.

Plex allows us to specify a directory where the transcoding data is written as it is used. It doesn't need to be very big. It occurred to me I could use a RAM disk for this and make it nice and snappy. Not to mention saving wear on the drive.

To create the drive first create a directory, then mount it as tmpfs

sudo mkdir /mnt/ramdisk  
sudo mount -t tmpfs -o size=512M tmpfs /mnt/ramdisk  

It's important the location is owned by root as we want to remount it at reboot. The default permissions for tmpfs make it writable by everyone, but just in case it's not chmod it to 1777.

To ensure the drive is remounted at boot add the following to the /etc/fstab file:

tmpfs /mnt/ramdisk tmpfs rw,size=512M 0 0  

We can use the df -h command to view the usage. The following shows my 1gb RAM disk with Plex using 67mb to transcode an SD video of a Porky Pig cartoon encoded in HEVC:

Filesystem      Size  Used Avail Use% Mounted on  
udev            2.9G     0  2.9G   0% /dev  
tmpfs           594M   33M  561M   6% /run  
/dev/sdc1        50G  9.3G   38G  20% /
tmpfs           2.9G   20K  2.9G   1% /dev/shm  
tmpfs           5.0M     0  5.0M   0% /run/lock  
tmpfs           2.9G     0  2.9G   0% /sys/fs/cgroup  
/dev/sdb1       1.8T  1.1T  664G  62% /mnt/media
/dev/sda1       688G  282G  372G  44% /mnt/tor
cgmfs           100K     0  100K   0% /run/cgmanager/fs  
tmpfs           594M     0  594M   0% /run/user/1000  
tmpfs           1.0G   67M  958M   7% /mnt/ramdisk  

Raspberry Pi Waveshare TFT Trouble

In order to monitor a small Raspberry Pi 3 running as a server I bought a Waveshare 3.5" TFT touchscreen which connects through the GPIO pins.

I followed the instructions that came with it and all went well. It booted in to X and I could use the touchscreen without any further configuration. I don't need X as I'm likely just to run htop most of the time. This is where the problems started.

The Pi would show the boot process on the TFT then suddenly stop. After waiting a while I hit some keys and the screen moved, but didn't change. Seems the boot process was finished but I couldn't see it.

I tried everything I could think of. It wasn't until I hit ctrl-alt-f2 and saw a login prompt that things started to click. After much Googling I found myself looking at the /etc/rc.local file which contained the following conspicuous line:

fbcp &  

It turns out this command copies the framebuffer with a small delay (~25ms). I remmed out this line, rebooted and all was well.

I suspect if I wanted to load X I might need this line back, but for now I'm happy.

Node on Ubuntu on Vagrant on Hyper-V

The following is how I got NodeJS running on Ubuntu using Vagrant in Windows with Hyper-V.

Why Hyper-V?

Paying for VMWare goes against my open-source leanings, and I'm a lone dev so any expenses have to be seriously considered.

Virtualbox is a great product and I've used it a lot in the past but I never liked the networking stuff it added to my system. I've tried using Vagrant with Virtualbox on Windows before and always come against folder and file permissions and syncing problems. I have problems with Oracle too.

Hyper-V on the other hand is baked natively in to Windows 8.1 and higher (I'm using Windows 10). It's straightforward to enable, configure and plays nicely with the Windows filesystem and networking stack.

Why Ubuntu?

I've used it more than any other Linux distribution. I'm very familiar with configuring it and running it in production. I dare say the steps I outline below are fairly 'box' agnostic though.

Step 1. Install Vagrant

Head on over to https://www.vagrantup.com/downloads.html to download the installer, run it and wait for it to finish.

Step 2. Enable Hyper-V

Go to Uninstall or change a program, you can find this in the toolbar on This PC, via the control panel, or just search for it.

Next click on Turn Windows features on or off on the left side of the screen. And make sure Hyper-V is checked.

Enabled Hyper-V

You might have to reboot to fully enable Hyper-V but once it's enabled you can check it under the performance tab in the task scheduler

Check Hyper-V is enabled

Step 3. Create Hyper-V network switch

This step is really important. If you do not do this, Vagrant will not be able to connect to the box. So hit your Start button and search for Hyper-V Manager. Once in, find Virtual Switch Manager...

Hyper-V Virtual Switch Manager link

In the 'Virtual Switch Manager' select New virtual network switch, now you have three choices:

New virtual network switch

External creates a network switch that is connected to your system's physical network. This will allow your Vagrant box to connect to the outside world, and vice versa.

Internal creates a network switch that allows your host system and the virtual machines in Hyper-V to talk to each other. If you select this option, your Vagrant will not have internet access.

Private creates a network switch that can only be used by the virtual machines. This is useless for Vagrant.

I suggest using External as it means I can use apt-get etc. So select External and hit Create Virtual Switch. All you need do now is give your virtual switch a name. Hit 'OK' and close the Hyper-V Manager.

Step 4. The Vagrantfile

Now we have the host operating system set up and Vagrant installed it's time to actually create a Vagrant box.

In the directory your project with be in type the following command:

vagrant init  

This will create a single file called Vagrantfile in your directory. This file is all you need and is where you'll put your instructions for setting up your Vagrant box.

Ignoring all the comments and remmed out statements, the basic Vagrantfile looks like this:

Vagrant.configure(2) do |config|  
  config.vm.box = "base"
end  

That's it. The base box is the default and is all well and good, but I want 64bit Ubuntu. So change "base" to "hashicorp/precise64". If you want a different base system, you can find more pre-built boxes at https://atlas.hashicorp.com/boxes/search

Next we have to tell Vagrant to use Hyper-V as I think it defaults to 'Virtualbox', so add the following line:

Vagrant.configure(2) do |config|  
  config.vm.box = "hashicorp/precise64"
  config.vm.provider "hyperv"
end  

Finally, we want to make sure the Vagrant box has access to the public network (the internet) so we can grab apt packages and the like. So add the following line:

Vagrant.configure(2) do |config|  
  config.vm.box = "hashicorp/precise64"
  config.vm.provider "hyperv"
  config.vm.network "public_network"
end  

This is the absolute basics we need. Save your Vagrantfile and we're ready to fire it up.

Step 5. vagrant up

One quirk of Vagrant on Hyper-V is that it must be run as Administrator. So whether you're using Command Prompt, Powershell, Cygwin, or Git Bash, you need to make sure you run it as Administrator.

So to get things going open your CLI window, navigate to your project folder and type:

vagrant up  

If you see an error regarding the provider, you may need to force the use of Hyper-V:

vagrant up --provider=hyperv  

If all goes well, you should see something along the lines of the following. It might take a while as it has to download the virtual hard drive for the Ubuntu version we selected. Also, because we're using Linux with Windows, Vagrant needs to set up a Samba share. So you'll need to enter your Windows credentials.

$ vagrant up
Bringing machine 'default' up with 'hyperv' provider...  
==> default: Verifying Hyper-V is enabled...
==> default: Importing a Hyper-V instance
    default: Cloning virtual hard drive...
    default: Creating and registering the VM...
    default: Successfully imported a VM with name: precise64
==> default: Starting the machine...
==> default: Waiting for the machine to report its IP address...
    default: Timeout: 120 seconds
    default: IP: 192.168.1.174
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 192.168.1.174:22
    default: SSH username: vagrant
    default: SSH auth method: password
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Preparing SMB shared folders...
    default: You will be asked for the username and password to use for the SMB
    default: folders shortly. Please use the proper username/password of your
    default: Windows account.
    default:
    default: Username: xxxxx
    default: Password (will be hidden):
    default: Password (will be hidden): xxxxxxxx
==> default: Mounting SMB shared folders...
    default: C:/Users/lewis/Desktop/vagrant test => /vagrant

If you see something like above then everything ran fine. You can SSH in to your box by typing vagrant ssh

Some useful Vagrant commands are:

vagrant upCreate a box or start from halt
vagrant haltGraceful shutdown
vagrant destroyRemove the box
vagrant suspendPause box at exact state
vagrant resumeResume from suspend
vagrant reloadReboot, maybe after a config change
vagrant provisionRe-run the provisioning stuff
vagrant sshSSH in to your box

More can be found on the Vagrant website

Step 6. Provisioning and NodeJS

Every time you create a box from a Vagrantfile or vagrant up after a vagrant destroy, Vagrant will create your box from scratch. While we could install all the software we need each time, it makes sense to tell Vagrant to do it for us. This is known as provisioning.

To get started with provisioning NodeJS, create a new file called bootstrap.sh. This is a bash script where we'll put in the commands we need to run. I'm going to install NVM, the node version manager. This is because NodeJS release new versions very fast so I'm happy to actually select my NodeJS version manually.

#!/usr/bin/env bash

wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.31.0/install.sh | bash  

Now in your Vagrantfile add the following line. The important part here is at the end, priveleged: false. By default provisioning scripts run as sudo but we want NVM installed as the vagrant user.

config.vm.provision :shell, path: "bootstrap.sh", privileged: false  

So now your whole Vagrantfile should look like this:

Vagrant.configure(2) do |config|  
  config.vm.box = "hashicorp/precise64"
  config.vm.provider "hyperv"
  config.vm.network "public_network"
  config.vm.provision :shell, path: "bootstrap.sh", privileged: false
end  

In your bootstrap.sh you would also put anything else you want run automatically. Such as setting environment variables, pulling from a remote repository, or even installing a database system like Redis.

All done

This is just basic steps to get NodeJS running in Ubuntu with Vagrant using Hyper-V on Windows. There is a lot more to Vagrant than I can cover here, so I suggest you start with their Getting Started guide.

Key only SSH

To cut back on the hacking attempts and make things just that little bit more secure, it's a good idea to disable the use of passwords to login via SSH.

Of course you'll need a way to access it so make sure you're public key is in your ~/.ssh/authorized_keys file.

To disable the use of passwords with SSH edit the sshd_config config file using something like nano. You'll need to run this as sudo.

sudo nano /etc/ssh/ssh_config  

Find the following lines and change them, or add them if they're missing:

RSAAuthentication yes  
PubkeyAuthentication yes  
ChallengeResponseAuthentication no  
PasswordAuthentication no  
UsePAM no  

One caveat here. I found on Ubuntu 12.04 that when I turn off UsePAM the banner I usually see when connecting with SSH is not shown.

To fix this I uncommented and ammended the line which reads #Banner /etc/issue.net:

Banner /etc/motd  

Of course you'll need to restart sshd, depending on what service management system you use, enter the following:

sudo service ssh restart  
or  
sudo /etc/init.d/sshd restart  

Important Don't lose your private keys which match the public keys you've used, or you'll never get back in!