Cache control using a map in Nginx

Google's delightful Lighthouse utility moaned at me about cache-control. For that particular site I use Nginx as a reverse proxy to NodeJS.

Nginx allows us to set a map for filetypes and apply that map to one or more server definitions. To define the map create the following near the top of your configuration file:

map $sent_http_content_type $expires {  
    default                    off;
    text/html                  epoch;
    text/css                   max;
    application/javascript     max;
    ~image/                    max;
}

In the above example, epoch sets the expiry time of all html to 1st January 1970 (which we all know was a Thursday). max on the other hand sets the expiry time to 31st December 2037 (a Friday). Other options off or a specific time.

The last entry, ~image will apply to rule to any content type beginning with image/, for example image/jpeg or image/png.

To apply the map add the following line inside the server block:

server {  
    listen...

    expires $expires;
}

Plex transcoding using a RAM disk

My home server is a server in software only. Hardware wise it's an old Dell Inspiron desktop which was my main machine for a good five years. It's a core 2 duo with 6gb of RAM and on-board graphics.

I run the Plex media server on it, as well as some other services, all on Ubuntu Server 18.04. Without a fancy GPU, Plex does all the transcoding of media files using software, something which can tax an old system like mine given I encode to HEVC to save space.

Plex allows us to specify a directory where the transcoding data is written as it is used. It doesn't need to be very big. It occurred to me I could use a RAM disk for this and make it nice and snappy. Not to mention saving wear on the drive.

To create the drive first create a directory, then mount it as tmpfs

sudo mkdir /mnt/ramdisk  
sudo mount -t tmpfs -o size=512M tmpfs /mnt/ramdisk  

It's important the location is owned by root as we want to remount it at reboot. The default permissions for tmpfs make it writable by everyone, but just in case it's not chmod it to 1777.

To ensure the drive is remounted at boot add the following to the /etc/fstab file:

tmpfs /mnt/ramdisk tmpfs rw,size=512M 0 0  

We can use the df -h command to view the usage. The following shows my 1gb RAM disk with Plex using 67mb to transcode an SD video of a Porky Pig cartoon encoded in HEVC:

Filesystem      Size  Used Avail Use% Mounted on  
udev            2.9G     0  2.9G   0% /dev  
tmpfs           594M   33M  561M   6% /run  
/dev/sdc1        50G  9.3G   38G  20% /
tmpfs           2.9G   20K  2.9G   1% /dev/shm  
tmpfs           5.0M     0  5.0M   0% /run/lock  
tmpfs           2.9G     0  2.9G   0% /sys/fs/cgroup  
/dev/sdb1       1.8T  1.1T  664G  62% /mnt/media
/dev/sda1       688G  282G  372G  44% /mnt/tor
cgmfs           100K     0  100K   0% /run/cgmanager/fs  
tmpfs           594M     0  594M   0% /run/user/1000  
tmpfs           1.0G   67M  958M   7% /mnt/ramdisk  

Repeating tasks with Watch

Today I had to call an API over one-thousand two-hundred times that would in turn insert rows in to a database.

I wrote the code to load the data, transform it as required, then post it to the API. It would do a handful of rows then the database server would fall over.

I could've thrown more resources at the database server but that seemed like unnecessary cost. I tweaked settings, added swap space, but nothing worked. I even tried adding a delay to the code, but I wrote it in a asynchronous way and it was becoming a headache.

Then it struck me. Alter the code so it only reads one row, transforms it, and posts it to the API. Then run the program every few seconds. Reducing the code was trivial, it just meant pulling out any loops. I had to add a way to mark each row as complete so it wouldn't keep processing the same data, but that was easy enough.

Initially I planned to use cron to repeat the program, but it's a one-off job. I only need it to process one set of rows then I won't need the cronjob anymore.

Enter watch. Watch is a unix/linux command to repeat a command, such as calling a program, at a given delay. It shows the first page of output on each run so I could see any errors or returned data. Since I wrote my program in NodeJS and wanted it to run every five seconds, I just called the following:

watch -n 5 "node process.js"

Last updated: September 10, 2018

Raspberry Pi Waveshare TFT Trouble

In order to monitor a small Raspberry Pi 3 running as a server I bought a Waveshare 3.5" TFT touchscreen which connects through the GPIO pins.

I followed the instructions that came with it and all went well. It booted in to X and I could use the touchscreen without any further configuration. I don't need X as I'm likely just to run htop most of the time. This is where the problems started.

The Pi would show the boot process on the TFT then suddenly stop. After waiting a while I hit some keys and the screen moved, but didn't change. Seems the boot process was finished but I couldn't see it.

I tried everything I could think of. It wasn't until I hit ctrl-alt-f2 and saw a login prompt that things started to click. After much Googling I found myself looking at the /etc/rc.local file which contained the following conspicuous line:

fbcp &  

It turns out this command copies the framebuffer with a small delay (~25ms). I remmed out this line, rebooted and all was well.

I suspect if I wanted to load X I might need this line back, but for now I'm happy.

Javascript Media Queries

Javascript media queries used to be a pain in the butt. I won't go in to why because thanks to window.matchMedia it's now very easy.

window.matchmedia has a matches property which returns a boolean value, so to run it once just use it something like and if statement:

if(window.matchMedia("(min-width: 500px)").matches) {  
  /* the viewport is at least 500 pixels wide */
} else {
  /* the viewport is less than 500 pixels wide */
}

It's also possible to add a listener to call a function to run some code whenever the query return value changes:

if(matchMedia){ // Only run if matchMedia is supported  
  const media_query = window.matchMedia("(max-width : 500px)"); // Create media query
  media_query.addListener(widthChange); // Optional listener with function to run
  widthChange(media_query); // Initial run
}

function widthChange(media_query){  
  if(media_query.matches){ // If the media query resolves true
    // window width is under 500px wide
    console.log('Under');
  } else {
    // window width is more than 500px wide
    console.log('Over');
  }
}