sudo in .bashrc

Due to a quirk with the Raspberry Pi's current implementation of Debian I had some trouble mounting CIFS shares at start up. The solution was to run the mount command once logged in, the easiest way to do this is in the .bashrc script.

However! To run the mount -a command to mount all shares requires elevated privileges but I didn't want to have to type password twice at login. The solution to this is add the command itself to the /etc/sudoers with the NOPASSWD flag:

pi    ALL=(ALL)    NOPASSWD: /bin/mount

One thing to note is that it's important to provide the full path to the command.

Extracting and renaming files from zip archives

I had a pile of zip files all containing different files, but also containing a file called preview.png that I wanted to extract and rename. Since the names of the zip are descriptive I wrote a bash script to loop over the files, extract the preview file and rename it. The interesting bit is stripping the .zip extension.

#!/bin/bash

FILES=./*.zip
for F in $FILES
do
  echo "Processing $F file..."
  unzip "$F" preview.png
  FN=${F%.zip} # strip .zip
  mv preview.png "${FN##*/}.png"
done

For tar or gzipped tar the following will work in place of the 'unzip' line above:

tar -xf "$F" path/to/preview.png # extract from tar
tar -zxf "$F" path/to/preview.png # extract from tar.gz

GeoIP blocking and redirecting on Apache

Recently I was asked to configure a WordPress install to block any traffic from Russia, and redirect any traffic from the USA to another US-centric site.

First thing to do is install mod-geoip for Apache (in my case on Ubuntu):

sudo apt install libapache2-mod-geoip

Next uncomment the GeoIP database in the GeoIP conf file (/etc/apache2/mods-available/geoip.conf):

<IfModule mod_geoip.c>
  GeoIPEnable On
  GeoIPDBFile /usr/share/GeoIP/GeoIP.dat
</IfModule>

Restart Apache:

sudo systemctl restart apache2

With GeoIP enabled we can apply our block and redirect in .htaccess

GeoIPEnable On

# deny for Russia
SetEnvIf GEOIP_COUNTRY_CODE RU DenyCountry

# redirect for USA
RewriteEngine on
RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ^(US)$
RewriteRule ^(.*)$ https://usa-site.com/$1 [L]

A list of ISO Alpha-2 country codes can be found on WikiPedia, and many other places online.

Repeating tasks with Watch

Today I had to call an API over one-thousand two-hundred times that would in turn insert rows in to a database.

I wrote the code to load the data, transform it as required, then post it to the API. It would do a handful of rows then the database server would fall over.

I could've thrown more resources at the database server but that seemed like unnecessary cost. I tweaked settings, added swap space, but nothing worked. I even tried adding a delay to the code, but I wrote it in a asynchronous way and it was becoming a headache.

Then it struck me. Alter the code so it only reads one row, transforms it, and posts it to the API. Then run the program every few seconds. Reducing the code was trivial, it just meant pulling out any loops. I had to add a way to mark each row as complete so it wouldn't keep processing the same data, but that was easy enough.

Initially I planned to use cron to repeat the program, but it's a one-off job. I only need it to process one set of rows then I won't need the cronjob anymore.

Enter watch. Watch is a unix/linux command to repeat a command, such as calling a program, at a given delay. It shows the first page of output on each run so I could see any errors or returned data. Since I wrote my program in NodeJS and wanted it to run every five seconds, I just called the following:

watch -n 5 "node process.js"

Cache control using a map in Nginx

Google's delightful Lighthouse utility moaned at me about cache-control. For that particular site I use Nginx as a reverse proxy to NodeJS.

Nginx allows us to set a map for filetypes and apply that map to one or more server definitions. To define the map create the following near the top of your configuration file:

map $sent_http_content_type $expires {
    default                    off;
    text/html                  epoch;
    text/css                   max;
    application/javascript     max;
    ~image/                    max;
}

In the above example, epoch sets the expiry time of all html to 1st January 1970 (which we all know was a Thursday). max on the other hand sets the expiry time to 31st December 2037 (a Friday). Other options off or a specific time.

The last entry, ~image will apply to rule to any content type beginning with image/, for example image/jpeg or image/png.

To apply the map add the following line inside the server block:

server {
    listen...

    expires $expires;
}