Thomas Mullaly

DevOps, Security and IT Leadership

Adding New Esxi Thin Disk to Ubuntu Linux 14.04

I’ve been rsyncing the server data to a local vm in our esx cluster then backing up the vm. I ran out of disk so here are my notes to add a new thin provisioned disk.

I had mounted a 100 GB virtual disk on /backups, now I’m replacing it with a 300 GB virtual disk.

Pure-FTPd on Ubuntu 12.04

This is a simple setup to allow your designer ftp access to a web directory. The problem that I encountered was for a web server running wordpress. Apache runs as the user www-data with an id of 33 on Ubuntu 12.04. The wordpress files have to be ownership as the webserver for you to download and install updates through wordpress’s web interface. I guess I could just have the core wordpress files and the plugins directory and given the rest the user account permissions but I didn’t. The Pure-ftp server allow me to create virtual users and I can give them any uid, except 0 (root). For security, I would just allow from the designers ip address.

Collectd on Ubuntu 12.04 and Nginx

Collectd is the new cool kid of system monitoring tools.

apt-get install collectd collectd-utils
vi /etc/collectd/collectd.conf
service collectd restart
/var/ossec/bin/syscheck_update -u local
cd /var
mkdir www
cd www
cp -r /usr/share/doc/collectd/examples/collection3 .
apt-get install librrds-perl libconfig-general-perl libhtml-parser-perl  libregexp-common-perl
apt-get install fast-cgi
apt-get install spawn-fcgi fcgiwrap
location /nginx_status {
          stub_status on;
          access_log off;
          allow 127.0.0.1;
          deny all;
        }
		
		location ~ .cgi$ {
	        	root /var/www;
	        	fastcgi_pass unix:/var/run/fcgiwrap.socket;
	        	include /etc/nginx/fastcgi_params;
	    	}

		location /collection3/share/ {
			alias /var/www/collection3/share/;
		}
service nginx restart

Secure Ubuntu 12.04 Wordpress Server Outline

  1. Operating System
    • Minimal os install
    • Minimal services running
    • Disable IPv6
    • Ossec to watch filesystem integrity, monitor log files and watch for rootkits
    • Script to alert when os/package updates are needed (patching)
    • Firewall (iptables)
  2. SSH
    • Harden the sshd config
  3. Apache
    • Harden apache
  4. MySql
    • Harden MySQL
  5. Postfix
    • Configured as satellite, will only send email to our domain from our relay
  6. Wordpress
    • Install security plugins
      1. iThemes Security
      2. Wordfence
      3. InfiniteWP
  7. Backup
    • Backup database (mysql)
    • Backup file system (wordpress directory)
    • Document how to perform full recovery

The process to patch is to apt-get update. apt-get upgrade, then update ossec’s database. This might be automated in the future. I’m leaving the default ubuntu 12.04 AppArmor policies in place.

How to Put a KVM Guest Domain on a Bridged Network

I want to put some of the KVM domains that I’ve been using a bridged network so I can access them on my LAN directly.

I’m using my old mac pro as my KVM server and it has two interfaces. I’m going to bridge the second interface.
Install bridge-utils:

Wordpress Security Cleanup Notes

Infected PHP Pages: I’ve been looking for infected php pages with this command:

find . -name "*.php" | xargs grep "gzinflate(base64_decode"

Basically, there’s malicious code inserted into the beginning of the php script and it is base64 enconded and compressed so if you look at the file, the first line looks like garbarge. There’s a backdoor on the system that keeps it coming back. I remove it with the following one liner:

CIDR Subnet Mask Cheat Sheet

Netmask              Netmask (binary)                 CIDR     Notes    
_____________________________________________________________________________
255.255.255.255  11111111.11111111.11111111.11111111  /32  Host (single addr)
255.255.255.254  11111111.11111111.11111111.11111110  /31  Unuseable
255.255.255.252  11111111.11111111.11111111.11111100  /30    2  useable
255.255.255.248  11111111.11111111.11111111.11111000  /29    6  useable
255.255.255.240  11111111.11111111.11111111.11110000  /28   14  useable
255.255.255.224  11111111.11111111.11111111.11100000  /27   30  useable
255.255.255.192  11111111.11111111.11111111.11000000  /26   62  useable
255.255.255.128  11111111.11111111.11111111.10000000  /25  126  useable
255.255.255.0    11111111.11111111.11111111.00000000  /24 "Class C" 254 useable

255.255.254.0    11111111.11111111.11111110.00000000  /23    2  Class C's
255.255.252.0    11111111.11111111.11111100.00000000  /22    4  Class C's
255.255.248.0    11111111.11111111.11111000.00000000  /21    8  Class C's
255.255.240.0    11111111.11111111.11110000.00000000  /20   16  Class C's
255.255.224.0    11111111.11111111.11100000.00000000  /19   32  Class C's
255.255.192.0    11111111.11111111.11000000.00000000  /18   64  Class C's
255.255.128.0    11111111.11111111.10000000.00000000  /17  128  Class C's
255.255.0.0      11111111.11111111.00000000.00000000  /16  "Class B"
     
255.254.0.0      11111111.11111110.00000000.00000000  /15    2  Class B's
255.252.0.0      11111111.11111100.00000000.00000000  /14    4  Class B's
255.248.0.0      11111111.11111000.00000000.00000000  /13    8  Class B's
255.240.0.0      11111111.11110000.00000000.00000000  /12   16  Class B's
255.224.0.0      11111111.11100000.00000000.00000000  /11   32  Class B's
255.192.0.0      11111111.11000000.00000000.00000000  /10   64  Class B's
255.128.0.0      11111111.10000000.00000000.00000000  /9   128  Class B's
255.0.0.0        11111111.00000000.00000000.00000000  /8   "Class A"
  
254.0.0.0        11111110.00000000.00000000.00000000  /7
252.0.0.0        11111100.00000000.00000000.00000000  /6
248.0.0.0        11111000.00000000.00000000.00000000  /5
240.0.0.0        11110000.00000000.00000000.00000000  /4
224.0.0.0        11100000.00000000.00000000.00000000  /3
192.0.0.0        11000000.00000000.00000000.00000000  /2
128.0.0.0        10000000.00000000.00000000.00000000  /1
0.0.0.0          00000000.00000000.00000000.00000000  /0   IP space

MikroTik RouterBoard Backup Script

#!/bin/bash

base=/home/backups/mikrotik/hosts

hour=`date "+%H"`
dow=`date "+%w"`
dom=`date "+%d"`

if [ "$dom" = "01" ]; then
	backup_file=`date "+%Y-%m-%d"`.monthly
elif [ "$dow" = "0" ]; then
	backup_file=`date "+%Y-%m-%d"`.weekly
else
	backup_file=`date "+%Y-%m-%d"`.daily
fi

find $base -mtime +30 -name "*weekly*" -exec rm {} \;
find $base -mtime +7 -name "*daily*" -exec rm {} \;

cd $base
ls -1 | while read host ; do
	echo backing up $host
	ssh -n admin-ssh@$host "/system backup save name=$backup_file" > /dev/null
	ssh -n admin-ssh@$host "/export file=$backup_file" > /dev/null
	scp admin-ssh@$host:$backup_file.backup $base/$host > /dev/null
	scp admin-ssh@$host:$backup_file.rsc $base/$host > /dev/null
	echo "rm $backup_file.backup" | sftp admin-ssh@$host &> /dev/null
	echo "rm $backup_file.rsc" | sftp admin-ssh@$host &> /dev/null
done