Small syslog server

My home network has several devices that do not have large persistent storage to keep log files. For example, my wireless routers based on OpenWRT doesn’t log to the limited local storage it has, and a Flukso energy metering device log power readings to a ramdisk. These devices log a fair amount of information that I ideally would like to keep for later analysis. I have never before seen a need to setup a syslogd server, thinking that storing logs locally and keeping regular backups of the machine is good enough. However, it appears like this situation calls for a syslogd server. I found an old NSLU2 in my drawer and installed Debian Squeeze on it following Martin Michlmayr’s instructions. I’m using a 4GB USB memory stick for storage, which should hold plenty of log data. I keep backups of the machine in case the USB memory stick wears out.

After customizing the installation to my preferences (disable ssh passwords, disable portmap/rpc.statd/exim4, installing etckeeper, emacs23-nox, etc) I am ready to configure Rsyslog. I found what looked like the perfect configuration example, “Storing messages from a remote system into a specific file”, but it requires me to hard code a bit too much information in the configuration file for my taste. Instead, I found the DynFile concept. With a file /etc/rsyslogd.d/logger.conf as below I can point any new device to my log server and it will automatically create a new file for it. And since the dates are embedded into the filename, I get log rotation suitable for rsync-style backups for free.

$ModLoad imudp
$UDPServerRun 514

$ModLoad imtcp
$InputTCPServerRun 514

$template DynFile,”/var/log/network-%HOSTNAME%-%$year%-%$month%-%$day%.log”
:fromhost-ip, !isequal, “127.0.0.1” ?DynFile
:fromhost-ip, !isequal, “127.0.0.1” ~

After this, I get log files written to /var/log/network-IP-YEAR-MONTH-DAY.log. For example:

pepparkaka:~# tail /var/log/network-192.168.1.47-2012-03-20.log 
Mar 20 13:40:21 192.168.1.47 avahi-daemon[1508]: Registering new address record for 192.168.1.47 on br-lan.IPv4.
Mar 20 13:40:21 192.168.1.47 avahi-daemon[1508]: Registering HINFO record with values 'MIPS'/'LINUX'.
Mar 20 13:40:21 192.168.1.47 sysinit: setting up led WAN LED (green)
Mar 20 13:40:21 192.168.1.47 kernel: ar71xx-wdt: enabling watchdog timer

Use uci to configure the OpenWRT boxes to send log messages to this server:

uci set system.@system[0].log_ip=192.168.1.51
uci commit

Update! By default rsylog performs reverse lookups of incoming requests. This easily causes problems in case your DNS server is unreachable. Rsyslogd appears to have a long timeout for DNS queries, so if you expect incoming log messages to end up in the log when they are sent, think again. In my testing, it can take minutes until they end up in the log. For me, reverse DNS lookups does not add anything of value. To disable DNS lookups, make sure rsyslogd is invoked with the ‘-x’ parameter. On Debian, this is done by adding ‘-x’ to /etc/defaults/rsyslog like this:

RSYSLOGD_OPTIONS=”-c4 -x”

Unattended SSH with Smartcard

I have several backup servers that run the excellent rsnapshot software, which uses Secure Shell (SSH) for remote access. The SSH private key of the backup server can be a weak link in the overall security. To see how it can be a problem, consider if someone breaks into your backup server and manages to copy your SSH private key, they will now have the ability to login to all machines that you take backups off (and that should be all of your machines, right?).

The traditional way to mitigate SSH private key theft is by password protecting the private key. This works poorly in an unattended server environment because either the decryption password needs to be stored in disk (where the attacker can read it) or the decrypted private key has to be available in decrypted form in memory (where attacker can read it).

A better way to deal with the problem is to move the SSH private key to a smartcard. The idea is that the private key cannot be copied by an attacker who roots your backup server. (Careful readers may have spotted a flaw here, and I need to explain one weakness with my solution: an attacker will still be able to login to all your systems by going through your backup server, however it will require an open inbound network connection to your backup server and the attacker will never know what your private key is. What this does is to allow you to more easily do damage control by removing the smartcard from the backup server.)

In this writeup, I’ll explain how to accomplish all this on a Debian/Ubuntu-system using a OpenPGP smartcard, a Gemalto USB Shell Token v2 with gpg-agent/scdaemon from GnuPG together with OpenSSH.

Continue reading Unattended SSH with Smartcard

Redmine on Debian Lenny Using Lighttpd

The GnuTLS trac installation is in a poor shape. To fix that, I looked into alternatives and found Redmine. Redmine appears to do most things that I liked in Trac (wiki, roadmap and issue tracking) plus it supports more than one project (would come in handy for my other projects) and has built-in git support. I would like to see better spam handling and OpenID support, but it is good enough for our purposes now, and there are similar concerns with trac.

However, getting it up and running with lighttpd on a modern debian lenny installation was not trivial, and I needed some help from #redmine (thanks stbuehler). After finally getting it up and running, I made a copy of the machine using rsync and rsnapshot, so I could re-create a working configuration if I get stuck, and then re-installed the virtual machine.

The notes below are the steps required to set up Redmine using Lighttpd and MySQL on a Debian Lenny. I’m posting this to help others searching for the error messages I got, and to help my own memory in case I need to re-install the server sometime.
Continue reading Redmine on Debian Lenny Using Lighttpd