Replicant 4.2 on Samsung S3

Since November 2013 I have been using Replicant on my Samsung S3 as an alternative OS. The experience has been good for everyday use. The limits (due to non-free software components) compared to a “normal” S3 (running vendor ROM or CyanogenMod) is lack of GPS/wifi/bluetooth/NFC/frontcamera functionality — although it is easy to get some of that working again, including GPS, which is nice for my geocaching hobby. The Replicant software is stable for being an Android platform; better than my Nexus 7 (2nd generation) tablet which I got around the same time that runs an unmodified version of Android. The S3 has crashed around ten times in these four months. I’ve lost track of the number of N7 crashes, especially after the upgrade to Android 4.4. I use the N7 significantly less than the S3, reinforcing my impression that Replicant is a stable Android. I have not had any other problem that I couldn’t explain, and have rarely had to reboot the device.

The Replicant project recently released version 4.2 and while I don’t expect the release to resolve any problem for me, I decided it was time to upgrade and learn something new. I initially tried the official ROM images, and later migrated to using my own build of the software (for no particular reason other than that I could).

Before the installation, I wanted to have a full backup of the phone to avoid losing data. I use SMS Backup+ to keep a backup of my call log, SMS and MMS on my own IMAP server. I use oandbackup to take a backup of all software and settings on the phone. I use DAVDroid for my contacts and calendar (using a Radicale server), and reluctantly still use aCal in order to access my Google Calendar (because Google does not implement RFC 5397 properly so it doesn’t work with DAVDroid). Alas all that software is not sufficient for backup purposes, for example photos are still not copied elsewhere. In order to have a complete backup of the phone, I’m using rsync over the android debug bridge (adb). More precisely, I connect the phone using a USB cable, push a rsyncd configuration file, start the rsync daemon on the phone, forward the TCP/IP port, and then launch rsync locally. The following commands are used:

jas@latte:~$ cat rsyncd.conf
address 127.0.0.1
uid = root
gid = root
[root]
path = /
jas@latte:~$ adb push rsyncd.conf /extSdCard/rsyncd.conf
* daemon not running. starting it now on port 5037 *
* daemon started successfully *
0 KB/s (57 bytes in 0.059s)
jas@latte:~$ adb root
jas@latte:~$ adb shell rsync --daemon --no-detach --config=/extSdCard/rsyncd.conf &
jas@latte:~$ adb forward tcp:6010 tcp:873
jas@latte:~$ sudo rsync -av --delete --exclude /dev --exclude /acct --exclude /sys --exclude /proc rsync://localhost:6010/root/ /root/s3-bup/
...

Now feeling safe that I would not lose any data, I remove the SIM card from my phone (to avoid having calls, SMS or cell data interrupt during the installation) and follow the Replicant Samsung S3 installation documentation. Installation was straightforward. I booted up the newly installed ROM and familiarized myself with it. My first reaction was that the graphics felt a bit slower compared to Replicant 4.0, but it is hard to tell for certain.

After installation, I took a quick rsync backup of the freshly installed phone, to have a starting point for future backups. Since my IMAP and CardDav/CalDav servers use certificates signed by CACert I first had to install the CACert trust anchors, to get SMS Backup+ and DAVDroid to connect. For some reason it was not sufficient to add only the root CACert certificate, so I had to add the intermediate CA cert as well. To load the certs, I invoke the following commands, selecting ‘Install from SD Card’ when the menu is invoked (twice).

adb push root.crt /sdcard/
adb shell am start -n "com.android.settings/.Settings\"\$\"SecuritySettingsActivity"
adb push class3.crt /sdcard/
adb shell am start -n "com.android.settings/.Settings\"\$\"SecuritySettingsActivity"

I restore apps with oandbackup, and I select a set of important apps that I want restored with settings preserved, including aCal, K9, Xabber, c:geo, OsmAnd~, NewsBlur, Google Authenticator. I install SMS Backup+ from FDroid separately and configure it, SMS Backup+ doesn’t seem to want to restore anything if the app was restored with settings using oandbackup. I install and configure the DAVdroid account with the server URL, and watch it populate my address book and calendar with information.

After organizing the icons on the launcher screen, and changing the wallpaper, I’m up and running with Replicant 4.2. This upgrade effort took me around two evenings to complete, with around half of the time consumed by exploring different ways to do the rsync backup before I settled on the rsync daemon approach. Compared to the last time, when I spent almost two weeks researching various options and preparing for the install, this felt like a swift process.

Continue reading

Using Replicant on Samsung Galaxy S III

For the last half-year I have used CyanogenMod on an Nexus 4 as my main phone. Recently the touch functionality stopped working on parts of the display, and the glass on the back has started to crack. It seems modern phones are not built to last. For comparison, before the N4 I used a Nokia N900 for around 3 years without any hardware damages (in my drawer now, still working). A few weeks ago I started looking for a replacement. My experience with CyanogenMod had been good, but the number of proprietary blobs on the N4 concerned me. Finding something better wasn’t easy though, so I’m documenting my experience here.

My requirements were, briefly, that I wanted a phone that I could buy locally that had a free software community around it that produced a stable environment. I have modest requirements for things I wouldn’t give up on: telephony, data connection (3G), email (IMAP+SMTP), chat (XMPP), and a web browser. I like the philosophy and openness around the Firefox OS but the more I have read about it, it seems unlikely that it would deliver what I need today. In particular none of the devices capable of running Firefox OS appealed to me, and the state of email reading seemed unclear. I’m sure I’ll revisit Firefox OS as an alternative for me in the future.

As I had been happy with CyanogenMod, but concerned about its freeness, it felt natural to move on and test the more free software friendly project Replicant. Replicant only supports a small number of devices. After talking with people in the #replicant IRC channel, it seemed the Samsung S3 would be a decent choice for me. The Samsung S2 would have worked as well, but it cost almost as much as the S3 where I looked. Despite the large number of Samsung S3 devices out there, it seems the prices even for used devices are high (around 2500 SEK in Sweden, ~380 USD). I ended up buying a brand new one for 3200 SEK (~500 USD) which felt expensive, especially after recalling the recent $199 sale for Nexus 4. Noticing that brand new Nexus 4 devices are still over 3000 SEK in Sweden comforted me a bit. I would have preferred a more robust phone, like the CAT B15, but the state of free software OSes on them seem unclear and I wanted something stable. So, enough about the background, let’s get started.

Building and installing Replicant on the device was straight forward. I followed the Replicant Samsung S3 Build instructions to build my own images. The only issue I had was that I had not set JAVA_HOME and the defaults were bad; make sure to set JAVA_HOME before building. I built everything on my Lenovo X201 running Debian Wheezy, with OpenJDK 6 as the Java implementation. Installing the newly built firmware was easy, I just followed the installation process documentation. I made sure to take a clockworkmod backup to an external SD card before wiping the old system. To get a really clean new device, I also re-formated /sdcard inside clockworkmod; I noticed there were some traces left of the old system there.

I spent about one week testing various configurations before settling on something I could use daily. A fair amount of time was spent looking into backup and restore options for Android devices. My idea was that I would take a backup of the apps I ran on the N4 and transfer them to the S3. The Android Debug Bridge (adb) has a backup/restore command, however it (intentionally) ignores apps marked as allowBackup=false which a number of apps has. It doesn’t seem possible to override that settings — so much for the freedom to backup your own device. I then discovered oandbackup. It can backup your entire system, saving each app (together with associated data) into a separate directory, for simple review and inspection before restore. You can do batch backups and batch restore. I couldn’t get it to automatically restore things, though, which would be neat for really automated re-installations (there is an open issue about this feature). After noticing that some apps did not like being moved from the N4 (running Android 4.2) to the S3 (running 4.0), I ended up installing most apps from scratch on a freshly installed Replicant. I use oandbackup to the external SD card so that I can quickly restore my phone. For backup/restore of SMS/MMS and Call Log, I use SMS Backup+ against my own IMAP server. Camera pictures are synced manually using adb when I am connected to my laptop.

There is a number of apps that deserve to be mentioned because they are what I use on a daily basis. All of them come via the free software market F-Droid. For email (IMAP/SMTP), I use K-9 Mail which is feature rich but still easy to use. For chat, I use Xabber. I use NewsBlur‘s free software app to read RSS flows. For two-factor authentication, I use Google Authenticator. I haven’t evaluated different PDF viewers, but the first one I tried (APV PDF Viewer) has worked fine so far. Handling a a synchronized address book and calendar deserve its own blog post because it is a challenging topic, but briefly, I’m currently using a combination of aCal and DAVdroid.

Finally, since Replicant is still work in progress, some words about stability and notes on what doesn’t work. This is probably the most interesting part if you are considering running Replicant on an S3 yourself. Overall system stability is flawless, I hadn’t had any crash or problem with the fundamental functionality (telephony, 3G, Camera). People have said graphics feels a bit laggy, but I cannot compare with the stock ROM and it doesn’t get in the way of daily use. First some notes about non-free aspects:

  • Bluetooth doesn’t work by default. After installing /system/vendor/firmware/bcm4334.hcd (MD5 b6207104da0ca4a0b1da66448af7ed4c) pairing and testing with a Bluetooth headset worked fine.
  • Front camera doesn’t work by default. After installing /system/vendor/firmware/fimc_is_fw.bin (MD5 52eeaf0889bc9206860075cd9b7f80b9and) and /system/vendor/firmware/setfile.bin (MD5 0e6fdeb378fb154c39fd508ae28eaf2a) it works. The extensions are *.bin but I don’t believe this code is executed on the main CPU.
  • GPS doesn’t work by default. After installing /system/bin/gpsd (MD5 6757ed2e2a283259e67c62e6b2b9cfef), /system/lib/libsecril-client.so (MD5 a836df0f947d2aa2ef20dcb213317517), /system/lib/hw/gps.exynos4.so (MD5 1ea1d67f297dd52d59d40dbad9b02a0a) it works. This is code that runs on the main CPU, and even more alarming, it embeds a copy of OpenSSL and talks to various online servers for A-GPS, and is thus a likely (and probably not too challenging) attack vector for anyone wanting to remotely exploit any phone.
  • Wifi doesn’t work, and I haven’t gotten this to work. There is a list of non-free S3 firmware on the Replicant wiki however my stock ROM did not ship with those files. I don’t believe any of the blobs run on the main CPU.
  • NFC doesn’t work, and I haven’t gotten it to work. It seems the infrastructure for NFC is missing in Replicant 4.0, it doesn’t even expose the NFC hardware permission capability. This is quite unfortunate for me, since I daily work with YubiKey NEO and would have preferred to replace Google Authenticator with the YubiOATH that uses the NEO for OATH secret storage.

Some other observations:

  • Panorama mode in the Camera crashes; see issue about it.
  • There is a number of smaller graphical issues. I believe these are related to the EGL but haven’t understood the details. What I’ve noticed are the following issues. The task switcher doesn’t show mini screenshots of all running apps (the screenshots are just black). ZXing is not able to QR decode images (I’m told this is because Replicant uses a RGB color plane instead of the required YUV color plane). Video playback in the gallery is laggy to the point of being unusable. Video playback on Youtube in the default web browser works in full screen (not laggy), but not embedded in the webpage.
  • MTP has been a bit unreliable, my main laptop is able to import photos, but another laptop (also running Debian Wheezy) just stalls when talking to it. This may be a host issue, I have experience similar issues with a Nexus 7 2nd generation device.

I am quite happy with the setup so far, and I will continue to use it as my primary phone.

Small syslog server

My home network has several devices that do not have large persistent storage to keep log files. For example, my wireless routers based on OpenWRT doesn’t log to the limited local storage it has, and a Flukso energy metering device log power readings to a ramdisk. These devices log a fair amount of information that I ideally would like to keep for later analysis. I have never before seen a need to setup a syslogd server, thinking that storing logs locally and keeping regular backups of the machine is good enough. However, it appears like this situation calls for a syslogd server. I found an old NSLU2 in my drawer and installed Debian Squeeze on it following Martin Michlmayr’s instructions. I’m using a 4GB USB memory stick for storage, which should hold plenty of log data. I keep backups of the machine in case the USB memory stick wears out.

After customizing the installation to my preferences (disable ssh passwords, disable portmap/rpc.statd/exim4, installing etckeeper, emacs23-nox, etc) I am ready to configure Rsyslog. I found what looked like the perfect configuration example, “Storing messages from a remote system into a specific file”, but it requires me to hard code a bit too much information in the configuration file for my taste. Instead, I found the DynFile concept. With a file /etc/rsyslogd.d/logger.conf as below I can point any new device to my log server and it will automatically create a new file for it. And since the dates are embedded into the filename, I get log rotation suitable for rsync-style backups for free.

$ModLoad imudp
$UDPServerRun 514

$ModLoad imtcp
$InputTCPServerRun 514

$template DynFile,”/var/log/network-%HOSTNAME%-%$year%-%$month%-%$day%.log”
:fromhost-ip, !isequal, “127.0.0.1″ ?DynFile
:fromhost-ip, !isequal, “127.0.0.1″ ~

After this, I get log files written to /var/log/network-IP-YEAR-MONTH-DAY.log. For example:

pepparkaka:~# tail /var/log/network-192.168.1.47-2012-03-20.log 
Mar 20 13:40:21 192.168.1.47 avahi-daemon[1508]: Registering new address record for 192.168.1.47 on br-lan.IPv4.
Mar 20 13:40:21 192.168.1.47 avahi-daemon[1508]: Registering HINFO record with values 'MIPS'/'LINUX'.
Mar 20 13:40:21 192.168.1.47 sysinit: setting up led WAN LED (green)
Mar 20 13:40:21 192.168.1.47 kernel: ar71xx-wdt: enabling watchdog timer

Use uci to configure the OpenWRT boxes to send log messages to this server:

uci set system.@system[0].log_ip=192.168.1.51
uci commit

Update! By default rsylog performs reverse lookups of incoming requests. This easily causes problems in case your DNS server is unreachable. Rsyslogd appears to have a long timeout for DNS queries, so if you expect incoming log messages to end up in the log when they are sent, think again. In my testing, it can take minutes until they end up in the log. For me, reverse DNS lookups does not add anything of value. To disable DNS lookups, make sure rsyslogd is invoked with the ‘-x’ parameter. On Debian, this is done by adding ‘-x’ to /etc/defaults/rsyslog like this:

RSYSLOGD_OPTIONS=”-c4 -x”

OpenWRT with Huawei E367 and TP-Link TL-WR1043ND

The ability to connect a 3G modem to a wireless router to form a Internet connected ad-hoc network of machines is very powerful. I’ve done this many times and have written about it before (e.g., see my OpenWRT writeup page) but I recently did it with modern hardware again. Here I will use the TP-Link TL-WR1043ND wireless router (available here for around $50) together with the Huawei E367 3G UMTS/HSDPA modem. Other wireless routers and modem should work fine. The software is OpenWRT 10.03 although I hope to redo this with LibreWRT eventually. My writeup is mostly focused around what is happening around the prompt, so it is mostly a cut’n'paste terminal session with a comment interlined.

TL-WR1043ND and Huawei 367

Router and modem


Continue reading

OpenWRT 10.03 “Backfire”

Earlier I have written about OpenWRT configuration for two routers in a home network and OpenWRT configuration for 3G dial-up (which succeeded my summerhouse OpenWRT writeup) before. The OpenWRT project recently announced a new release, OpenWRT 10.03 Backfire. Thus, this appeared like a good opportunity to wipe out the old configurations on my routers and rewrite the articles using the latest software. I have two articles:

Home Wireless Network

Using OpenWRT with WPA-PSK 2 on Broadcom WLAN routers have been stuck on a quite old bug. Recently someone suggested that it may have been fixed in trunk, which caused me to test it. And it works!

It took some time to work out the details here. To save myself time to reconstruct the commands, and hopefully save you some time too, I wrote down how to use OpenWRT with two Asus WL-500g Premium linked together wirelessly using WDS and PSK2 encryption.

The writeup is long, so I put it on a separate page:

http://josefsson.org/openwrt/wlan.html.

If you are interested in using OpenWRT with a 3G connection, you may find my summer house internet writeup more useful.

Home Audio Server

Procrastinating real work, I documented my home audio server setup. I needed a cross-platform solution, and as a first step, I settled with MPD. The setup is only a few days old, and I may decide to change software eventually. But the current setup works under Gnome, Windows, Mac OS X and even on my Nokia 6233.

Home Audio Server

What may be missing is FM/DAB Radio and streaming of TV, but I’m not sure the little NSLU2 is up to it. We’ll see.

The writeup on how to do this is long, so I put it at a separate page:
http://josefsson.org/grisslan/audio.html

(This is a continuation of my series to document the devices that run my home, the first was the internet setup).

Building GnuTLS and GNU SASL without running ./configure

Sometimes it can be useful to build things without the autoconf ./configure machinery, and just use a simple and hand-maintained makefile and config.h. This is needed to build things in older uClinux environments. I wrote some instructions on how to build GnuTLS and GNU SASL, and their dependencies (libgpg-error, libgcrypt, libtasn1) without running ./configure, see:

http://josefsson.org/uclinux/old/

The makefile/config.h aren’t specific to uClinux, so if you for some reason need to build these projects in some other environment, without autoconf, the files may be useful.

(Although if you want to build GnuTLS/GSASL properly under a modern uClinux, you’ll be better of reading an earlier post.)

Linksys WRT54G3G + Huawei E600 + OpenWRT Kamikaze = Internet at summer house

Spending vacation at the summer house without Internet connectivity? Unthinkable.

Linksys WRT54G3GHuawei E600

The first few days, I connected the laptop to my cell phone using Bluetooth, and then to the Internet using 3G/UMTS.

However, we have more than one laptop here, and the range of bluetooth is limited. I ended up setting up a wireless access point with a PCMCIA slot for a 3G/UMTS card. It has worked flawlessly for several days.

The writeup on how to do this is long, so I put it at a separate page:
http://josefsson.org/openwrt/internet.html

Update: I have written a similar howto for OpenWRT 8.09 and Huawei E220, see http://josefsson.org/openwrt/dongle.html

Porting to uClinux

Building software for embedded systems is quite simple today. A returning customer asked me to clarify how to build gsasl and gnutls under uClinux, and I finally created a web page collecting the instructions and patch.

http://josefsson.org/uclinux/