I’ve always found the operation of apt software package repositories to be a mystery. There appears to be a lack of transparency into which people have access to important apt package repositories out there, how the automatic non-human update mechanism is implemented, and what changes are published. I’m thinking of big distributions like Ubuntu and Debian, but also the free GNU/Linux distributions like Trisquel and PureOS that are derived from the more well-known distributions.
As far as I can tell, anyone who has the OpenPGP private key trusted by a apt-based GNU/Linux distribution can sign a modified Release/InRelease file and if my machine somehow downloads that version of the release file, my machine could be made to download and install packages that the distribution didn’t intend me to install. Further, it seems that anyone who has access to the main HTTP server, or any of its mirrors, or is anywhere on the network between them and my machine (when plaintext HTTP is used), can either stall security updates on my machine (on a per-IP basis), or use it to send my machine (again, on a per-IP basis to avoid detection) a modified Release/InRelease file if they had been able to obtain the private signing key for the archive. These are mighty powers that warrant overview.
I’ve always put off learning about the processes to protect the apt infrastructure, mentally filing it under “so many people rely on this infrastructure that enough people are likely to have invested time reviewing and improving these processes”. Simultaneous, I’ve always followed the more free-software friendly Debian-derived distributions such as gNewSense and have run it on some machines. I’ve never put them into serious production use, because the trust issues with their apt package repositories has been a big question mark for me. The “enough people” part of my rationale for deferring this is not convincing. Even the simple question of “is someone updating the apt repository” is not easy to understand on a running gNewSense system. At some point in time the gNewSense cron job to pull in security updates from Debian must have stopped working, and I wouldn’t have had any good mechanism to notice that. Most likely it happened without any public announcement. I’ve recently switched to Trisquel on production machines, and these questions has come back to haunt me.
The situation is unsatisfying and I looked into what could be done to improve it. I could try to understand who are the key people involved in each project, and may even learn what hardware component is used, or what software is involved to update and sign apt repositories. Is the server running non-free software? Proprietary BIOS or NIC firmware? Are the GnuPG private keys on disk? Smartcard? TPM? YubiKey? HSM? Where is the server co-located, and who has access to it? I tried to do a bit of this, and discovered things like Trisquel having a DSA1024 key in its default apt trust store (although for fairness, it seems that apt by default does not trust such signatures). However, I’m not certain understanding this more would scale to securing my machines against attacks on this infrastructure. Even people with the best intentions, and the state of the art hardware and software, will have problems.
To increase my trust in Trisquel I set out to understand how it worked. To make it easier to sort out what the interesting parts of the Trisquel archive to audit further were, I created debdistdiff to produce human readable text output comparing one apt archive with another apt archive. There is a GitLab CI/CD cron job that runs this every day, producing output comparing Trisquel vs Ubuntu and PureOS vs Debian. Working with these output files has made me learn more about how the process works, and I even stumbled upon something that is likely a bug where Trisquel aramo was imported from Ubuntu jammy while it contained a couple of package (e.g., gcc-8, python3.9) that were removed for the final Ubuntu jammy release.
After working on auditing the Trisquel archive manually that way, I realized that whatever I could tell from comparing Trisquel with Ubuntu, it would only be something based on a current snapshot of the archives. Tomorrow it may look completely different. What felt necessary was to audit the differences of the Trisquel archive continously. I was quite happy to have developed debdistdiff for one purpose (comparing two different archives like Trisquel and Ubuntu) and discovered that the tool could be used for another purpose (comparing the Trisquel archive at two different points in time). At this time I realized that I needed a log of all different apt archive metadata to be able to produce an audit log of the differences in time for the archive. I create manually curated git-repositories with the Release/InRelease and the Packages files for each architecture/component of the well-known distributions Trisquel, Ubuntu, Debian and PureOS. Eventually I wrote scripts to automate this, which are now published in the debdistget project.
At this point, one of the early question about per-IP substitution of Release files were lingering in my mind. However with the tooling I now had available, coming up with a way to resolve this was simple! Merely have apt compute a SHA256 checksum of the just downloaded InRelease file, and see if my git repository had the same file. At this point I started reading the Apt source code, and now I had more doubts about the security of my systems than I ever had before. Oh boy how the name Apt has never before felt more… Apt?! Oh well, we must leave some exercises for the students. Eventually I realized I wanted to touch as little of apt code basis as possible, and noticed the SigVerify::CopyAndVerify function called ExecGPGV which called apt-key verify which called GnuPG’s gpgv. By setting Apt::Key::gpgvcommand I could get apt-key verify to call another tool than gpgv. See where I’m going? I thought wrapping this up would now be trivial but for some reason the hash checksum I computed locally never matched what was on my server. I gave up and started working on other things instead.
Today I came back to this idea, and started to debug exactly how the local files looked that I got from apt and how they differed from what I had in my git repositories, that came straight from the apt archives. Eventually I traced this back to SplitClearSignedFile which takes an InRelease file and splits it into two files, probably mimicking the (old?) way of distributing both Release and Release.gpg. So the clearsigned InRelease file is split into one cleartext file (similar to the Release file) and one OpenPGP signature file (similar to the Release.gpg file). But why didn’t the cleartext variant of the InRelease file hash to the same value as the hash of the Release file? Sadly they differ by the final newline.
Having solved this technicality, wrapping the pieces up was easy, and I came up with a project apt-canary that provides a script apt-canary-gpgv that verify the local apt release files against something I call a “apt canary witness” file stored at a URL somewhere.
I’m now running apt-canary on my Trisquel aramo laptop, a Trisquel nabia server, and Talos II ppc64el Debian machine. This means I have solved the per-IP substitution worries (or at least made them less likely to occur, having to send the same malicious release files to both GitLab and my system), and allow me to have an audit log of all release files that I actually use for installing and downloading packages.
What do you think? There are clearly a lot of work and improvements to be made. This is a proof-of-concept implementation of an idea, but instead of refining it until perfection and delaying feedback, I wanted to publish this to get others to think about the problems and various ways to resolve them.
Btw, I’m going to be at FOSDEM’23 this weekend, helping to manage the Security Devroom. Catch me if you want to chat about this or other things. Happy Hacking!
I use GnuPG to compute cryptographic signatures for my emails, git commits/tags, and software release artifacts (tarballs). Part of GnuPG is gpg-agent which talks to OpenSSH, which I login to remote servers and to clone git repositories. I dislike storing cryptographic keys on general-purpose machines, and have used hardware-backed OpenPGP keys since around 2006 when I got a FSFE Fellowship Card. GnuPG via gpg-agent handles this well, and the private key never leaves the hardware. The ZeitControl cards were (to my knowledge) proprietary hardware running some non-free operating system and OpenPGP implementation. By late 2012 the YubiKey NEO supported OpenPGP, and while the hardware and operating system on it was not free, at least it ran a free software OpenPGP implementation and eventually I setup my primary RSA key on it. This worked well for a couple of years, and when I in 2019 wished to migrate to a new key, the FST-01G device with open hardware running free software that supported Ed25519 had become available. I created a key and have been using the FST-01G on my main laptop since then. This little device has been working, the signature counter on it is around 14501 which means around 10 signatures/day since then!
Currently I am in the process of migrating towards a new laptop, and moving the FST-01G device between them is cumbersome, especially if I want to use both laptops in parallel. That’s why I need to setup a new hardware device to hold my OpenPGP key, which can go with my new laptop. This is a good time to re-visit alternatives. I quickly decided that I did not want to create a new key, only to import my current one to keep everything working. My requirements on the device to chose hasn’t changed since 2019, see my summary at the end of the earlier blog post. Unfortunately the FST-01G is out of stock and the newer FST-01SZ has also out of stock. While Tillitis looks promising (and I have one to play with), it does not support OpenPGP (yet). What to do? Fortunately, I found some FST-01SZ device in my drawer, and decided to use it pending a more satisfactory answer. Hopefully once I get around to generate a new OpenPGP key in a year or so, I will do a better survey of options that are available on the market then. What are your (freedom-respecting) OpenPGP hardware recommendations?
FST-01SZ circuit board
Similar to setting up the FST-01G, the FST-01SZ needs to be setup before use. I’m doing the following from Trisquel 11 but any GNU/Linux system would work. When the device is inserted at first time, some kernel messages are shown (see /var/log/syslog or use the dmesg command):
usb 3-3: new full-speed USB device number 39 using xhci_hcd
usb 3-3: New USB device found, idVendor=234b, idProduct=0004, bcdDevice= 2.00
usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 3-3: Product: Fraucheky
usb 3-3: Manufacturer: Free Software Initiative of Japan
usb 3-3: SerialNumber: FSIJ-0.0
usb-storage 3-3:1.0: USB Mass Storage device detected
scsi host1: usb-storage 3-3:1.0
scsi 1:0:0:0: Direct-Access FSIJ Fraucheky 1.0 PQ: 0 ANSI: 0
sd 1:0:0:0: Attached scsi generic sg2 type 0
sd 1:0:0:0: [sdc] 128 512-byte logical blocks: (65.5 kB/64.0 KiB)
sd 1:0:0:0: [sdc] Write Protect is off
sd 1:0:0:0: [sdc] Mode Sense: 03 00 00 00
sd 1:0:0:0: [sdc] No Caching mode page found
sd 1:0:0:0: [sdc] Assuming drive cache: write through
sdc:
sd 1:0:0:0: [sdc] Attached SCSI removable disk
Interestingly, the NeuG software installed on the device I got appears to be version 1.0.9:
jas@kaka:~$ head /media/jas/Fraucheky/README
NeuG - a true random number generator implementation
Version 1.0.9
2018-11-20
Niibe Yutaka
Free Software Initiative of Japan
What's NeuG?
============
jas@kaka:~$
I could not find version 1.0.9 published anywhere, but the device came with a SD-card that contain a copy of the source, so I uploaded it until a more canonical place is located. Putting the device in the serial mode can be done using a sudo eject /dev/sdc command which results in the following syslog output.
usb 3-3: reset full-speed USB device number 39 using xhci_hcd
usb 3-3: device firmware changed
usb 3-3: USB disconnect, device number 39
sdc: detected capacity change from 128 to 0
usb 3-3: new full-speed USB device number 40 using xhci_hcd
usb 3-3: New USB device found, idVendor=234b, idProduct=0001, bcdDevice= 2.00
usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 3-3: Product: NeuG True RNG
usb 3-3: Manufacturer: Free Software Initiative of Japan
usb 3-3: SerialNumber: FSIJ-1.0.9-42315277
cdc_acm 3-3:1.0: ttyACM0: USB ACM device
Now download Gnuk, verify its integrity and build it. You may need some additional packages installed, try apt-get install gcc-arm-none-eabi openocd python3-usb. As you can see, I’m using the stable 1.2 branch of Gnuk, currently on version 1.2.20. The ./configure parameters deserve some explanation. The kdf_do=required sets up the device to require KDF usage. The --enable-factory-reset allows me to use the command factory-reset (with admin PIN) inside gpg --card-edit to completely wipe the card. Some may consider that too dangerous, but my view is that if someone has your admin PIN it is game over anyway. The --vidpid=234b:0000 is specifies the USB VID/PID to use, and --target=FST_01SZ is critical to set the platform (you’ll may brick the device if you pick the wrong --target setting).
jas@kaka:~/src$ rm -rf gnuk neug
jas@kaka:~/src$ git clone https://gitlab.com/jas/neug.git
Cloning into 'neug'...
remote: Enumerating objects: 2034, done.
remote: Counting objects: 100% (2034/2034), done.
remote: Compressing objects: 100% (603/603), done.
remote: Total 2034 (delta 1405), reused 2013 (delta 1405), pack-reused 0
Receiving objects: 100% (2034/2034), 910.34 KiB | 3.50 MiB/s, done.
Resolving deltas: 100% (1405/1405), done.
jas@kaka:~/src$ git clone https://salsa.debian.org/gnuk-team/gnuk/gnuk.git
Cloning into 'gnuk'...
remote: Enumerating objects: 13765, done.
remote: Counting objects: 100% (959/959), done.
remote: Compressing objects: 100% (337/337), done.
remote: Total 13765 (delta 629), reused 907 (delta 599), pack-reused 12806
Receiving objects: 100% (13765/13765), 12.59 MiB | 3.05 MiB/s, done.
Resolving deltas: 100% (10077/10077), done.
jas@kaka:~/src$ cd neug
jas@kaka:~/src/neug$ git describe
release/1.0.9
jas@kaka:~/src/neug$ git tag -v `git describe`
object 5d51022a97a5b7358d0ea62bbbc00628c6cec06a
type commit
tag release/1.0.9
tagger NIIBE Yutaka <gniibe@fsij.org> 1542701768 +0900
Version 1.0.9.
gpg: Signature made Tue Nov 20 09:16:08 2018 CET
gpg: using EDDSA key 249CB3771750745D5CDD323CE267B052364F028D
gpg: issuer "gniibe@fsij.org"
gpg: Good signature from "NIIBE Yutaka <gniibe@fsij.org>" [unknown]
gpg: aka "NIIBE Yutaka <gniibe@debian.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 249C B377 1750 745D 5CDD 323C E267 B052 364F 028D
jas@kaka:~/src/neug$ cd ../gnuk/
jas@kaka:~/src/gnuk$ git checkout STABLE-BRANCH-1-2
Branch 'STABLE-BRANCH-1-2' set up to track remote branch 'STABLE-BRANCH-1-2' from 'origin'.
Switched to a new branch 'STABLE-BRANCH-1-2'
jas@kaka:~/src/gnuk$ git describe
release/1.2.20
jas@kaka:~/src/gnuk$ git tag -v `git describe`
object 9d3c08bd2beb73ce942b016d4328f0a596096c02
type commit
tag release/1.2.20
tagger NIIBE Yutaka <gniibe@fsij.org> 1650594032 +0900
Gnuk: Version 1.2.20
gpg: Signature made Fri Apr 22 04:20:32 2022 CEST
gpg: using EDDSA key 249CB3771750745D5CDD323CE267B052364F028D
gpg: Good signature from "NIIBE Yutaka <gniibe@fsij.org>" [unknown]
gpg: aka "NIIBE Yutaka <gniibe@debian.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 249C B377 1750 745D 5CDD 323C E267 B052 364F 028D
jas@kaka:~/src/gnuk/src$ git submodule update --init
Submodule 'chopstx' (https://salsa.debian.org/gnuk-team/chopstx/chopstx.git) registered for path '../chopstx'
Cloning into '/home/jas/src/gnuk/chopstx'...
Submodule path '../chopstx': checked out 'e12a7e0bb3f004c7bca41cfdb24c8b66daf3db89'
jas@kaka:~/src/gnuk$ cd chopstx
jas@kaka:~/src/gnuk/chopstx$ git describe
release/1.21
jas@kaka:~/src/gnuk/chopstx$ git tag -v `git describe`
object e12a7e0bb3f004c7bca41cfdb24c8b66daf3db89
type commit
tag release/1.21
tagger NIIBE Yutaka <gniibe@fsij.org> 1650593697 +0900
Chopstx: Version 1.21
gpg: Signature made Fri Apr 22 04:14:57 2022 CEST
gpg: using EDDSA key 249CB3771750745D5CDD323CE267B052364F028D
gpg: Good signature from "NIIBE Yutaka <gniibe@fsij.org>" [unknown]
gpg: aka "NIIBE Yutaka <gniibe@debian.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 249C B377 1750 745D 5CDD 323C E267 B052 364F 028D
jas@kaka:~/src/gnuk/chopstx$ cd ../src
jas@kaka:~/src/gnuk/src$ kdf_do=required ./configure --enable-factory-reset --vidpid=234b:0000 --target=FST_01SZ
Header file is: board-fst-01sz.h
Debug option disabled
Configured for bare system (no-DFU)
PIN pad option disabled
CERT.3 Data Object is NOT supported
Card insert/removal by HID device is NOT supported
Life cycle management is supported
Acknowledge button is supported
KDF DO is required before key import/generation
jas@kaka:~/src/gnuk/src$ make | less
jas@kaka:~/src/gnuk/src$ cd ../regnual/
jas@kaka:~/src/gnuk/regnual$ make | less
jas@kaka:~/src/gnuk/regnual$ cd ../../
jas@kaka:~/src$ sudo python3 neug/tool/neug_upgrade.py -f gnuk/regnual/regnual.bin gnuk/src/build/gnuk.bin
gnuk/regnual/regnual.bin: 4608
gnuk/src/build/gnuk.bin: 109568
CRC32: b93ca829
Device:
Configuration: 1
Interface: 1
20000e00:20005000
Downloading flash upgrade program...
start 20000e00
end 20002000
# 20002000: 32 : 4
Run flash upgrade program...
Wait 1 second...
Wait 1 second...
Device:
08001000:08020000
Downloading the program
start 08001000
end 0801ac00
jas@kaka:~/src$
The kernel log will contain the following, and the card is ready to use as an OpenPGP card. You may unplug it and re-insert it as you wish.
usb 3-3: reset full-speed USB device number 41 using xhci_hcd
usb 3-3: device firmware changed
usb 3-3: USB disconnect, device number 41
usb 3-3: new full-speed USB device number 42 using xhci_hcd
usb 3-3: New USB device found, idVendor=234b, idProduct=0000, bcdDevice= 2.00
usb 3-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 3-3: Product: Gnuk Token
usb 3-3: Manufacturer: Free Software Initiative of Japan
usb 3-3: SerialNumber: FSIJ-1.2.20-42315277
Setting up the card is the next step, and there are many tutorials around for this, eventually I settled with the following sequence. Let’s start with setting the admin PIN. First make sure that pcscd nor scdaemon is running, which is good hygien since those processes cache some information and with a stale connection this easily leads to confusion. Cache invalidation… sigh.
jas@kaka:~$ gpg-connect-agent "SCD KILLSCD" "SCD BYE" /bye
jas@kaka:~$ ps auxww|grep -e pcsc -e scd
jas 30221 0.0 0.0 3468 1692 pts/3 R+ 11:49 0:00 grep --color=auto -e pcsc -e scd
jas@kaka:~$ gpg --card-edit
Reader ...........: 234B:0000:FSIJ-1.2.20-42315277:0
Application ID ...: D276000124010200FFFE423152770000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 42315277
Name of cardholder: [not set]
Language prefs ...: [not set]
Salutation .......:
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa2048 rsa2048 rsa2048
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
KDF setting ......: off
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
gpg/card> admin
Admin commands are allowed
gpg/card> kdf-setup
gpg/card> passwd
gpg: OpenPGP card no. D276000124010200FFFE423152770000 detected
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? 3
PIN changed.
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection?
Now it would be natural to setup the PIN and reset code. However the Gnuk software is configured to not allow this until the keys are imported. You would get the following somewhat cryptical error messages if you try. This took me a while to understand, since this is device-specific, and some other OpenPGP implementations allows you to configure a PIN and reset code before key import.
Your selection? 4
Error setting the Reset Code: Card error
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? 1
Error changing the PIN: Conditions of use not satisfied
1 - change PIN
2 - unblock PIN
3 - change Admin PIN
4 - set the Reset Code
Q - quit
Your selection? q
Continue to configure the card and make it ready for key import. Some settings deserve comments. The lang field may be used to setup the language, but I have rarely seen it use, and I set it to ‘sv‘ (Swedish) mostly to be able to experiment if any software adhears to it. The URL is important to point to somewhere where your public key is stored, the fetch command of gpg --card-edit downloads it and sets up GnuPG with it when you are on a clean new laptop. The forcesig command changes the default so that a PIN code is not required for every digital signature operation, remember that I averaged 10 signatures per day for the past 2-3 years? Think of the wasted energy typing those PIN codes every time! Changing the cryptographic key type is required when I import 25519-based keys.
gpg/card> name
Cardholder's surname: Josefsson
Cardholder's given name: Simon
gpg/card> lang
Language preferences: sv
gpg/card> sex
Salutation (M = Mr., F = Ms., or space): m
gpg/card> login
Login data (account name): jas
gpg/card> url
URL to retrieve public key: https://josefsson.org/key-20190320.txt
gpg/card> forcesig
gpg/card> key-attr
Changing card key attribute for: Signature key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: ed25519
Note: There is no guarantee that the card supports the requested size.
If the key generation does not succeed, please check the
documentation of your card to see what sizes are allowed.
Changing card key attribute for: Encryption key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: cv25519
Changing card key attribute for: Authentication key
Please select what kind of key you want:
(1) RSA
(2) ECC
Your selection? 2
Please select which elliptic curve you want:
(1) Curve 25519
(4) NIST P-384
Your selection? 1
The card will now be re-configured to generate a key of type: ed25519
gpg/card>
Reader ...........: 234B:0000:FSIJ-1.2.20-42315277:0
Application ID ...: D276000124010200FFFE423152770000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 42315277
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Salutation .......: Mr.
URL of public key : https://josefsson.org/key-20190320.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: ed25519 cv25519 ed25519
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 3 3
Signature counter : 0
KDF setting ......: on
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]
gpg/card>
The device is now ready for key import! Bring out your offline laptop and boot it and use the keytocard command on the subkeys to import them. This assumes you saved a copy of the GnuPG home directory after generating the master and subkeys before, which I did in my own previous tutorial when I generated the keys. This may be a bit unusual, and there are simpler ways to do this (e.g., import a copy of the secret keys into a fresh GnuPG home directory).
$ cp -a gnupghome-backup-mastersubkeys gnupghome-import-fst01sz-42315277-2022-12-24
$ ps auxww|grep -e pcsc -e scd
$ gpg --homedir $PWD/gnupghome-import-fst01sz-42315277-2022-12-24 --edit-key B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE
...
Secret key is available.
gpg: checking the trustdb
gpg: marginals needed: 3 completes needed: 1 trust model: pgp
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> key 1
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb* cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> keytocard
Please select where to store the key:
(2) Encryption key
Your selection? 2
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb* cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> key 1
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> key 2
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb* ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> keytocard
Please select where to store the key:
(3) Authentication key
Your selection? 3
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb* ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> key 2
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> key 3
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb* ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> keytocard
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1
sec ed25519/D73CF638C53C06BE
created: 2019-03-20 expired: 2019-10-22 usage: SC
trust: ultimate validity: expired
ssb cv25519/02923D7EE76EBD60
created: 2019-03-20 expired: 2019-10-22 usage: E
ssb ed25519/80260EE8A9B92B2B
created: 2019-03-20 expired: 2019-10-22 usage: A
ssb* ed25519/51722B08FE4745A2
created: 2019-03-20 expired: 2019-10-22 usage: S
[ expired] (1). Simon Josefsson <simon@josefsson.org>
gpg> quit
Save changes? (y/N) y
$
Now insert it into your daily laptop and have GnuPG and learn about the new private keys and forget about any earlier locally available card bindings — this usually manifests itself by GnuPG asking you to insert a OpenPGP card with another serial number. Earlier I did rm -rf ~/.gnupg/private-keys-v1.d/ but the scd serialno followed by learn --force is nicer. I also sets up trust setting for my own key.
jas@kaka:~$ gpg-connect-agent "scd serialno" "learn --force" /bye
...
jas@kaka:~$ echo "B1D2BD1375BECB784CF4F8C4D73CF638C53C06BE:6:" | gpg --import-ownertrust
jas@kaka:~$ gpg --card-status
Reader ...........: 234B:0000:FSIJ-1.2.20-42315277:0
Application ID ...: D276000124010200FFFE423152770000
Application type .: OpenPGP
Version ..........: 2.0
Manufacturer .....: unmanaged S/N range
Serial number ....: 42315277
Name of cardholder: Simon Josefsson
Language prefs ...: sv
Salutation .......: Mr.
URL of public key : https://josefsson.org/key-20190320.txt
Login data .......: jas
Signature PIN ....: not forced
Key attributes ...: ed25519 cv25519 ed25519
Max. PIN lengths .: 127 127 127
PIN retry counter : 5 5 5
Signature counter : 3
KDF setting ......: on
Signature key ....: A3CC 9C87 0B9D 310A BAD4 CF2F 5172 2B08 FE47 45A2
created ....: 2019-03-20 23:40:49
Encryption key....: A9EC 8F4D 7F1E 50ED 3DEF 49A9 0292 3D7E E76E BD60
created ....: 2019-03-20 23:40:26
Authentication key: CA7E 3716 4342 DF31 33DF 3497 8026 0EE8 A9B9 2B2B
created ....: 2019-03-20 23:40:37
General key info..: sub ed25519/51722B08FE4745A2 2019-03-20 Simon Josefsson <simon@josefsson.org>
sec# ed25519/D73CF638C53C06BE created: 2019-03-20 expires: 2023-09-19
ssb> ed25519/80260EE8A9B92B2B created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
ssb> ed25519/51722B08FE4745A2 created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
ssb> cv25519/02923D7EE76EBD60 created: 2019-03-20 expires: 2023-09-19
card-no: FFFE 42315277
jas@kaka:~$
Verify that you can digitally sign and authenticate using the key and you are done!
jas@kaka:~$ echo foo|gpg -a --sign|gpg --verify
gpg: Signature made Sat Dec 24 13:49:59 2022 CET
gpg: using EDDSA key A3CC9C870B9D310ABAD4CF2F51722B08FE4745A2
gpg: Good signature from "Simon Josefsson <simon@josefsson.org>" [ultimate]
jas@kaka:~$ ssh-add -L
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILzCFcHHrKzVSPDDarZPYqn89H5TPaxwcORgRg+4DagE cardno:FFFE42315277
jas@kaka:~$
So time to relax and celebrate christmas? Hold on… not so fast! Astute readers will have noticed that the output said ‘PIN retry counter: 5 5 5‘. That’s not the default PIN retry counter for Gnuk! How did that happen? Indeed, good catch and great question, my dear reader. I wanted to include how you can modify the Gnuk source code, re-build it and re-flash the Gnuk as well. This method is different than flashing Gnuk onto a device that is running NeuG so the commands I used to flash the firmware in the start of this blog post no longer works in a device running Gnuk. Fortunately modern Gnuk supports updating firmware by specifying the Admin PIN code only, and provides a simple script to achieve this as well. The PIN retry counter setting is hard coded in the openpgp-do.c file, and we run a a perl command to modify the file, rebuild Gnuk and upgrade the FST-01SZ. This of course wipes all your settings, so you will have the opportunity to practice all the commands earlier in this post once again!
jas@kaka:~/src/gnuk/src$ perl -pi -e 's/PASSWORD_ERRORS_MAX 3/PASSWORD_ERRORS_MAX 5/' openpgp-do.c
jas@kaka:~/src/gnuk/src$ make | less
jas@kaka:~/src/gnuk/src$ cd ../tool/
jas@kaka:~/src/gnuk/tool$ ./upgrade_by_passwd.py
Admin password:
Device:
Configuration: 1
Interface: 0
../regnual/regnual.bin: 4608
../src/build/gnuk.bin: 110592
CRC32: b93ca829
Device:
Configuration: 1
Interface: 0
20002a00:20005000
Downloading flash upgrade program...
start 20002a00
end 20003c00
Run flash upgrade program...
Waiting for device to appear:
Wait 1 second...
Wait 1 second...
Device:
08001000:08020000
Downloading the program
start 08001000
end 0801b000
Protecting device
Finish flashing
Resetting device
Update procedure finished
jas@kaka:~/src/gnuk/tool$
Now finally, I wish you all a Merry Christmas and Happy Hacking!
I’m about to migrate to a new laptop, having done a brief pre-purchase review of options on Fosstodon and reaching a decision to buy the NovaCustom NV41. Given the rapid launch and decline of Mastodon instances, I thought I’d better summarize my process and conclusion on my self-hosted blog until the fediverse self-hosting situation improves.
Since 2010 my main portable computing device has been the Lenovo X201 that replaced the Dell Precision M65 that I bought in 2006. I have been incredibly happy with the X201, even to the point that in 2015 when I wanted to find a replacement, I couldn’t settle on a decision and eventually realized I couldn’t articulate what was wrong with the X201 and decided to just buy another X201 second-hand for my second office. There is still no deal-breaker with the X201, and I’m doing most of my computing on it including writing this post. However, today I can better articulate what is lacking with the X201 that I desire, and the state of the available options on the market has improved since my last attempt in 2015.
Briefly, my desired properties are:
Portable – weight under 1.5kg
Screen size 9-14″
ISO keyboard layout, preferably Swedish layout
Mouse trackpad, WiFi, USB and external screen connector
Decent market availability: I should be able to purchase it from Sweden and have consumer protection, warranty, and some hope of getting service parts for the device
Manufactured and sold by a vendor that is supportive of free software
Preferably RJ45 connector (for data center visits)
My workload for the machine is Emacs, Firefox, Nextcloud client, GNOME, Evolution (mail & calendar), LibreOffice Calc/Writer, compiling software and some podman/qemu for testing. I have used Debian as the main operating system for the entire life of this laptop, but have experimented with PureOS recently. My current X201 is useful enough for this, although support for 4K displays and a faster machine wouldn’t hurt.
Based on my experience in 2015 that led me to make no decision, I changed perspective. This is a judgement call and I will not be able to fulfil all criteria. I will have to decide on a balance and the final choice will include elements that I really dislike, but still it will hopefully be better than nothing. The conflict for me mainly center around these parts:
Non-free BIOS. This is software that runs on the main CPU and has full control of everything. I want this to run free software as much as possible. Coreboot is the main project in this area, although I prefer the more freedom-oriented Libreboot.
Proprietary and software-upgradeable parts of the main CPU. This includes CPU microcode that is not distributed as free software. The Intel Management Engine (AMD and other CPU vendors has similar technology) falls into this category as well, and is problematic because it is an entire non-free operating system running within the CPU, with many security and freedom problems. This aspect is explored in the Libreboot FAQ further. Even if these parts can be disabled (Intel ME) or not utilized (CPU microcode), I believe the mere presence of these components in the design of the CPU is a problem, and I would prefer a CPU without these properties.
Non-free software in other microprocessors in the laptop. Ultimately, I tend agree with the FSF’s “secondary processor” argument but when it is possible to chose between a secondary processor that runs free software and one that runs proprietary software, I would prefer as many secondary processors as possible to run free software. The libreboot binary blob reduction policy describes a move towards stronger requirements.
Non-free firmware that has to be loaded during runtime into CPU or secondary processors. Using Linux-libre solves this but can cause some hardware to be unusable.
WiFi, BlueTooth and physical network interface (NIC/RJ45). This is the most notable example of secondary processor problem with running non-free software and requiring non-free firmware. Sometimes these may even require non-free drivers, although in recent years this has usually been reduced into requiring non-free firmware.
A simple choice for me would be to buy one of the FSF RYF certified laptops. Right now that list only contains the 10+ year old Lenovo series, and I actually already have a X200 with libreboot that I bought earlier for comparison. The reason the X200 didn’t work out as a replacement for me was the lack of a mouse trackpad, concerns about non-free EC firmware, Intel ME uncertainty (is it really neutralized?) and non-free CPU microcode (what are the bugs that it fixes?), but primarily that for some reason that I can’t fully articulate it feels weird to use a laptop manufactured by Lenovo but modified by third parties to be useful. I believe in market forces to pressure manufacturers into Doing The Right Thing, and feel that there is no incentive for Lenovo to use libreboot in the future when this market niche is already fulfilled by re-sellers modifying Lenovo laptops. So I’d be happier buying a laptop from someone who is natively supportive of they way I’m computing. I’m sure this aspect could be discussed a lot more, and maybe I’ll come back to do that, and could even reconsider my thinking (the right-to-repair argument is compelling). I will definitely continue to monitor the list of RYF-certified laptops to see if future entries are more suitable options for me.
Eventually I decided to buy the NovaCustom NV41 laptop, and it arrived quickly and I’m in the process of setting it up. I hope to write a separate blog about it next.
The Guile bindings for GnuTLS has been part of GnuTLS since spring 2007 when Ludovic Courtès contributed it after some initial discussion. I have been looking into getting back to do GnuTLS coding, and during a recent GnuTLS meeting one topic was Guile bindings. It seemed like a fairly self-contained project to pick up on. It is interesting to re-read the old thread when this work was included: some of the concerns brought up there now have track record to be evaluated on. My opinion that the cost of introducing a new project per language binding today is smaller than the cost of maintaining language bindings as part of the core project. I believe the cost/benefit ratio has changed during the past 15 years: introducing a new project used to come with a significant cost but this is no longer the case, as tooling and processes for packaging have improved. I have had similar experience with Java, C# and Emacs Lisp bindings for GNU Libidn as well, where maintaining them centralized slow down the pace of updates. Andreas Metzler pointed to a similar conclusion reached by Russ Allbery.
There are many ways to separate a project into two projects; just copying the files into a new git repository would have been the simplest and was my original plan. However Ludo’ mentioned git-filter-branch in an email, and the idea of keeping all git history for some of the relevant files seemed worth pursuing to me. I quickly found git-filter-repo which appears to be the recommend approach, and experimenting with it I found a way to filter out the GnuTLS repo into a small git repository that Guile-GnuTLS could be based on. The commands I used were the following, if you want to reproduce things.
I debated with myself back and forth whether to include some files that would be named the same in the new repository but would share little to no similar lines, for example configure.ac, Makefile.am not to mention README and NEWS. Initially I thought it would be nice to preserve the history for all lines that went into the new project, but this is a subjective judgement call. What brought me over to a more minimal approach was that the contributor history and attribution would be quite strange for the new repository: Should Guile-GnuTLS attribute the work of the thousands of commits to configure.ac which had nothing to do with Guile? Should the people who wrote that be mentioned as contributor of Guile-GnuTLS? I think not.
The next step was to get a reasonable GitLab CI/CD pipeline up, to make sure the project builds on some free GNU/Linux distributions like Trisquel and PureOS as well as the usual non-free distributions like Debian and Fedora to have coverage of dpkg and rpm based distributions. I included builds on Alpine and ArchLinux as well, because they tend to trigger other portability issues. I wish there were GNU Guix docker images available for easy testing on that platform as well. The GitLab CI/CD rules for a project like this are fairly simple.
To get things out of the door, I tagged the result as v3.7.9 and published a GitLab release page for Guile-GnuTLS that includes OpenPGP-signed source tarballs manually uploaded built on my laptop. The URLs for these tarballs are not very pleasant to work with, and discovering new releases automatically appears unreliable, but I don’t know of a better approach.
To finish this project, I have proposed a GnuTLS merge request to remove all Guile-related parts from the GnuTLS core.
Doing some GnuTLS-related work again felt nice, it was quite some time ago so thank you for giving me this opportunity. Thoughts or comments? Happy hacking!
To protect web resources with Kerberos you may use Apache HTTPD with mod_auth_gssapi — however, all web scripts (e.g., PHP) run under Apache will have access to the Kerberos long-term symmetric secret credential (keytab). If someone can get it, they can impersonate your server, which is bad.
This should give you a working and running web server. Verify it is operational under the proper hostname, I’ll use foo.sjd.se in this writeup.
root@foo:~# curl --head http://foo.sjd.se/ HTTP/1.1 200 OK …
The next step is to create a keytab containing the Kerberos V5 secrets for your host, the exact steps depends on your environment (usually kadmin ktadd or ipa-getkeytab), but use the string “HTTP/foo.sjd.se” and then confirm using something like the following.
The file should be owned by root and not be in the default /etc/krb5.keytab location, so Apache’s libapache2-mod-auth-gssapi will have to use gssproxy to use it.
Then configure gssproxy to find the credential and use it with Apache.
You should see something like this in the log file:
Sep 18 07:03:15 foo gssproxy[4076]: [2022/09/18 05:03:15]: Exiting after receiving a signal Sep 18 07:03:15 foo systemd[1]: Stopping GSSAPI Proxy Daemon… Sep 18 07:03:15 foo systemd[1]: gssproxy.service: Deactivated successfully. Sep 18 07:03:15 foo systemd[1]: Stopped GSSAPI Proxy Daemon. Sep 18 07:03:15 foo gssproxy[4092]: [2022/09/18 05:03:15]: Debug Enabled (level: 1) Sep 18 07:03:15 foo systemd[1]: Starting GSSAPI Proxy Daemon… Sep 18 07:03:15 foo gssproxy[4093]: [2022/09/18 05:03:15]: Kernel doesn't support GSS-Proxy (can't open /proc/net/rpc/use-gss-proxy: 2 (No such file or directory)) Sep 18 07:03:15 foo gssproxy[4093]: [2022/09/18 05:03:15]: Problem with kernel communication! NFS server will not work Sep 18 07:03:15 foo systemd[1]: Started GSSAPI Proxy Daemon. Sep 18 07:03:15 foo gssproxy[4093]: [2022/09/18 05:03:15]: Initialization complete.
The NFS-related errors is due to a default gssproxy configuration file, it is harmless and if you don’t use NFS with GSS-API you can silence it like this:
To instruct Apache — or actually, the MIT Kerberos V5 GSS-API library used by mod_auth_gssap loaded by Apache — to use gssproxy instead of using /etc/krb5.keytab as usual, Apache needs to be started in an environment that has GSS_USE_PROXY=1 set. The background is covered by the gssproxy-mech(8) man page and explained by the gssproxy README.
When systemd is used the following can be used to set the environment variable, note the final command to reload systemd.
Enable the configuration and restart Apache — the suggested use of reload is not sufficient, because then it won’t be restarted with the newly introduced GSS_USE_PROXY variable. This just applies to the first time, after the first restart you may use reload again.
root@foo:~# a2enconf private Enabling conf private. To activate the new configuration, you need to run: systemctl reload apache2 root@foo:~# systemctl restart apache2
When you have debug messages enabled, the log may look like this:
To verify that the server is working properly you may acquire tickets locally and then use curl to retrieve the GSS-API protected resource. The "--negotiate" enables SPNEGO and "--user :" asks curl to use username from the environment.
root@foo:~# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: jas@GSSPROXY.EXAMPLE.ORG
Valid starting Expires Service principal
09/18/22 07:40:37 09/19/22 07:40:37 krbtgt/GSSPROXY.EXAMPLE.ORG@GSSPROXY.EXAMPLE.ORG
root@foo:~# curl --negotiate --user : http://foo.sjd.se/private
OK
root@foo:~#
