ecksun.com

CPU performance: Why do my cores perform differently?

Sat, 04 Jan 2025 00:00:00 +0100

This post follows up on CPU Performance: Why is that one core so heavily utilized? where we explored why one core was much more heavily utilized than others, even with an evenly distributed workload.

In this post we will look at the other strange behaviour I saw during that load testing work.

Background

If you have already read the the previous post you can skip the background section as it contains the same information.

Last year, I was conducting load testing on a system at work and noticed a couple of intriguing behaviours.

I was working with a product that streams MPEG-TS video between processes. This is commonly achieved by using UDP over localhost as its what most programs support. But even over localhost there are cases where packets might arrive out of order, specifically when the sending processes migrates between CPU cores.

The simplest way of mitigating this issue is to pin the various threads to specific CPU cores to ensure each thread only ever runs on one core.

During load testing I pinned 200 processes to 20 cores, with 10 threads per core. Despite each CPU performing the same amount of work the load distribution was uneven:

When I first saw this, two questions immediately came to mind:

What is going on with cpu8?
What is causing cpu12–cpu19 to have a higher load than cpu0–cpu11?

This post will only address the last question, check out the previous post for a deep dive into the first.

After solving the first issue the CPU load looked like this:

What is causing cpu12-cpu19 to have a higher load than cpu0-cpu11?

In three words; Heterogeneous CPU topology.

Many consumer Intel CPUs use something they call P-cores and E-cores, for Performance cores and Efficiency cores. Arm’s implementation is know as big.LITTLE.

The system I tested used an Intel Core i5-13500E, which has 6 P-cores and 8 E-cores. The P-cores use Intel’s Hyper-threading, so each appears as two logical processors to the operating system.

Thus the first 12 cores in the screenshot are the P-cores and the last 8 are the E-cores. Evidently the E-cores are less performant than the P-cores and thus the same work results in higher utilization.

Why is that a problem?

For most consumer devices — like desktops, laptops and phones — this isn’t a problem. The system uses the power-efficient E-cores for lighter tasks and the high-performance P-cores for heavier work, providing both efficiency and speed.

However in my case I’d like to utilize the computer as much as possible to get the most value out of it. Thus if I naively assign tasks to cores as in the screenshot above I can only put as many tasks on the computer as the least performant cores allows, leaving resources on the table on the more performant cores.

The solution

There are many different CPU topologies, designs and architectures. There is no simple way of taking it all into account. Especially since it also heavily depends on the workload.

There is a program called lstopo(1) that I really like. It visualizes your system’s CPU topology, showing how cores and threads are distributed across the system. It’s a simple but powerful tool and I highly recommend you try it out yourself.

lstopo(1) has this excellent page called “The Best of lstopo” which very well illustrates how many different CPU topologies and architectures there exists out there.

My solution was to simply use the maximum CPU clock frequency for each CPU. I read the maximum frequency from /sys/devices/system/cpu/cpu[0-9]*/cpufreq/cpuinfo_max_freq and used the ratio between the core frequencies to balance task distribution.

Result

The result is a more evenly distributed load across the CPU cores. The system is still running 200 processes, but they are now better balanced across the cores:

As a result, on a fully loaded system, CPU utilization can be increased by about 20% (from 240 to 288 processes).

CPU performance: Why is that one core so heavily utilized?

Thu, 02 Jan 2025 00:00:00 +0100

Background

Last year, I was conducting load testing on a system at work and noticed a couple of intriguing behaviours.

I was working with a product that streams MPEG-TS video between processes. This is commonly achieved by using UDP over localhost as it’s what most programs support. But even over localhost there are cases where packets might arrive out of order, specifically when the sending processes migrates between CPU cores.

The simplest way of mitigating this issue is to pin the various threads to specific CPU cores to ensure each thread only ever runs on one core.

During load testing I pinned 200 processes to 20 cores, with 10 threads per core. Despite each CPU performing the same amount of work the load distribution was uneven:

When I first saw this, two questions immediately came to mind:

What is going on with cpu8?
What is causing cpu12–cpu19 to have a higher load than cpu0–cpu11?

This post will only address the first question, the second is coming in a later post.

What is going on with cpu8?

Trying to understand what was happening, I turned to mpstat(1) which showed that cpu8 was doing much more %soft than the other CPU cores:

%soft refers to software interrupts and is a mechanism used by the kernel to handle I/O such as networking more efficiently.

/proc/interrupts showed about the same thing as mpstat(1) (I have not included a screenshot as it is a very wide table). However it also indicated that all cores were processing interrupts for the network interface, however cpu8 handled many more than the rest.

This led me to investigate the algorithm the network driver uses to distribute packets across CPU cores. ethtool(8) is a tool to query and control network driver and hardware settings. Specifically it can both show and change that algorithm if the network driver supports it. We are looking for “network flow classification”, specifically rx-flow-hash:

$ sudo ethtool --show-nfc eno3 rx-flow-hash udp4
UDP over IPV4 flows use these fields for computing Hash flow key:
IP SA
IP DA

The ethtool(8) manpage shows what the different letters mean:

Letter	Description
s	Hash on the IP source address of the rx packet.
d	Hash on the IP destination address of the rx packet.

As we can see from the output above only the source and destination IP were used for hashing.

Interestingly, this behaviour differed from what I observed on other systems. I’m not sure if it related to the network driver (ixgbe), the operating system (Ubuntu 22.04) or some other aspect of the system. The algorithm also differs from tcp4:

$ sudo ethtool --show-nfc eno3 rx-flow-hash tcp4
TCP over IPV4 flows use these fields for computing Hash flow key:
IP SA
IP DA
L4 bytes 0 & 1 [TCP/UDP src port]
L4 bytes 2 & 3 [TCP/UDP dst port]

Since the load test only used one load-generator and one device under test, both with a single IP configured, both the source and destination IP were the same. This caused all packets for the load test to be processed on the same CPU core, that is cpu8.

Solution

Once we understood the problem, the solution was simple. We only need to configure the network driver to also use the source and destination port for hashing.

Because hashing occurs on the raw IP packet before the protocol (TCP, UDP etc) is parsed it is not possible to directly tell it to use the source and destination port.

Lets look at the UDP packet header:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Source Port          |        Destination Port       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|             Length            |            Checksum           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

And compare that with the table from the ethtool(8) manpage:

Letter	Description
s	Hash on the IP source address of the rx packet.
d	Hash on the IP destination address of the rx packet.
f	Hash on bytes 0 and 1 of the Layer 4 header of the rx packet.
n	Hash on bytes 2 and 3 of the Layer 4 header of the rx packet.

We need to hash bytes 0-4 of the UDP header, that is we want the fields fn in addition to sd, thus, to ensure the packets is evenly distributed across cores, we run:

sudo ethtool --config-nfc enp15s0 rx-flow-hash udp4 sdfn

Result

The result was a much more balanced load distribution, with a slight increase in load on all other cores:

The difference between the two images is highlighted by fading between them:

Repairing the SHIFT6mq

Fri, 15 Mar 2024 00:00:00 +0100

One morning my phone stopped recieving a charge. I tried a number of different chargers, cables and cleaned the USB-C recepticle without luck.

Shiftphone

One of the reasons I bought a Shiftphone was their approach to repairability and sustainability. In practice this means I can buy all spare parts directly from their webshop and can repair the entire phone with nothing but a screwdriver. The SHIFT6mq even has a user-replacable battery!

Ordering parts

When my phone stopped charging I went to their store and found the Sub-PCB. The product description gave repair instructions and suggested one re-plug the ribbon cable between the SUB-PCB and the main board. I tried it but it did not work for me.

Even though they mention the ribben cable is often at fault I ordered both. I like having spare parts close at hand and I did not want to wait longer than needed in case the SUB-PCB was broken.

Repairing

When the parts arrived it also included a small pamphlet with a link to iFixit repair instructions. Though they detail how to replace the display most of the steps are the same.

I dissasembled my phone and tried to replace the SUB-PCB but it still did not charge.

I then replaced the ribbon cable as well and when I connected the USB-C charger it worked!

To verify what was broken I reconnected the old SUB-PCB and tried again and the phone was still accepting a charge!

Turns out the ribbon cable was broken, exactly as the item description said.

Broken again

Sadly, not being very experienced repairing phones I managed to rip the antenna cable out of the connector, so my phone is still broken :(

Thankfully the antenna cable is only 3€, lets hope it ships quickly.

Read receipts and delivery status notifications

Wed, 02 Nov 2022 00:00:00 +0100

Sometimes I like to get an explicit acknowledgment that an email has been delivered and read. While this is possible with anti-patterns such as tracking-pixels or similar technologies there are other well-established ways of getting such information in a privacy preserving manner. Lets look at how.

Read receipts

Read receipts was first proposed in RFC 2298, updated by RFC 3798 and finally became an official internet standard in RFC 8098.

The header of interest to us is Disposition-Notification-To. By setting the header most clients will prompt the user if they want to send a read receipt back. For example in thunderbird it looks like this:

In mutt the default shortcut to edit headers is E, we can then add the header:

Disposition-Notification-To: John <john@example.com>

Delivery status notifications

The mechanism to request Delivery status notifications, also called DSN, was first proposed in RFC 1891 and later in RFC 3461. It works by adding an extension to SMTP which means it is not part of the email itself like in the case of read receipts. Instead it is configured by the client MTA (your SMTP client).

In my case I use msmtp(1) so I only need to specify the --dsn-notify flag. Per default this is disabled and which means the mail system decides when to send notifications. In my case I would like to know as much as possible so I specify --dsn-notify failure,delay,success.

To configure msmtp(1) correctly from mutt we can set sendmail:

:set sendmail = '/usr/bin/msmtp --read-envelope-from --dsn-notify failure,delay,success'

Grow an ext4 filesystem past 16TiB

Tue, 15 Sep 2020 00:00:00 +0200

One of my filesystems just ran out of space. When I tried to grow it it complained:

$ sudo lvextend --size +400G /dev/frej/frej /dev/mapper/frej --resizefs
  Size of logical volume frej/frej changed from <15.95 TiB (4180236 extents) to <16.34 TiB (4282636 extents).
  Logical volume frej/frej successfully resized.
resize2fs 1.44.5 (15-Dec-2018)
resize2fs: New size too large to be expressed in 32 bits

fsadm: Resize ext4 failed.
  /sbin/fsadm failed: 1

The magic limit turns out to be exactly 16TiB, or $16 * 2⁴⁰B$ .

The block size is $4096B = 2¹²B$
There are $2³²$ blocks.
$2¹²B * 2³² = 2⁴⁴B = 16 * 2⁴⁰B = 16 TiB$

The solution is naturally to use 64 bits instead. 64bit mode is described in ext4(5):

Enables the file system to be larger than 2^32 blocks. This feature is set automatically, as needed, but it can be useful to specify this feature explicitly if the file system might need to be resized larger than 2^32 blocks, even if it was smaller than that threshold when it was originally created. Note that some older kernels and older versions of e2fsprogs will not support file systems with this ext4 feature enabled.

This filesystem was created long ago. Way before I could have volumes larger than 16TiB. Over time I upgraded the hardware and resized the volume. Today I reached 16TiB and all of a sudden 32 bits was no longer enough.

Changing the volume to 64bit mode is quite easy. Simply pass the -b flag to resize2fs(8):

$ sudo resize2fs -b /dev/frej/frej
resize2fs 1.44.5 (15-Dec-2018)
Converting the filesystem to 64-bit.
resize2fs: No space left on device while trying to resize /dev/frej/frej
Please run 'e2fsck -fy /dev/frej/frej' to fix the filesystem
after the aborted resize operation.

It failed.

It took me quite a while to figure out why there was no space left. I first tried to allocate slightly more space on the underlying logical volume. That failed too.

Turns out resize2fs require some free space in the filesystem itself. Since I ran out of space just before growing the filesystem none was available. Together with this filesystem not having any reserved blocks (tune2fs -m 0) meant the conversion did not work. I removed about 5 GB and tried again:

$ sudo resize2fs -b -p /dev/frej/frej
resize2fs 1.44.5 (15-Dec-2018)
Please run 'e2fsck -f /dev/frej/frej' first.

$ sudo e2fsck -f /dev/frej/frej
e2fsck 1.44.5 (15-Dec-2018)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
frej: 165788/535072768 files (34.3% non-contiguous), 4279267286/4280561664 blocks
$ sudo resize2fs -b -p /dev/frej/frej
resize2fs 1.44.5 (15-Dec-2018)
Converting the filesystem to 64-bit.
Begin pass 2 (max = 25488)
Relocating blocks             XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 130633)
Scanning inode table          XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 5 (max = 4)
Moving inode table            XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The filesystem on /dev/frej/frej is now 4280561664 (4k) blocks long.

And now the filesystem has the 64bit feature enabled and we can grow it past 16TiB.

Presseeding a BeagleBone Black

Wed, 17 Jun 2020 00:00:00 +0200

Introduction

I have gotten a hold of quite a few BeagleBone Blacks, a single-board computer much like the more well known Raspberry Pi. The important difference to a Rasperry is that the BeagleBone Black can run a completely open-source system. BeagleBone does provide images based on Debian that you can flash directly, however I prefer to run upstream Debian without modifications.

As an example one of my use-cases for some BeagleBones is as simple WiFi Access Points in order to extend the coverage of my home WiFi network. I might detail that setup in a future post.

This post will show how I go about installing Debian on a BeagleBone Black.

First steps

When I first try to boot the Debian installer from an SD-card it fails, the only output on the serial console is an ever growing number of the character C:

CCCCCCCCCCCCCCCCCCCCCCCCCC

I don’t understand why this is however I think its related to the u-boot version that is flashed on my BeagleBones (which is not the same as what they come shipped with).

Installing BeagleBones official image

In order to solve the boot issue I first install the BeagleBone provided image, which upgrades the u-boot version.

First off, lets download the latest image from BeagleBones website and flash it to an SD-card:

xzcat ~/Downloads/bone-debian-10.3-iot-armhf-2020-04-06-4gb.img.xz > /dev/sdb

In order to get the BeagleBone to boot from the SD-card you need to hold down the button next to the USB-port while you power the BeagleBone, otherwise it will simply boot from the eMMC.

Then I follow these instructions for flashing that image to the eMMC. Essentially this only involves uncommenting the following line in /boot/uEnv.txt:

# cmdline=init=/opt/scripts/tools/eMMC/init-eMMC-flasher-v3.sh

and then reboot, which will start flashing the image to the eMMC.

Installing Debian

After the BeagleBone provided image has been flashed I can boot the ordinary Debian installer from an SD-card. According to the Debian manual the images can be downloaded from a debian mirror and written to an SD-card:

wget http://http.us.debian.org/debian/dists/buster/main/installer-armhf/current/images/netboot/SD-card-images/firmware.BeagleBoneBlack.img.gz
wget http://http.us.debian.org/debian/dists/buster/main/installer-armhf/current/images/netboot/SD-card-images/partition.img.gz

zcat firmware.BeagleBoneBlack.img.gz partition.img.gz > /dev/SD_CARD_DEVICE

Once inserted into the BeagleBone once again hold down the button next to the USB-port while powering it to boot from the SD-card.

Preseeding

We will use preseeding to automate the installation. This makes the system more easily reproducible and allows us to leave the installation mostly unattended, which is especially nice on a slower system such as the BeagleBone. In my case I chose to preseed from a webserver as it requires the least amount of setup beforehand.

First off, we need to stop the automatic boot, I generally just spam the space bar as soon as I have powered on the system. The serial-port will display something similar to:

Press SPACE to abort autoboot in 0 seconds

Then we just need to provide our own boot arguments to the installer to have it fetch the preseed file:

env set bootargs auto url=https://example.com/preseed.cfg
boot

Once the installer has started we can press ESC and select the preseed entry in the menu:

Download debconf preconfiguration file

Before the installer downloads the preseed file it will ask a few questions. The questions are relating to network configuration and will not be preseeded if the preseed file is supplied via the network.

However, once the installer has downloaded the preseed file it will continue with the rest of the installation unattended and leave you with a working system by the end.

Date issues

The BeagleBone does not have a persistent hardware clock so it is not uncommon for the time of the device to be off by years. This is problematic when the installer needs to fetch the preseed file over https as that requires a system time to be somewhat close to reality. When it is too far off, i.e. years, the certificates will be invalid as their notBefore date is in the far future.

This can easily be solved by just setting the time, press ESC and select the shell entry in the menu:

Execute a shell

Once in the shell, set the date and exit:

date --set "2020-06-17 00:00"
exit

Once done retry the “Download debconf preconfiguration file” step.

Pingeling

Fri, 17 Feb 2017 00:00:00 +0100

A common issue when managing a larger number of hosts is knowing whether they are online or not. A couple of weeks ago I got a little time on my hands and felt like solving that problem.

The idea is to ping every possible host every so often, parse the response and finally publish a JSON object to kafka for future processing.

We split our problem into four distinct parts and tackle them one at a time:

Ping all hosts continuously with fping
Parse the output from fping with bash’s regular expressions
Create a JSON message with jq
Send the finished JSON object to kafka with kafkacat

In essence what we are doing is performing simple stream processing. The source of our stream is fping. The data is then transformed with jq into JSON and finally the sent to our sink, kafka.

Ping hosts

There are many easy ways of sending ICMP echo requests to all hosts in a network, such as forking ping a bunch of times or using paralell. However that doesn’t scale particularly well and is difficult to parse. Therefore we use a tool called fping which gives us the ability to ping entire subnets very easily and efficiently.

fping -A -e -l -b12 -p 60 -g 192.168.1.0/24

An explanation of the arguments, these can obviously be found in the manpage.

Argument	Explanation
`-A`	Display IP address instead of DNS name
`-e`	Show round trip time, we want to know how responsive the host is
`-l`	Continue forever
`-b`	Message size, the minimal size is 12 bytes
`-p`	Time between ICMP echo requests
`-g`	The netmask to ping

Handling fping output

Reading the output

We do not care about the statistics fping prints, as they are printed to stderr we can easily ignoring them by piping stderr to /dev/null with 2>/dev/null.

Each ping reply is printed to its own line, we thus use the bash builtin read to read one line, for example:

(echo first line && echo second line) | while read LINE; do
    echo "$LINE";
done

Parsing the output

As bash have native support for regular expressions we use a regex to parse the fping output.

We define an IP address as anything matching 0-9a-fA-F.:. 0-9. matches IPv4 addresses only, while the extra characters a-fA-F: are used to match IPv6 addresses as well

An example of what we want to parse:

192.168.1.1   : [0], 40 bytes, 0.38 ms (0.38 avg, 0% loss)

Field	Regex
IP Address	`([0-9a-fA-F.:]+) +: \[(.)\], (.) bytes, (.) ms $(.) avg, (.*)% loss$`
Sequence number	`([0-9a-fA-F.:]+) +: \[(.)\], (.) bytes, (.) ms $(.) avg, (.*)% loss$`
Message size	`([0-9a-fA-F.:]+) +: \[(.)\], (.) bytes, (.) ms $(.) avg, (.*)% loss$`
Round trip time	`([0-9a-fA-F.:]+) +: \[(.)\], (.) bytes, (.) ms $(.) avg, (.*)% loss$`
Avarage response time	`([0-9a-fA-F.:]+) +: \[(.)\], (.) bytes, (.) ms $(.) avg, (.*)% loss$`
Ratio of lost packages	`([0-9a-fA-F.:]+) +: \[(.)\], (.) bytes, (.) ms $(.) avg, (.*)% loss$`

The final regex ended up looking like this:

([0-9a-fA-F.:]+) +: \[(.*)\], (.*) bytes, (.*) ms \((.*) avg, (.*)% loss\)

Formatting data before sending

jq is a very powerful tool if you wish to either parse or generate JSON.

The final JSON object which we send to kafka should contain four fields, ip, size, roundtrip and time. For readability’s sake I have added newlines and indentations to the example below, however we will skip that in the final object.

{
    "ip": "192.168.1.1",
    "size": "40",
    "roundtrip": 0.38,
    "time": "2015-06-23T22:42:14,033793204+0200"
}

Since everything in bash is strings we need to transform the roundtrip and size to numbers, we can do this with the jq function tonumber. Our final jq filter will thus look like this:

{ "ip": $ip, "size": $size|tonumber, "roundtrip": $roundtrip|tonumber, "time": $now }

We tell jq to only generate with --null-input and the -output flags lets us define a nice format for sending to kafka. Moreover we can provide arguments to jq with the --arg argument, arguments in jq behave much like arguments in bash and will be substituted for their value.

jq \
    --null-input \
    --compact-output --ascii-output --monochrome-output \
    --arg ip "192.168.1.1" \
    --arg size "40" \
    --arg roundtrip "0.38" \
    --arg now "2015-06-23T22:42:14,033793204+0200" \
    '{ "ip": $ip, "size": $size|tonumber, "roundtrip": $roundtrip|tonumber, "time": $now }'

Kafka

Finally we want to send the output to kafka, this is trivial by simply piping the result to kafkacat. kafkacat will interpret each line as a separate message and send it to the topic specified.

The finished script

Putting everything together we get something similar to this:

#!/bin/bash

TARGET_NETWORK="10.0.0.0/22"
WAIT_TIMEOUT=$((5*60*1000))
BROKERS="zk.example.com"

fping -A -e -l -b12 -p "$WAIT_TIMEOUT" -g "$TARGET_NETWORK" 2>/dev/null \
    | while read -r PING_RESPONSE; do
    
    regex="([0-9a-fA-F.:]+) +: \[(.*)\], (.*) bytes, (.*) ms \((.*) avg, (.*)% loss\)"
    [[ $PING_RESPONSE =~ $regex ]]
    IP="${BASH_REMATCH[1]}"
    SIZE="${BASH_REMATCH[3]}"
    ROUNDTRIP="${BASH_REMATCH[4]}"

    NOW=$(date --iso-8601=ns)

    jq \
        --null-input \
        --compact-output --ascii-output --monochrome-output \
        --arg ip "$IP" \
        --arg size "$SIZE" \
        --arg roundtrip "$ROUNDTRIP" \
        --arg now "$NOW" \
        '{ "ip": $ip, "size": $size|tonumber, "roundtrip": $roundtrip|tonumber, "time": $now }'

done | kafkacat -P -b "$BROKERS" -t event.JSON.ICMP_response

Conclusion

I started writing this post a while ago but never published. Since then we have used this script at work in production to keep track of our mobile connected devices (which is the reason for the small packet size). It has been running for a long while and has worked flawlessly and without interruption ever since we first deployed it demonstrating the power of combining simple tools.

SSH Certificate Authorities

Sat, 14 Jan 2017 00:00:00 +0100

A common nuisance when first connecting with SSH to a server is to verify the fingerprint. Especially if you have many servers with multiple users, when everyone needs to know all the fingerprints of all the servers. This can easily be improved with SSH CA host certificates.

For example if we were to sign a server’s public key when we provision it, everyone that already trusts the CA can then also connect to that server without having to manually verify the fingerprint.

User certificates on the other hand can help us authorize users to a server without manually managing each individual key. By deploying a SSH user CA to a remote host anyone with a valid certificate can connect to it.

Manpages

The manpages of ssh-keygen(1) and sshd(8) contains everything you need to known about SSH Certificates, of most interest is the section “CERTIFICATES” of ssh-keygen(1) and sections “AUTHORIZED_KEYS FILE FORMAT” and “SSH_KNOWN_HOSTS FILE FORMAT” of sshd(8). Use them as a reference and think of this post as a way to get started.

Host certificates

Creating

The CA itself is generated like any other SSH key.

ssh-keygen -f host_ca -t ed25519

Trusting

Users can then choose to trust this CA by adding the a line like toe following to their known_hosts file (defined in sshd(8) section “AUTHORIZED_KEYS FILE FORMAT”):

@cert-authority *.example.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIzHKhkOE4C58Zgg/7AO2xXVGKBSAt2iKs9vJgkCu8hh user@host

The second field is to limit the scope of the key, in this case we trust it to verify everything from *.example.com but not for example secret.example.test.

Signing

This is how we sign a server’s public key, that is, generate a certificate for that server:

ssh-keygen -s "$HOST_CA" -I "$IDENTIFIER" -h -n "$SERVER_DOMAIN" "$PUBLIC_KEY"

A quick explanation to the arguments:

Argument	Description
`-s`	The CA key to sign with
`-I`	An identifier for the certificate, used for logging and revocation
`-h`	Create a host certificate
`-n`	Principals, in practice which domain this certificate will be valid for

Its also possible to limit when a certificate is valid with -V.

In order to sign the public key ssh_host_ed25519_key.pub of server server.example.com with the host_ca CA we can run:

ssh-keygen -s "host_ca" -I "user_1@host_ca" -h -n "server.example.com" "ssh_host_ed25519_key.pub"

This will create a file called ssh_host_ed25519_key-cert.pub that we need to configure the server to use. This is done with the HostCertificate directive from sshd_config(5).

HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub

Demoing

Lets put everything together into a working example.

ssh-keygen -f host_ca -t ed25519
echo "@cert-authority *.example.com $(cat host_ca.pub)" >> ~/.ssh/known_hosts
scp server.example.com:/etc/ssh/ssh_host_ed25519_key.pub .
ssh-keygen -s host_ca -I "user_1@host_ca" -h -n server.example.com \
    ssh_host_ed25519_key.pub
scp ssh_host_ed25519_key-cert.pub server.example.com:
ssh -t server.example.com 'echo "HostCertificate /etc/ssh/ssh_host_ed25519_key-cert.pub" | sudo tee --append /etc/ssh/sshd_config'
ssh -t server.example.com sudo systemctl reload ssh

Lets test it out

echo "@cert-authority *.example.com $(cat host_ca.pub)" \
    | sudo -u newuser -i bash -c "cat > ~/.ssh/known_hosts"
sudo -u newuser ssh server.example.com

Notice how it does not ask you to verify the fingerprint, as we already trusted the CA.

User certificates

By installing a CA’s public key into a remote users authorized_keys file (or globally on the server) everyone who has their public key signed by the CA (i.e. a certificate) will have access to the server.

Creating

We create user CA’s in the exact same manner as host CA’s, or ordinary ssh keys for that matter.

ssh-keygen -f "user_ca" -t "ed25519"

Trusting

By adding the user CA’s public key to a remote user’s authorized_keys file we will grant access to anyone who has a valid certificate from the CA. For this to work we also need to give the key the cert-authority option. The format is specified by section “AUTHORIZED_KEYS FILE FORMAT” of sshd(8).

For example:

cert-authority ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK624pJT4N/5RrU9AE4I5U1fZCVGwlyqM4nylreB15oB user@host

It is also possible to globally install a CA key by using the TrustedUserCAKeys of sshd_config(5). In that case it is however very important to limit the scope of the certificates as a certificate will otherwise give access to any account on the server.

Signing

A user certificate is generated in much the same way as a host certificate, the big difference is that the -h argument is missing.

ssh-keygen -s "$USER_CA" -I "$IDENTIFIER" "$PUBLIC_KEY"

The other difference is the principals defined with -n. With user certificates they define which users the certificate is valid for, instead of which hosts. If you have installed the CA globally (with TrustedUserCAKeys in sshd_config) it is strongly recommended you use this, as the certificate otherwise will grant access to any account.

ssh-keygen -s "$USER_CA" -I "$IDENTIFIER" -n "$USER" "$PUBLIC_KEY"

Demoing

Lets try out a working example of user certificates:

ssh-keygen -f user_ca -t ed25519
echo "cert-authority $(cat user_ca.pub)" \
    | ssh server.example.com "cat >> .ssh/authorized_keys"

And test if it works:

# We use sudo to be able to write to newuser's homedir in order to simplify
# this demo
sudo ssh-keygen -s user_ca -I "newuser@user_ca" ~newuser/.ssh/id_ed25519.pub
sudo -u newuser -i ssh user@server.example.com

Notice that no password is required, in fact, if you followed this post in order you should have been logged in without neither verifying a fingerprint nor providing a password.

Revocation

Revoking keys is also possible, however because of the length of this post and the fact that I have yet to configure any key revocation lists I have chosen not to discuss that in this post. You can however read about it in the section “KEY REVOCATION LISTS” of ssh-keygen(1).

One container per connection

Sat, 13 Aug 2016 00:00:00 +0200

I’m thinking about setting up a wargame like scenario for work and thought it would be nice if I could have the users ssh to some machines for different steps of the game. However since the users aren’t really trusted I would prefer to only give them access to a container. Moreover I don’t want the different teams to be able to interact with each other on the machine itself, so I needed to spawn one container for every team.

This is when I realized that socket-activation is exactly what I want, I just need to bundle it with docker and all should be well.

Running sshd from docker

First, we need a Dockerfile

FROM debian:stretch

RUN (\
        export DEBIAN_FRONTEND=noninteractive && \
        apt-get update  && \
        apt-get install -y --no-install-recommends \
            openssh-server \
        && \
        apt-get autoremove && \
        apt-get clean \
    )
RUN (\
    useradd --create-home user && \
    echo 'user:woop' | chpasswd && \
    mkdir /var/run/sshd \
)

CMD /usr/sbin/sshd -i

Everything in this Dockerfile is to get sshd to run. However the -i flag is special. It tells sshd that it is run from inetd(8). This is what we need to get socket-activation to work.

Systemd units

Now we need to define our systemd units. First we define the unit (sshd@.service) to start the container and sshd in it. This was heavly inspired by the sshd service file in Arch.

[Unit]
Description=sshd container

[Service]
ExecStart=-/usr/bin/docker run --rm -i sshd
StandardInput=socket
StandardError=syslog

To be honest I’m not sure if the -i flag is required, however it helped with debugging because it made the container behave nicely with ^C.

StandardInput=socket is what will make systemd run this service like inetd(8) would have.

We also need to setup the sshd.socket unit file.

[Unit]
Description=SSH socket

[Socket]
ListenStream=0.0.0.0:22
Accept=yes

Testing

With all of this up and running, we should be able to connect to the ssh server.

ssh \
	-o UserKnownHostsFile=~/tmp-known_hosts \
	-o ControlMaster=no \
	-o ControlPath=none \
	user@localhost

I use a custom identityfile as every time you rebuild your container it will generate a new private key and I don’t want to clutter my ordinary known_hosts file. Moreover I had to disable ControlMaster as otherwise ssh would simply use that to connect to the same container I had already connected to.

Now for the actual test, for brevity I have omitted the motd.

$ ssh -o UserKnownHostsFile=~/tmp-known_hosts -o ControlMaster=no -o ControlPath=none user@localhost
user@localhost's password: 
$ ls
$ touch asdf

and then, without closing that terminal:

$ ssh -o UserKnownHostsFile=~/tmp-known_hosts -o ControlMaster=no -o ControlPath=none user@localhost
user@localhost's password: 
$ ls
$

Success!

Resources

Some resources I used to arrive at this solution:

Upgrading storage

Wed, 25 Mar 2015 00:00:00 +0100

A while ago I realized one of my RAID arrays was running out of space. Since I didn’t have the space required to take a backup of everything I needed to perform the upgrade in place. In this post I replace all drives in my RAID5 array, resize the mdadm array, resize both the LVM physical volume and logical volume, the LUKS container and lastly the file system.

Upgrading the hardware

I have no more space left in my case nor do I have any SATA connections left on my motherboard, thus I bought a SATA to USB adapter which I used for replacing drives.

In this section we replace one drive at a time, we assume the old drive is called /dev/sda and the new drive is called /dev/sdb. The mdadm volume we operate on is /dev/md126.

Partitioning the new drives

I partitioned the drives with the same sized partitions as the old drives, I don’t think thats required (anything bigger works), I just wanted to postpone taking a decision on the partition-size.

# Check the size of the partition of the previous drive
$ sudo parted /dev/sda unit s print
Model: ATA ST4000DM000-1F21 (scsi)
Disk /dev/sda: 7814037168s
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start  End          Size         File system  Name     Flags
 1      2048s  2930277160s  2930275113s               primary

# Partition the new drive in the same manner.
$ sudo parted /dev/sdb
(parted) mktable gpt
(parted) mkpart primary 2048s 2930277160s

# Check that it looks alright
$ sudo parted /dev/sdb unit s print
Model: ATA WDC WD40EFRX-68W (scsi)
Disk /dev/sdb: 7814037168s
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start  End          Size         File system  Name     Flags
 1      2048s  2930277160s  2930275113s               primary

Replacing the drives

Its easy to replace drives with mdadm, especially since version 3.3.

$ sudo mdadm --manage /dev/md126 --add-spare /dev/sdb1
mdadm: added /dev/sdb1
$ sudo mdadm /dev/md126 --replace /dev/sda1
mdadm: Marked /dev/sda1 (device 4 in /dev/md126) for replacement

--replace will replace the drive as soon as a replacement is available. When the drive is replaced the old drive will be marked as faulty. Using --replace compared to simply swapping the drives has the advantage of never putting the array in a degraded state, as the old drive will continue to be used until it is replaced.

Once the drive is replaced an email is generated with a Fail event, we then need to remove the old drive from the array:

$ sudo mdadm --manage /dev/md126 --remove failed
mdadm: hot removed 8:1 from /dev/md126

Increasing available space

In this section the mdadm device is still called /dev/md126 and both the LVM logical volume and physical volume is called frej.

Partition resizing

NOTE: I suggest you simply create the partitions as big as you want them from the beginning, it makes the upgrade simpler.

Stop

I do not know why it would be beneficial to not use the partitions when resizing them. However since there seem to be an equal divide between people saying you should and people saying it doesn’t matter my goal was to take the safe path and not use the partitions while resizing. I did however forget to stop the RAID when resizing 4 out of my 5 partitions and never noticed any issues.

$ sudo umount /mnt/frej
$ sudo lvchange -an /dev/frej/frej
$ sudo vgchange -an frej
  0 logical volume(s) in volume group "frej" now active
$ sudo cryptsetup close /dev/mapper/frej
$ sudo mdadm --stop /dev/md126
mdadm: stopped /dev/md126

Resize

When resizing with parted you provide the end of the partition, not the size of it. I started specifying this as 4TB but then, as the start of my partition is 1048576B, my partition only became 3999998951936B big. Not having 4TB would have annoyed me so I resized it to the first multiple of 1 MiB above 4TB. Moreover we want the end of the sector, not the beginning of the next one, so we subtract one:

⌈(4*10¹² + 1024²)/1024²⌉*1024² - 1 = 4000001818623.

$ sudo parted /dev/sdb resizepart 1 4000001818623B

$ sudo parted /dev/sdb unit b print
Model: ATA WDC WD40EFRX-68W (scsi)
Disk /dev/sdb: 4000787030016B
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:

Number  Start     End             Size            File system  Name     Flags
 1      1048576B  4000001818623B  4000000770048B               primary

Restart

After resizing all partitions lets see if it worked:

$ sudo mdadm --assemble --scan
mdadm: /dev/md/frej has been started with 5 drives.
$ sudo vgscan
  Reading all physical volumes.  This may take a while...
  Found volume group "frej" using metadata type lvm2
$ sudo vgchange -ay frej
  1 logical volume(s) in volume group "frej" now active
$ sudo mount -a

It seemed I didn’t have to reopen the LUKS container, I’m guessing it is related to me having the device in crypttab and something causes the device to be reopened.

Growing the RAID

$ sudo mdadm --grow /dev/md126 --size max
mdadm: component size of /dev/md126 has been set to 3906249728K
unfreeze

This will take a while, since it needs to resync the unused space.

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [raid1]
md126 : active raid5 sdb1[5] sdf1[6] sde1[7] sdd1[9] sdc1[8]
      15624998912 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]
      [=======>.............]  resync = 37.5% (1468675572/3906249728) finish=767.9min speed=52901K/sec

For me it took about 13 hours to complete the resync, thus mdadm’s estimate was quite accurate.

Resizing the LUKS container and the LVM physical volume

$ sudo cryptsetup resize /dev/mapper/frej
$ sudo pvresize /dev/mapper/frej
  Physical volume "/dev/mapper/frej" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

Filling the empty space with random data

Since we have newly allocated space we should fill it with random data to make sure no information is leaked through our LUKS container. I have chosen to do this by simply creating a new logical volume with the remaining free space, create an encrypted device over it and fill that devices with zeros. Since the zeros are encrypted with a random key the final data should be random.

$ sudo lvcreate --extents 100%FREE --name filltemp frej
$ sudo cryptsetup --key-file=/dev/urandom create filltempcrypt /dev/frej/filltemp
$ sudo dd if=/dev/zero of=/dev/mapper/filltempcrypt bs=1M # This takes a _long_ time
$ sudo cryptsetup close /dev/mapper/filltempcrypt
$ sudo lvchange -an /dev/frej/filltemp
$ sudo lvremove /dev/frej/filltemp
  Logical volume "filltemp" successfully removed

You can check the progress of dd by sending it the SIGUSR1 signal:

$ sudo pkill -f '^dd if=/dev/zero'

The output from dd will look something like this:

220340+0 records in
220340+0 records out
112814080 bytes (113 MB) copied, 0.521177 s, 116 MB/s

NOTE: I ran into the same performance issue I did years ago when setting up the array, since I had reinitialized the array after boot my fixes did not get applied (they are run from rc.local).

Resize the LVM logical volume and file system

Now we need to extend the size of our logical volume and file system. I choose to add 3 TiB as thats what I needed to migrate the data I had on a different volume.

$ sudo lvextend --size +3T /dev/frej/frej
  Size of logical volume frej/frej changed from 5.46 TiB (1430796 extents) to 8.46 TiB (2217228 extents).
  Logical volume frej successfully resized

$ sudo resize2fs /dev/mapper/frej-frej
resize2fs 1.42.12 (29-Aug-2014)
Filesystem at /dev/mapper/frej-frej is mounted on /mnt/frej; on-line resizing required
old_desc_blocks = 350, new_desc_blocks = 542
The filesystem on /dev/mapper/frej-frej is now 2270441472 (4k) blocks long.

Conclusion

Using modern technologies such as mdadm, LVM and LUKS it is really easy to increase the storage capabilities of a server. Most of the steps can also be performed online. My chassi does not have the ability to easily replace drives, which means I risk damaging them if I try to replace a drive physically while the system is still online.

Had I only had a better chassi I could have performed this entire procedure online, completely without downtime.

Encrypting bitlbee traffic with stunnel

Wed, 09 Apr 2014 00:00:00 +0200

While going through all my externally facing services I noticed bitlbee is not actually encrypted. This was quite a surprise for me, mainly for the fact that I did not notice it while setting bitlbee up the first time. On the other hand not much was compromised as the services I use via bitlbee are all run by untrustworthy corporations. All communication through them is as good as unencrypted anyway :). Likewise since I have setup my firewall to only accept connections from one other host, which I also control, gaining control over bitlbee itself would have been difficult for an attacker.

Regardless, I needed to reset my passwords for all accounts used and obviously make sure my communication with bitlbee was encrypted. Since bitlbee have been working so well for me I opted to use stunnel to encrypt the traffic. I was also curious to see if I could get client-side certificates to work, which is a feature stunnel provides.

I have omitted the output from most commands on this page in order reduce clutter. Only a few commands are interactive, such as openssl req.

stunnel

I based my stunnel config of the sample provided in debian in /usr/share/doc/stunnel4/examples/stunnel.conf-sample. I have gathered most information from the sample, manpages and http://www.stunnel.org/static/stunnel.html.

This is my the entire contents of my config file (/etc/stunnel/stunnel.conf) excluding the service definitions:

chroot = /var/lib/stunnel4/
setuid = stunnel4
setgid = stunnel4
pid = /stunnel4.pid
cert = /etc/stunnel/stunnel.pem
verify = 3
CApath = /certs
options = NO_SSLv3
options = SINGLE_ECDH_USE
options = SINGLE_DH_USE

chroot

It is interesting to note that since we are running a chroot it is not possible to reload the config file. When stunnel first starts it reads the proper one, however if you try to reload it (sudo /etc/init.d/stunnel reload) it will not be able to find the file, as it is searching in the chroot.

The chroot itself needs to contain certain files, I simply assumed that most of the files listed in the documentation were required.

user@server $ ls /var/lib/stunnel4/
certs  nsswitch.conf  resolv.conf  timezone

Certificates

We need to generate the key and certificate on the server. Remember to specify the domain name of the server correctly when creating the certificate request (openssl req) is the same that we later use to connect to via irssi. It must match the host you use when connecting from irssi, if they do not match the certificate will be invalid and irssi will refuse to connect.

user@server $ openssl genrsa 4096 > stunnel.key
user@server $ openssl req -new -key stunnel.key -x509 -days 1095 -out stunnel.crt
user@server $ cat stunnel.{key,crt} > stunnel.pem
user@server $ openssl gendh 2048 >> stunnel.pem
user@server $ sudo mv stunnel.pem /etc/stunnel/stunnel.pem
user@server $ # Let stunnel4 own it
user@server $ sudo chown stunnel4:stunnel4 /etc/stunnel/stunnel.pem
user@server $ # Make sure no-one else can read it
user@server $ sudo chmod 600 /etc/stunnel/stunnel.pem

And on the client we need a key and certificate as well:

user@client $ openssl genrsa 4096 > client.key
user@client $ openssl req -new -key client.key -x509 -days 1095 -out client.crt
user@client $ cat client.{key,crt} > client.pem
user@client $ scp client.crt server:                # Upload the client certificate
user@client $ scp server:stunnel.crt .              # Download the server certificate

Now we need to put the client certificate into the certs folder on the server

user@server $ sudo mv client.crt /var/lib/stunnel4/certs
user@server $ # Generate symlinks with checksums for the certificates, this
user@server $ # is the files that are actually read by stunnel
user@server $ sudo c_rehash /var/lib/stunnel4/certs

Bitlbee

Setting up bitlbee is simple, we only have to listen to the port specified in the stunnel config file. I also recommend listening only to localhost: DaemonInterface = 127.0.0.1

NOTE: On debian the port from bitlbee.conf is ignored, run sudo dpkg-reconfigure bitlbee-common instead

irssi

The last step is to setup irssi, its a simple one-liner:

/server add -ssl -ssl_cert client.pem -ssl_cafile stunnel.crt -ssl_verify <host> <port>

Getting OpenElec working on an Intel NUC

Mon, 18 Nov 2013 00:00:00 +0100

I have long had a Boxee box as my main HTPC. It has never really been able to do everything I wanted and was never able to properly deal with my media library. I thus decided to get a Intel NUC to run OpenElec with XBMC on it, the parts I got were:

Intel NUC D54250WYK - I5-4250U
Intel 525 series 60GB SSD mSATA
Crucial Ballistix Sport 4GB PC12800/1600MHz CL9

Fan noise

The first thing I noticed after setting it up was that it was quite noisy. The cooling section in the UEFI BIOS told me the fan was running at slightly over 3000 rpm when idle, which seemed slightly high. My solution was to simply change the Minimum Duty Cycle (%) from 40 to 0. This reduced the fan speed drastically. At times (mostly right after boot) the fan was completely turned of, however most often when the system is idling the fan speed is between 1500 and 2300 rpm.

Shutdown issue

I’m also having problems with turning off the device. Turning the device off via the UI or poweroff simply reboots it (it goes through BIOS, so it is not simply reloading XBMC as others have had issues with). Running halt makes the UI hang but does not power off the device.

However, if I have a keyboard plugged in, it works as expected, the device shuts off and stays off. The same is however not true for USB-keys (the only other USB-device I have tried), as that also makes the device reboot instead of shutting down.

Solution

I found a discussion around the issue on intels forums where a Intel employee posted a beta version of the BIOS (based on version 4024) that seems to resolve the issue.

Expanding a live ext4 filesystem on LVM

Sun, 04 Aug 2013 00:00:00 +0200

When installing my machine I used an encrypted LVM with ext4 as the filesystem for both /home and /. I allocated 50 GB for /home and 30 GB for / and left the rest unused, in order to be able to use it where needed, mostly to try out and see if it is a nice setup or not.

Today I was messing around with virtual machines and noticed I only had 15 GB free space on my home partition and needed more. So I did some searching in order to find out how to increase the size of /home and was surprised at how easy it was:

$ sudo lvextend -L+50G /dev/System/home
Extending logical volume home to 102 GiB
Logical volume home successfully resized
$ sudo resize2fs /dev/mapper/System-home
resize2fs 1.42.8 (20-Jun-2013)
Filesystem at /dev/mapper/home is mounted on /home; on-line resizing required
old desc_blocks = 180, new_desc_blocks = 261

This can even be done online, that is without unmounting the filesystem, if you have a kernel newer than 2.6.

Updating Loopia domains with DD-WRT

Sun, 20 May 2012 00:00:00 +0200

I recently switched from running tomato on my router to running DD-WRT. One of the issues I noticed was that DD-WRT don’t have a way to use loopia.se DynDNS API, as that requires the IP to be part of the URI.

Searching the problem I found some ways to do it, however none did it properly. For example some solutions will get the IP at boot-time while others doesn’t take into account the IP the domain actually resolves to.

So I wrote a small script to update my domain if my current external IP differs from the IP my domain currently resolves to, the sources can be found at github .

The biggest issue I had with getting my DNS-updater to work was to actually store the script on the device, which required me to re-flash my router with a different version of DD-WRT with JFFS support. However once that was done and the cronjob (check the github repo for info) was configured everything worked like a charm.

PR 1.2

Thu, 01 Sep 2011 00:00:00 +0200

When the OTA arrived and I tried to install the update it told me to use Nokia Software Updater to flash the device, which ofc didn’t work. It complained about not being able to download the update and got stuck at 33.6 MB. According to the folks at TMO it was Nokias fault :)

I also read about people being able to install the update after removing some packages, so I did. When I had removed about 20 applications it actually began to download and install, however when everything was completed and the device rebooted I got a rather peculiar error:

I then decided it was a bit to much work to debug so I used the Maemo flasher and began again.

TI - Evalbot

Mon, 31 Jan 2011 00:00:00 +0100

Three month ago I ordered a TI Stellaris Development board (aka evalbot) for 25$, 100$ off the original price. I had no idea if it actually was going to arrive, but earlier today it did.

TI (or fedex?) apparently don’t know how to handle non-ascii characters, so fedex called me this morning to ask for my real address.

Just because I have no idea what to do with it, I decided to do an unboxing, images follow.

The box the evalbot arrived in

There were a whole bunch of packaging material.

There were two boxes inside the evalbot-box, inside the packaging box, alas there were two boxes in the box in the box.

A box in the box

With two boxes

The hardware contents

The software contents

Content of the software box:

A cord
Four AA batteries
A bunch of screws and other random stuff

The actual evalbot! Most stuff is already mounted.

A whole bunch of connections

A better picture of the evalbot

The evalbot main chip (I guess) and the display

A closeup of the main chip.

The backside of the evalbot, not much interesting

So, thats it, now I only need to figure out how to use it :)

TVRage in bash

Tue, 11 Jan 2011 00:00:00 +0100

So, I read two nice pages about bash some weeks ago and wanted to see if you could build something proper.

I often check tvrage.com to see the latest and next episode of a show, but the search is horribly stupid and the page requires interaction before I can get the information I want.

So, yesterday I wrote a bash script that fetches some information from tvrage, I have published it on https://github.com/ecksun/tvrage.

It has three ways of finding information. First it uses a internal associative array to see if the search is present there, if it is (and searching is not specifically specified) it will get information about that show. If there isn’t an exact match in the preconfigured array it will search through it with a regex. If it cannot find anything then either (or hasn’t find the specified number of hits) it searches on tvrage.

I don’t know if anyone else has any use for this, but I like getting information from the CLI, so hopefully it will easy my everyday life :D

Have a good day!

Acer TimelineX 3820T

Thu, 02 Sep 2010 00:00:00 +0200

Wired network card

There exists a rather easy fix for getting the wired ethernet card to work. Download the drivers from atheros homepage (the makers of the ethernet chipset), the drivers can be found here (dead link), search for the file AR81Family-Linux-v1.0.1.9.tar.gz.

Extract the file, you probably want to this in a new empty directory as the archive otherwise will clutter the current directory with a buch of files.
```
 tar -zxvf AR81Family-Linux-v1.0.1.9.tar.gz
```
and read the readme :)
Untar/unzip archive:
```
 tar zxf arl1e-x.x.x.x.tar.gz
```
Change to the driver src directory:
```
 cd arl1e-x.x.x.x/src/
```

Compile the driver module:

 make install

The binary will be installed as:

 /lib/modules//kernel/drivers/net/arl1e.[k]o

The install locations listed above are the default locations. They might not be correct for certain Linux distributions. For more information, see the ldistrib.txt file included in the driver tar.

To use the module execute:

sudo insmod /lib/modules/`uname -r`/kernel/drivers/net/atl1e/atl1e.ko

And that should be all.

Note: This must be done every time the kernel is recompiled

Disk encryption

Mon, 23 Aug 2010 00:00:00 +0200

This is just a quick reminder to mostly myself on what to do when installing a new disk. Note that all of the following commands need to be run as root/with sudo.

Begin by writing random data to the disk, this is very important, if it is skipped the security might be severely compromised. I use badblocks for this as /dev/urandom (and definitely /dev/random) is to slow to be practical (takes weeks, or years in the case of /dev/random).

badblocks -c 10240 -s -w -t random -v $DEVICE

Where -c sets the block size, -s shows the progress, -w sets badblocks to write mode, -t specifies the test pattern, in this case random and -v is verbose, naturally.

This usually takes a couple of hours.

NOTE

Its not a good idea to use badblocks, instead use dd if=/dev/urandom of=$DEVICE. I was planning to do this in the future, and will give details on it once I do it. One cool thing is that its possible to run one instance of dd on each core you got, in your entire household. Its not that hard to use nc and help with the random generation over the network.

When done, we need to create a partition with fdisk

fdisk

n - create new partition
w - write the partition table to disk and quit

I usually create just one partition that covers the whole disk, because I have no need for several partitions as most of my drives is used for storage.

Now we create the luks partition:

cryptsetup --verbose --verify-passphrase --key-size 128 luksFormat $DEVICE

i use 128 bits as it has been shown that it is more resistant to a brute-force attack then AES 192 and 256 bits. cryptsetup will ask for a pass phrase, I usually have this as a backup in case I loose my encryption key, like in a disk crash.

Lets add the encryption key.

cryptsetup luksAddKey $DEVICE $NEW_KEY_FILE

Now we open the luks device in order to format it.

cryptsetup luksOpen $DEVICE $DEVICE_NAME --key-file $KEY_FILE

This will create a new entry in /dev/mapper on which we will write the new file system.

Writing the file system:

mkfs.ext3 -L $DEVICE_LABEL -m 0 /dev/mapper/$DEVICE_NAME

The -m switch specifies how much space will be reserved for the super-user, as my drives is mostly for storage, he doesn’t get any.

Now we just need to make sure the device is added automatically when we boot. We need to edit (add a new line) both /etc/crypttab and /etc/fstab in order to get this to work. The first file opens the luks devices and the second mounts the unencrypted devices in the file system.

/etc/cryptsetup:

$DEVICE_NAME    $DEVICE_PATH    $KEY_FILE   luks

/etc/fstab:

/dev/mapper/$DEVICE_NAME    $MOUNT_POINT ext3   defaults    0   2

A tip is to get the device path from /dev/disk/by-id/, because then it doesn’t matter where on your motherboard/controller card the drive is connected.

That should be all you need to know to get a proper setup with encrypted drives.

GSoC 2010 Midterm update

Sun, 11 Jul 2010 00:00:00 +0200

Thought I would give a status update as it is midterm tomorrow.

Attila notified me the other day that the shepherd source code is up on gitorioius, you can find it here. I have made a clone of the repository with all my work, which can be found here.

As it is just now we get access to the shepherd source the plug-ins haven’t been adapted to work with the shepherd API, which is what I’m working on atm.

I have also run into a bit of trouble as my hard-drive seems to be failing. A while after boot/reboot I stop to be able to access files from the drive, giving me an I/O error at some sector (different depending on the file). Files/binaries which I have previously used and are cached in ram seem to work fine. I’m going to call Acer tomorrow but most likely I’m going to have to buy a new disk myself. As I’m currently in the middle of nowhere (parents summer-house) I’m rather limited in what I can do and as I’m only on a HSDPA connection it will take a couple of hours to download debian, updates, scratchbox, Qt SDK and so on.

Other interesting stuff I have run into since my last update:

QtDBus does per default not define an interface for the objects we try to register (I can at least not find out which interface it used). This was fixed by using:

 Q_CLASSINFO("D-Bus Interface", "name.of.interface");

It seems alarm_action_set_dbus_args only take an array as arguments
If you want to receive a string through DBus with QtDBus it apparently cannot be a std::string, it needs to be a QString

Debian Squeeze HDMI Sound problems

Wed, 07 Jul 2010 00:00:00 +0200

Apparently the sound over HDMI didn’t work out of the box on my debian squeeze installation, I have now gotten it to work and thought I should document my efforts.

My setup is as follows:

Acer 3820T
Intel Core i3 330m
Intel HDA graphics card (on the same die as the CPU)
Debian Squeeze with gnome

I came across a post on forums.debian.net with a possible fix to my problem.

When I listed the audio-devices i got the following results:

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Intel [HDA Intel], device 0: ALC269 Analog [ALC269 Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 1: ALC269 Digital [ALC269 Digital]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 3: INTEL HDMI [INTEL HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0

This indicated that there where no problem with the driver/device itself (which I also knew because it was working flawlessly in windows). Others around the net seemed to have problem with the device not showing up as it should, which was probably related to them having dedicated graphic cards and trying to use the sound card on the mobo or something like that.

I tried to use aplay to play a sound through the device but were unsuccessful, so I continued to search for other solutions. However I later discovered that speaker-test also had the ability to play sound through a certain device, which worked. I assumed there were something wrong with aplay or the way I used it and tried to change the switches I enabled in gnome volume controller.

I saw three switches:

IEC958
IEC958 Default PCM
IEC958 1

With only the switch IEC958 1 activated I heard the test-sound over HDMI, success.

I now continued with boogachamp’s suggestions and added /etc/asound.conf. This is how it looks like:

$ cat /etc/asound.conf 
pcm.!default {
    type plug
    slave.pcm {
        type hw
        card 0
        device 3
    }
}

Which I did, restarted alsa-utils (don’t know if that really was necessary) and the media player I was using and it worked like charm.

GSoC 2010 Progress update

Wed, 09 Jun 2010 00:00:00 +0200

I have just completed another example implementation of an action. This time it was the notification.

This means I have working examples for:

Trigger - WLAN SSID
Trigger - Calendar
Action - Change profile
Action - Secure device
Action - Display a notification
Action - Turn radio on/off and switch between 2G/3G

I have only implemented a way to pull information from the calendar, which is not optimal. I would like to have a Qt slot telling me when changes are made as I have done in the SSID trigger but I have not been able to do so yet.

I am also having some troubles with the dbus commands for turning the radio on/off and switching between 2G/3G, I have found the right dbus command but I can’t translate what I found to c++ code. The DBus commands I intend to use is describe here.

My next task will be to tackle getting the geographical position from GPS or Cell towers, I looked into that when I wrote my submission for GSoC, so hopefully it will work out smoothly.

Extending Shepherd - GSoC Proposal

Sun, 30 May 2010 00:00:00 +0200

Abstract

Shepherd is an advanced scheduler that can do a wide variety of tasks depending on a number of triggers.

The project will aim to improve on the capabilities of Shepherd. I plan to add more ways of triggering an action and more actions to be taken when the triggers is meet.

Full Description

Shepherd currently has the ability to start and stop processes depending upon time, connection type and power requirements. The goal of my project is to extend the capabilities to include a whole range of triggers and also extend Shepherds capabilities to perform different actions.

In a certain situation you might not want to be disturbed, say for example at your workplace. A configuration might then have a trigger on the SSID of the WLAN-stations at your workplace and the actions might be to silence the phone and set the telepathy presence to Busy.

I plan to add a number of plug-ins to shepherd in order to expand its capabilities as well as help improve on the current code in order to make it stable enough for a release. Shepherd is written in C++ but I also hope to enable functionality for python modules.

Tasks and deliverables

Back-end

The back-end is the most important part of Shepherd and is mostly in place. The back-end uses several parts of Qt, among others QtDBus for inter-process communication and QtNetwork for networking. Utilizing Qt not only makes the code simpler but it does also help improve portability.

The API for the plug-ins (triggers/actions) need to be stabilized in order for the development of new plug-ins to begin. The functionality for python modules does also need to be added.

Each situation should have a priority in order to determine what actions are most important. There might be cases where several situations are applicable but define different values for the same actions.

Front-end

At the moment Shepherd has no graphical user interface. The GUI should give end-users the ability to configure Shepherd as well as add, edit and remove situations. The plan is to use Qt in the form of QtGui for the design of the front end.

Triggers

Each trigger is a criteria in a situation. For example the location can be a trigger, if the device is within a certain range of a point then the criteria are met.

Here follows a list of triggers I plan to implement followed by a list of possible triggers for future implementation.

WLAN SSID - Check if a WLAN SSID is nearby
Location - Check if the device is in a certain location
- GPS - By using the GPS
- CellID - Or by using the cell-towers, uses less power than GPS.
Calendar - Check if certain events are happening.

The following triggers might be implemented if time permits.

Telepathy status - Are certain accounts online/busy/offline?
Accelerometer - Check if the device is in a certain position
Proximity sensor - Check if something is nearby, e.g. if the device is in the pocket
Light sensor - Check if the light is above/below a certain limit.
Open/closed HW keyboard
USB Cable connected
- in Mass storage mode
- in PC Suite mode
Headphone connected
Certain bluetooth devices nearby

All triggers may be negated in order to increase the configuration possibilities.

Actions

When a certain situation arise one or more actions are performed. I’m planning to implement the following list of actions:

Change profile
Turn Radio on/off
- for WLAN
- for 3G
Change Telepathy status
Secure device
Display a notification

If I have time I will also try to implement some of the following:

Change volume
Turn vibration on/off
Turn screen on/off
Turn Radio on/off
- for the FM transmitter
- for the FM Receiver
- for GSM
Go in/out of offline mode
Send IM/SMS/EMail
Shut phone off
Change background
Change ring-tone

Schedule

April 26 - May 24

Find out what is needed for the different actions and triggers. Take a decision on how the API for plug-ins should look like.

May 24 - June 13

Work on the back-end and make sure the API is stable and usable.

June 14 - June 27

Work on the front-end.

June 28 - July 4

Triggers to be completed:

Location

Actions to be completed:

Display notification

July 5 - July 11

Make sure the application works properly, all documentation is in place and everything is in order.

July 11 - July 16

Midterm evaluations.

July 17 - July 25

Triggers to be completed:

Calendar
WLAN SSID

Actions to be completed:

Change profile

July 26 - August 1

Actions to be completed:

Telepathy status
Turn WLAN/3G on/off

August 1 - August 9

Actions to be completed:

Secure device
Change telepathy status

August 9 - August 16

Make sure the application works properly, all documentation is in place and everything is in order

August 16 - August 20

Final evaluations.

Other commitments

I have no other commitments during the summer and Shepherd will be my highest priority.

Why me

I have long seen the need for an application such as Shepherd and I believe it has a lot of potential. I’m very motivated because of the wide range of use-cases that exists for Shepherd as well as the opportunity for me to learn from and contribute to the community around maemo.

Ever since I heard about Nokia N900 around the release last year and discovered maemo.org I have been looking forward to develop for the maemo platform. I finally feel I have found a device and a community I really feel at home with. I see this project as the perfect opportunity to become a serious developer and begin contribute in order to give back to the community what I feel it has given me.

Last year I participated in GSoC and wrote a command line interface in java to SIP-Communicator, a VoIP and chat client. I have also participated in NCPC, Nordic Collegiate Programming Contest, in both 2008 and 2009 which has helped me improve my problem solving abilities.

I have worked with C++ in school and have written programs ranging from simple command line games to integer factorisation algorithms using GMP. I perform most of my developing in debian using Git in a number of languages, including python.

Community aspects

To be able to change settings automatically is something not only useful to myself, but to a wide range of users. This is evident by the responses to the thread http://talk.maemo.org/showthread.php?t=31524 in talk and also the great respons Locale have recived on the Android platform.

Each plug-in does also serve as a good example on how to utilize the functionality provided within them. This might be useful even for people that are not directly interested in Shepherd but who are searching for a way to perform a particular task provided in one of the plug-ins.

Contact information

Email: linuswa@kth.se IRC: ecksun at freenode

XMPP: linus.wallgren@gmail.com

Skype: ecksun

SIP: linuswa@kth.se

Biography

I’m a student at the Royal institute of technology in Stockholm, Sweden. Currently I’m doing my third year of a five year computer science masters program. At the moment I’m working on my bachelor thesis which is about studying how Q-learning behaves under different circumstances.

As mentioned earlier I participated in GSoC last year when I worked for SIP-Communicator and extended its functionality in order to control the client from the terminal. I did also mention that I competed in the Nordic Collegiate Programming Contest the last two years.

On my spare time, when I’m not at my computer, I practice a mixed martial art called Goshindo or if the weather and time of year allows I love to wake-board.

Application Suppliment

achipa wrote:

I would very much like to see the proposal extended/thought out with regard to the plugin UI. The original problem with plugin UI/API was that it was difficult to make a consistent GUI for the plugins. The plugins should have a unified GUI, it would be bad practice for each plugin to have a separate UI mechanism (except where technically warranted, like spatial input). It would help avoid situations where contributed plugins get outdated and incompatible because they rely on different versions of Qt, bindings, etc. This would also make it reusable for a desktop version of Shepherd.

My respons:

The first part of the GUI the user will see is a list over all available situations, when you press a situation you can see and edit a list of the actions and triggers that are utilized in that particular situation.

Almost all plug-ins will use the same basic types of input:

Text input
Drop-down selection
Slider for numbers
Do nothing

The first three types will be used by several plug-ins, for example the WLAN SSID trigger might use the text input type and changing profile might use a drop-down selection. For example to shut the phone off does not really require any configuration, which merits the existence of a input type that does nothing. The only plug-in mentioned above that might have use of another type of input is the Location trigger which could use a map for inputting the spatial data.

There will be methods for automatically creating the above input types. This can be done by calling a GUI component of shepherd and asking it to add one of the components. For example the code to configure the WLAN SSID trigger might look something like:

ShepherdGUI::addTextInput("ssid", "Please provide an SSID: ");

If no input type is specified the GUI will assume the plug-in will provide its own GUI and use QUiLoader to load a .ui file in which the plug-in have defined how the configuration interface will look like.

The information entered in the input dialogs can be fetched either by the plug-in asking for it or by a callback, depending on what seems more appropriate when the time comes to decide how the API should look like.

Scratchbox info

Fri, 28 May 2010 00:00:00 +0200

This is just a bunch of info around scratchbox

Startup

In order to disable VSDO on debian lenny AMD 64:

sudo sysctl abi.vsyscall32=2

sudo sysctl abi.vsyscall32=0

Both work, but when changint the vsdo to 2 i belieave its called compat mode, I have no idea what that actually is but I think i read somewhere that its a bit better in some ways than 0.

It might be possible that changing the mmap_min_addr also is nescesary:

sudo sysctl vm.mmap_min_addr=65536

I read somewhere that 4096 should suffice, but I have as of yet not run into any problems with the mmap_min_addr at 65536.

Xephyr

To start xehpyr

Xephyr :2 -host-cursor -screen 800x480x16 -dpi 96 -ac

wiki.maemo.org have -kb & at the end, however -kb doesn’t exist in the version provided with lenny

And in scratchbox, set the right display

export DISPLAY=:2

maemo-env

Starting the maemo environment inside scratchbox/xephyr af-sb-init.sh start

Other Issues

Had some troubles connecting to the internet through scratchbox, found out it was related to me not using my normal configuration (I’m tethering instead of using my wlan). When running ifconfig I got this error:

SIOCGIFCONF: Bad address

Through some IRC-logs (thx Lateralus for having the same issue :)) i found out resolv.conf in scratchbox used the wrong nameserver. I corrected the following three files to get it to work:

/scratchbox/etc/resolv.conf
/scratchbox/users/xun/targets/FREMANTLE_X86/etc/resolv.conf
/scratchbox/users/xun/targets/FREMANTLE_ARMEL/etc/resolv.conf

Where xun is my scratchbox user, ofc.

Hello World

Thu, 27 May 2010 00:00:00 +0200

My exams are over and most of my school work is done (need to write some java ee until next week), which means I will finally be able to focus on GSoC.

As you might guess my GSoC project is about improving Shepherd and later on I will publish my accepted proposal.

My progress thus far is rather limited. I have been able to get a working dev-env with scratchbox and the final maemo SDK up and running. I have installed and configured a blog-thingy and I have been able to compile and run some Qt applications. As I have no previous experience with Qt and no work experience with c++ I figure it will take some time for me to really get started.

I have found several sources of information that can be utilized for the actions/triggers I plan to implement. My next step is to build the ground of what will become the wlan-ssid trigger. I found the method allConfigurations() in QNetworkConfigurationManager which with the right filter should give me all avalible wlans. It does also seem that the same method could give me a trigger for other types of internet connections aswell, as GPRS och HSDPA for example.

With the release of PR1.2 Qt 4.6 is introduced, something almost required for my work and also shepherd. However I have failed to upgrade my device :( . The app manager tells me that I have to use Nokia software updater but when I tried that it failed with some error about not being able to download the update.