Sunday, June 7, 2020

Enabling IPv6 on Scaleway Dedibox using FreeBSD

First steps

There are two ways to configure IPv6 on a Dedibox:

  1. either your server supports SLAAC (see Dedibox' IPv6 SLAAC to see if it's the case)
  2. or using a DHCPv6 client

Either way, the first thing to do is to accept ICMPv6 Router Advertisements, this will get you a default route using a link-local IPv6 address:

# ifconfig bce0 inet6 accept_rtadv
# grep ^ifconfig_bce0_ipv6 /etc/rc.conf
ifconfig_bce0_ipv6="inet6 accept_rtadv"
# # netstat -rnf inet6 | grep ^default
default                           fe80::be16:65ff:fefb:d23f%bce0 UG        bce0

Also it doesn't hurt to start rtsold(8) (it looked like it worked without it though):

# grep ^rtsold /etc/rc.conf
rtsold_enable="YES"
# /etc/rc.d/rcsold start
Starting rtsold.

If your server supports SLAAC then I believe you can stop here. In my case I was unlucky and I had to use DHCPv6.

After some experimentation, I realized that you need to make you DHCP client request Prefix Delegation (PD), otherwise the router won't let you through. This is very loosely defined in Online´s documentation, because they use dhclient -P. But PD doesn't request a normal address, so you also need to requests a Non-temporary Address (NA) for this.

Unfortunately, FreeBSD's dhclient doesn't support IPv6, so you need another one. There are some references on Internet of people using net/dual-dhclient (GitHub) but I can't use this since my IPv4 setup is static.

Using ISC's dhclient from ports (failed)

Then I moved to net/isc-dhcp44-client because it supports IPv6. The first problem here is that it's not clear to me that FreeBSD's rc.conf(5) allows you to use DHCP only for IPv6. After glancing at /etc/network.subr it's not clear to me that the ifconfig_<if>_ipv6 actually support "DHCP" as a parameter but I may be wrong (and I didn't test that far, see below). Some post seems to indicate that's the case, since they use "DHCP" both in $ifconfig_em0 and ifconfig_em0_ipv6.

As explained in Dedibox' /48 IPv6 prefix page, you need to configure your DHCP unique identifier (DUID) in your dhclient.conf(5) file, which is /usr/local/etc/dhclient.conf (given you use dhclient(8) from the package). The default configuration contains some stuff, you can remove it all:

# cat /usr/local/etc/dhclient.conf
interface "bce0" {
        # DUID given by Dedibox
        # https://console.online.net/en/network/
        send dhcp6.client-id 01:23:45:67:89:AB:CD:EF:01:23;
}

Optionally, if you don't want dhclient-script(8) to overwrite resolv.conf(5), you need to create the following file (I hate the fact the path is hard coded but there are worse evils in the world):

# cat /etc/dhclient-enter-hooks 
make_resolv_conf() { :; }

Here is what you get:

# /usr/local/sbin/dhclient -6 -d -v bce0
Internet Systems Consortium DHCP Client 4.4.2
Copyright 2004-2020 Internet Systems Consortium.
All rights reserved.                                                  
For info, please visit https://www.isc.org/software/dhcp/ 
                                   
Listening on Socket/bce0                                              
Sending on   Socket/bce0                                              
PRC: Soliciting for leases (INIT).                                    
XMT: Forming Solicit, 0 ms elapsed.                                                                                                          
XMT:  X-- IA_NA 52:cd:78:96                                           
XMT:  | X-- Request renew in  +3600        
XMT:  | X-- Request rebind in +5400 
XMT: Solicit on bce0, interval 1090ms.
RCV: Advertise message on bce0 from fe80::be16:65ff:fefb:d23f.
RCV:  X-- Preference 255.
RCV:  X-- IA_NA 52:cd:78:96
RCV:  | X-- starts 1591003129
RCV:  | X-- t1 - renew  +10800
RCV:  | X-- t2 - rebind +172800
RCV:  | X-- [Options]
RCV:  | | X-- IAADDR 2001:aaaa:3bac::1
RCV:  | | | X-- Preferred lifetime 54000.
RCV:  | | | X-- Max lifetime 86400. 
RCV:  X-- Server ID: 00:01:00:01:1b:ac:bc:2d:10:60:4b:9b:0a:f4
RCV:  Advertisement immediately selected.
PRC: Selecting best advertised lease.
PRC: Considering best lease.
PRC:  X-- Initial candidate 00:01:00:01:1b:ac:bc:2d:10:60:4b:9b:0a:f4 (s: 10105, p: 255).
XMT: Forming Request, 0 ms elapsed. 
XMT:  X-- IA_NA 52:cd:78:96
XMT:  | X-- Requested renew  +3600
XMT:  | X-- Requested rebind +5400
XMT:  | | X-- IAADDR 2001:aaaa:3bac::1
XMT:  | | | X-- Preferred lifetime +7200
XMT:  | | | X-- Max lifetime +7500
XMT:  V IA_NA appended.
XMT: Request on bce0, interval 1040ms.
RCV: Reply message on bce0 from fe80::be16:65ff:fefb:d23f.
RCV:  X-- Preference 255.
RCV:  X-- IA_NA 52:cd:78:96
RCV:  | X-- starts 1591003129
RCV:  | X-- t1 - renew  +10800
RCV:  | X-- t2 - rebind +172800
RCV:  | X-- [Options]
RCV:  | | X-- IAADDR 2001:aaaa:3bac::1
RCV:  | | | X-- Preferred lifetime 7200.
RCV:  | | | X-- Max lifetime 86400. 
RCV:  X-- Server ID: 00:01:00:01:1b:ac:bc:2d:10:60:4b:9b:0a:f4
PRC: Bound to lease 00:01:00:01:1b:ac:bc:2d:10:60:4b:9b:0a:f4.
PRC: Renewal event scheduled in 10800 seconds, to run for 162000 seconds.
PRC: Depreference scheduled in 7200 seconds.
PRC: Expiration scheduled in 86400 seconds.

The interface will get an global IPv6 address but as I said above, Online's router won't let you unless request Prefix Delegation PD, so this is pretty useless. You do this with the -P flag but if you read the manual:

-P Enable IPv6 prefix delegation. This implies −6 and also disables the normal address query. See −N to restore it. Multiple prefixes can be requested with multiple −P flags. Note only one requested interface is allowed.

-N Restore normal address query for IPv6. This implies -6. It is used to restore normal operation after using -T or -P. Multiple addresses can be requested with multiple −N flags.

So that's what I did:

# /usr/local/sbin/dhclient  -d -v -6 bce0 -P -N
Internet Systems Consortium DHCP Client 4.4.2
Copyright 2004-2020 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
Listening on Socket/bce0
Sending on   Socket/bce0
PRC: Confirming active lease (INIT-REBOOT).
XMT: Forming Rebind, 0 ms elapsed.
XMT:  X-- IA_NA 52:cd:78:96
XMT:  | X-- Request renew in  +3600
XMT:  | X-- Request rebind in +5400
XMT:  X-- IA_PD 52:cd:78:96
XMT:  | X-- Requested renew  +3600
XMT:  | X-- Requested rebind +5400
XMT:  | | X-- IAPREFIX 2001:aaaa:3bac::/48
XMT:  | | | X-- Preferred lifetime +7200
XMT:  | | | X-- Max lifetime +7500
XMT:  V IA_PD appended.
XMT: Rebind on bce0, interval 990ms.
RCV: Reply message on bce0 from fe80::be16:65ff:fefb:d23f.
RCV:  X-- Preference 255.
RCV:  X-- IA_NA 52:cd:78:96
RCV:  | X-- starts 1591559136
RCV:  | X-- t1 - renew  +10800
RCV:  | X-- t2 - rebind +172800
RCV:  | X-- [Options]
RCV:  | | X-- IAADDR 2001:aaaa:3bac::1
RCV:  | | | X-- Preferred lifetime 54000.
RCV:  | | | X-- Max lifetime 86400.
RCV:  X-- IA_PD 52:cd:78:96
RCV:  | X-- starts 1591559136
RCV:  | X-- t1 - renew  +10800
RCV:  | X-- t2 - rebind +172800
RCV:  | X-- [Options]
RCV:  | | X-- IAPREFIX 2001:aaaa:3bac::/48
RCV:  | | | X-- Preferred lifetime 7200.
RCV:  | | | X-- Max lifetime 86400. 
RCV:  X-- Server ID: 00:01:00:01:1b:ac:bc:2d:10:60:4b:9b:0a:f4
PRC: Bound to lease 00:01:00:01:1b:ac:bc:2d:10:60:4b:9b:0a:f4.
Prefix REBIND6 old=2001:aaaa:3bac::/48 new=2001:aaaa:3bac::/48
PRC: Renewal event scheduled in 10800 seconds, to run for 162000 seconds.
PRC: Depreference scheduled in 7200 seconds.
PRC: Expiration scheduled in 86400 seconds.

But then I check my interface but the IPv6 address hadn't been assigned. This is were I threw the towel as I considered I had been too deep in the rabbit whole already.

Theorically I could just have used -P to open the router's gate and then rely on static IPv6 configuration. That would have actually been my favorite solution but I'm not sure FreeBSD's rc.conf(5) would allow this. Actually, it maybe be possible by usng /etc/start_if.bce0; to start /usr/local/sbin/dhclient -6 -P bce0 (and probably remove the ifdisabled inet6 flag on the interface first), but I haven't tried.

Using KAME's dhcp6

As I was wondering about Prefix Delegation and how I could re-use the information I got from the DHCP server to assign IPv6 addresses to my hosts, I came across this post. It's the way it should be: simple and straightforward. dhcp6c requests both PD and NA and in the PD response handler, you tell it how to assign an IPv6 address to the other interfaces. Then rtadvd(8) does the rest.

But I digress and I'm not there yet, I just want an IPv6 for my host. My point is that it's neat. I created a super basic configuration file for dhcp6c:

# cat /usr/local/etc/dhcp6c.conf
interface bce0 {
        send ia-na 1;
        send ia-pd 1;
        send rapid-commit;
};

id-assoc pd 1 {
};

id-assoc na 1 {
};

The attentive reader will notice there's a problem here. Where is the DUID configured? Well it turns out that dhcp6c doesn't allow to configure it by hand... I found this on Google Books (page 469 of IPv6 Advanced Protocols Implementation):

A DHCPv6 client or server needs its DUID for the protocol operation. The user does not have to configure the ID by hand: dhcp6c and dhcp6s automatically generate their type 1 DUIDs on their first invocation, and them them in volatile files, /var/db/dhcp6c_duid and /var/db/dhcp6sduid.

So we need to generate /var/db/dhcp6c_duid. Fortunately someone already did the work:

echo 01:23:45:67:89:AB:CD:EF:01:23 | \
  awk '{ gsub(":"," "); printf "0: 0a 00 %s\n", $0 }' | \
  xxd -r > /var/db/dhcp6c_duid

Let's try it manually:

# dhcp6c -df -c /usr/local/etc/dhcp6c.conf bce0
Jun/07/2020 23:35:12: failed to open /usr/local/etc/dhcp6cctlkey: No such file or directory
Jun/07/2020 23:35:12: failed initialize control message authentication
Jun/07/2020 23:35:12: skip opening control port
Jun/07/2020 23:35:13: Sending Solicit
Jun/07/2020 23:35:13: unexpected advertise
Jun/07/2020 23:35:13: Sending Request
Jun/07/2020 23:35:13: dhcp6c Received REQUEST
Jun/07/2020 23:35:13: add an address 2001:aaaa:3bac::1/128 on bce0

# ping6 -qc 1 google.com
PING6(56=40+8+8 bytes) 2001:aaaa:3bac::1 --> 2a00:1450:4007:811::200e

--- google.com ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 2.047/2.047/2.047/0.000 ms

Let's just enable dhcp6c at boot time:

# grep ^dhcp6 /etc/rc.conf
dhcp6c_enable="YES"
dhcp6c_interfaces="bce0"

One last thing: if you're using a firewall, be sure to let IPv6 UDP packets on port 546 come in. This is how DHCPv6 works. With pf, this would look like:

# grep 'port 546' /etc/pf.conf 
pass in log inet6 proto udp to port 546

And voilĂ !

Friday, May 1, 2020

Using Lenovo X1 Carbon volume/brightness keys under a bare X11

This is going to be a quick post.

When you don't use a fancy Window Manager (I'm using i3), there are many things which don't come out of the box. Well i3 supports keyboard bindings, but I'm not sure about special keys like volume/brightness. You may ask "but what are you using this then?", well it's another topic but I'll just say that I like its lightweightness.

Anyway, I'm sure there's more than one way to do this but one pretty neat method I found and like is using xbindkeys(1). Here is the relevant snippet of my $HOME/.xbindkeyrc:

"xterm"
  control + b:2

# Increase backlight
"xbacklight -inc 5"
  XF86MonBrightnessUp

# Decrease backlight
"xbacklight -dec 5"
  XF86MonBrightnessDown

# Increase volume
"amixer set Master 5%+"
  XF86AudioRaiseVolume

# Decrease volume
"amixer set Master 5%-"
  XF86AudioLowerVolume

# Mute
"amixer set Master toggle"
  XF86AudioMute

I let you figure out what this does, but that shouldn't be too hard...

Note that of other XF86 symbols, you can visit this page: http://wiki.linuxquestions.org/wiki/XF86_keyboard_symbols. You can also use xev(1) to figure out which code is emitted when one key is pressed.

Finally, xbindkeys(1) is a daemon which needs to be started, so just add this to your $HOME/.xinitrc:

xbindkeys

Have fun!

Saturday, February 15, 2020

Using acme.sh to generate a Let's Encrypt certificate

Like almost any individual on Earth involved a little bit in open-source, I use Let's Encrypt to generate my web certificates.

The first client I used for this was acme-client. It was conveniently available as FreeBSD's security/acme-client port but it has been removed. The reason, as far as I understand, is a bit silly: acme-client has been imported in OpenBSD and is now maintained there. The code on the original website is not updated anymore so the port has been removed. (It's a bit more work to maintain a tool from OpenBSD as a port, as someone needs to manually extract it from time to time and store it somewhere, I guess that's the fundamental reason.)

Anyway I had to find something else and after a bit of googling, I found a replacement: acme.sh, which is packaged as security/acme.sh. At first this is a bit scary (7000 lines of shell script, but well people have reported it works well and it's simple, so I gave it a try.

So here is how set it up (note that the ordering is important):

  1. First you need to configure you web server so that Let's Encrypt's service can read the challenge and verify the domain you claim is indeed yours. In my case, with Nginx:
        # This host only exist on port 443. Just accept Let's Encrypt challenges
        # in plain text and redirect anything else to the port 443.
        server {
            listen       80;
            server_name  foo.chchile.org;
            access_log   /var/log/nginx/foo-access.log;
            error_log    /var/log/nginx/foo-error.log info;
            allow all;
    
            root   /var/empty;
    
            location ^~ /.well-known/acme-challenge/ {
                alias /usr/local/www/acme/.well-known/acme-challenge/;
            }
    
            location / {
                return  301 https://foo.chchile.org$request_uri;
            }
        }
        
  2. Then you can run acme.sh to issue the certificate. It's really straightforward:
    # /usr/local/sbin/acme.sh  --issue -d foo.chchile.org -w /usr/local/www/acme
                            
    [Sat Feb 15 21:40:18 UTC 2020] Create account key ok.
    [Sat Feb 15 21:40:18 UTC 2020] Registering account                                                                                           
    [Sat Feb 15 21:40:19 UTC 2020] Registered                                                                                                    
    [Sat Feb 15 21:40:19 UTC 2020] ACCOUNT_THUMBPRINT='...'
    [Sat Feb 15 21:40:19 UTC 2020] Creating domain key
    [Sat Feb 15 21:40:19 UTC 2020] The domain key is here: /root/.acme.sh/foo.chchile.org/foo.chchile.org.key                        [Sat Feb 15 21:40:19 UTC 2020] Single domain='foo.chchile.org'                                                                         [Sat Feb 15 21:40:19 UTC 2020] Getting domain auth token for each domain                             
    [Sat Feb 15 21:40:20 UTC 2020] Getting webroot for domain='foo.chchile.org'
    [Sat Feb 15 21:40:20 UTC 2020] Verifying: foo.chchile.org
    [Sat Feb 15 21:40:24 UTC 2020] Success
    [Sat Feb 15 21:40:24 UTC 2020] Verify finished, start to sign.
    [Sat Feb 15 21:40:24 UTC 2020] Lets finalize the order, Le_OrderFinalize: https://acme-v02.api.letsencrypt.org/acme/finalize/78264375/2343222
    453
    [Sat Feb 15 21:40:25 UTC 2020] Download cert, Le_LinkCert: https://acme-v02.api.letsencrypt.org/acme/cert/04236a2f1bd00075bda7c3ed8bb9d2953b0
    0
    [Sat Feb 15 21:40:25 UTC 2020] Cert success.
    -----BEGIN CERTIFICATE-----
    [...]
    -----END CERTIFICATE-----
    [Sat Feb 15 21:40:25 UTC 2020] Your cert is in  /root/.acme.sh/foo.chchile.org/foo.chchile.org.cer 
    [Sat Feb 15 21:40:25 UTC 2020] Your cert key is in  /root/.acme.sh/foo.chchile.org/foo.chchile.org.key 
    [Sat Feb 15 21:40:25 UTC 2020] The intermediate CA cert is in  /root/.acme.sh/foo.chchile.org/ca.cer 
    [Sat Feb 15 21:40:25 UTC 2020] And the full chain certs is there:  /root/.acme.sh/foo.chchile.org/fullchain.cer 
        
  3. Now, let's configure the desired certificate paths in Nginx (there's not there yet, this will be the next step):
       server {
            listen       443 ssl;
            server_name  foo.chchile.org;
            access_log   /var/log/nginx/foo-access.log;
            error_log    /var/log/nginx/foo-error.log debug;
    
            ssl_certificate      /usr/local/etc/ssl/acme/foo.chchile.org/fullchain.pem;
            ssl_certificate_key  /usr/local/etc/ssl/acme/private/foo.chchile.org/privkey.pem;
    
            ssl_session_timeout  5m;
    
            ssl_protocols  TLSv1.1 TLSv1.2;
            ssl_ciphers  HIGH:!aNULL:!MD5;
            ssl_prefer_server_ciphers   on;
    
            [...]
        }
        
  4. It's almost done. acme.sh uses /root/.acme.sh as its work space. The certificate and the key are there but it's not advised to use them as is since the internal layout may chance in the future. So you need to install (read "copy") the certificate:
    # /usr/local/sbin/acme.sh --install-cert -d foo.chchile.org \
      --key-file /usr/local/etc/ssl/acme/private/foo.chchile.org/privkey.pem \
      --cert-file /usr/local/etc/ssl/acme/foo.chchile.org/cert.pem \
      --fullchain-file /usr/local/etc/ssl/acme/foo.chchile.org/fullchain.pem \
      --reloadcmd "service nginx restart"
    [Sat Feb 15 21:42:24 UTC 2020] Installing cert to:/usr/local/etc/ssl/acme/foo.chchile.org/cert.pem
    [Sat Feb 15 21:42:24 UTC 2020] Installing key to:/usr/local/etc/ssl/acme/private/foo.chchile.org/privkey.pem
    [Sat Feb 15 21:42:24 UTC 2020] Installing full chain to:/usr/local/etc/ssl/acme/foo.chchile.org/fullchain.pem
    [Sat Feb 15 21:42:24 UTC 2020] Run reload cmd: service nginx restart
    Performing sanity check on nginx configuration:
    nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
    nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
    Stopping nginx.
    Waiting for PIDS: 29714.
    Performing sanity check on nginx configuration:
    nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
    nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
    Starting nginx.
    [Sat Feb 15 21:42:24 UTC 2020] Reload success
        
  5. And finally, set up a cron job to renew the certificate whenever it's needed. That's where acme.sh is neat, everything you typed above has been recorded in /root/acme.sh, so the crontab(5) line just looks like:
    0       0       *       *       *       root    /usr/local/sbin/acme.sh --cron > /dev/null
        

Blog re-purpose

It's a been a long time since my last post.

My life has changed a lot since then and I don't have much time anymore to play with my computer for fun, even less to write articles to share my knowledge and learning. Hence from now on, when I will post something this will be mostly as a memo for myself for future use. Since it's public, I will still try to make the posts look decent but the style will be much more terse.

So why keep posting at all? I doubt I have any reader anyway. But maybe sometimes someone will be desperate enough to go to Google's 10th result page and will come across my blog. If this can help this soul, I'll be more than happy!

Sunday, July 27, 2014

Installing FreeBSD on a Dedibox (online.net)

Created: 2014/07/27
Updated: 2014/08/12 - higher-end Dedibox
Updated: 2014/08/14 - software RAID-1

My first server was a Dedibox. I then switched to OVH's Kimsufi (for the anecdote, "Kimsufi" sounds like "enough for me" in French) which at the time was more attractive (15 EUR/month instead of 20).  My setup then evolved to make use of failover IPs (an extra IP address which you can switch from one OVH server to another).

But now my Kimsufi servers are getting old and slow or expensive, depending for which side you're looking at it and I'd like to upgrade them. OVH has very attractive prices like 5 EUR/month for the cheapest one (at least theoritically given I've never seen them available despite them constantly announcing the last server was shipped a few hours ago).  For the price I pay today overall, I could get an i5 CPU on the main server and a cheap and low-grade server for the failover...

... if only OVH was still proposing failover IPs on Kimsufi grade servers. The Kimsufi offer has spun off to really focus on home-servers where -- I guess -- people don't need such fancy services. If you want to have a failover IP at OVH now, you need to get an enterprise grade server which costs at least 80 EUR/month.  I guess this move has been motivated by the IPv4 addresses crunch.

Anyway.  I'm now going back to Dedibox which still offers failover IPs. Also they offer access to the server's console through a Java applet, which can be super useful. Something OVH did back in the time but then removed. However Dedibox don't offer FreeBSD by default (which Kumsufi did at the time, and maybe still do) so you have to install it yourself. Here is how:

  • Boot an Ubuntu rescue system, login and run sudo to be root.
  • Download a FreeBSD release or snapshot ISO; 10-STABLE can be found here: http://ftp2.fr.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/10.0/
  • Install QEMU:
    apt-get update && apt-get install qemu-kvm
    
  • Start QEMU with a VNS server, attached to the raw disk, booting on the ISO:
    qemu-system-x86_64 -no-kvm -hda /dev/sda -cdrom FreeBSD-*.iso -net nic,model=e1000 -vnc :1 -boot d
    or, if you have two disks:
    qemu-system-x86_64 -no-kvm -hda /dev/sda -hdb /dev/sdb -cdrom FreeBSD-*.iso -net nic,model=e1000 -vnc :1 -boot d
  • Connect in using VNC:
    xvncviewer ${server_ip}:1
Now you can install FreeBSD. The only thing is that the bootloader won't be installed (correctly?) for some unknown reason. So you need to do it yourself.
I couldn't come up with the a way to partition the server the way I want with the bsdinstaller, so I typically switch to the console (Alt-F4) right after choosing the keymap to make it using gpart. I'll describe it here for the record, so it will save me from research the next time I'll do it.

Here is the partitioning scheme:
  • We're in 2014, so we're using GPT boot, which requires a small partition for the bootloader.
  • I want the base system in UFS, mainly because it's less brittle to deal with remotely and you can use nextboot(8) fully (the new kernel will be booted only once and they the previous one will be reinstated, so you can try kernels); although all this is less mandatory if you have a working console as with Dedibox.
  • A small swap partition.
  • The remaining in ZFS.

You have a single disk or a hardware RAID controller

Here is how to do it from scratch:

###
### Setup partitions
###
# dd if=/dev/zero of=/dev/ada0 bs=64k count=128
128+0 records in
128+0 records out
8388608 bytes transferred in 0.042453 secs (197599244 bytes/sec)
# gpart show
# gpart create -s GPT ada0
ada0 created
# gpart add -t freebsd-boot -s 64k -i 1 ada0
ada0p1 added
# gpart add -t freebsd-ufs -s 20G -i 2 ada0
ada0p2 added
# gpart add -t freebsd-swap -s 2G -i 3 ada0
ada0p3 added
# gpart add -t freebsd-zfs -i 4 ada0
ada0p4 added
###
### Create the filesystems
###
# newfs -j /dev/ada0p2
[...]
# zpool create tank /dev/ada0p4
cannot mount '/tank': failed to create mountpoint (this is expected and harmless)
###
### Install the bootcode manually as it will fail for some reason
###
# gpart bootcode -b /boot/pbmr -p /boot/gptboot -i 1 ada0
bootcode written to ada0

Now go ahead and install FreeBSD.

You want to software RAID-1

Note that we use geom_mirror for the first three partition, and ZFS mirroring for the pool as I think it has better performances.

Also, note that the current bsdinstaller does not seem to understand what is a geom_mirror device and wants us to create a partitioning scheme on it.  We will therefore mount the partition to /mnt manually.

###
### Setup partitions on the first disk, ada0
###
# dd if=/dev/zero of=/dev/ada0 bs=64k count=128
128+0 records in
128+0 records out
8388608 bytes transferred in 0.042453 secs (197599244 bytes/sec)
# gpart show
# gpart create -s GPT ada0
ada0 created
# gpart add -t freebsd-boot -s 64k -i 1 ada0
ada0p1 added
# gpart add -t freebsd-ufs -s 20G -i 2 ada0
ada0p2 added
# gpart add -t freebsd-swap -s 2G -i 3 ada0
ada0p3 added
# gpart add -t freebsd-zfs -i 4 ada0
ada0p4 added
###
### Now duplicate those steps for ada1.
###
[...]
###
### Now create the mirror
###
# kldload geom_mirror
# gmirror label gm-root ada0p2 ada1p2
# gmirror label gm-swap ada0p3 ada1p3
###
### Create the filesystems
###
# newfs -j /dev/ada0p2
[...]
# zpool create tank mirror /dev/ada0p4 /dev/ada1p4
cannot mount '/tank': failed to create mountpoint (this is expected and harmless)
###
### Install the bootcode manually as it will fail for some reason
###
# gpart bootcode -b /boot/pbmr -p /boot/gptboot -i 1 ada0
bootcode written to ada0
# gpart bootcode -b /boot/pbmr -p /boot/gptboot -i 1 ada1
bootcode written to ada1
###
### Mount the partition to install FreeBSD
###
# mount /dev/mirror/gm-root /mnt

Now when the installer asks you about the partitioning, select "Shell" and FreeBSD will be installed to /mnt once you exit this shell.

End of installation

A few notes (some for myself):
  • The default router in online.net's network is .1.
  • Add the following lines to /etc/rc.conf
  • sendmail_submit_enable=NO
    sendmail_outbound_enable=NO
    sendmail_msp_queue_enable=NO
    
  • YMMV but on QEMU I have an em0 interface, but on the real server it can be igb0 or bce0; so I need to change my /etc/rc.conf accordingly. What I typically do if I'm not sure, is duplicate the ifconfig_em0 line to ifconfig_igb0 and ifconfig_bce0.
  • Add a user or enable root login on sshd.
  • If you are using software RAID, add geom_mirror_load="YES" to /boot/loader.conf.
  • If your server has a PERC h200 controller, the disk won't be /dev/ada0 but /dev/da0, even on FreeBSD10, so change /etc/fstab accordingly.
  • Add the swap to /etc/fstab.
Small example of what /etc/fstab look like (with a software mirror in that case, but you cannot boot on a mirror, the subsystem is not there yet at boot, so you need to pick one of the disk):
/dev/mirror/gm-root     /               ufs     rw      1       1
/dev/mirror/gm-swap     none            swap    sw      0       0
Now before trying to boot FreeBSD on the real server, just try it on QEMU, this will save you some time if you missed something. First umount cleanly your disks and then kill QEMU from the host. Now re-run it using:
qemu-system-x86_64 -no-kvm -hda /dev/sda -net nic,model=e1000 -vnc :1 -boot c
This should go to the boot pompt. Shut down cleanly. Then you can try to boot it on the real server. If it does not come only, you can still use the Java console to debug it (for example, the interface name may not be correct).

Tuesday, May 6, 2014

Netboot OpenBSD on a Soekris

This post is a reminder for me on how to do this. This is the second time I spend two entire evenings to achieve this, and I seem to remember both time I succeeded using different methods. This time I decided to dump what I have in mind because in 2014, this is just silly how backward and complicated it is :). By the way, I know some people more experienced than me in OpenBSD who may read this, feel free to comment and tell me what I'm doing wrong to end up with something so convoluted :).

I'm using a Debian machine as a server. There is a direct Ethernet cable from the Soekris to the server (10.0.0.1), nothing else is on the network.

  1. Download the OpenBSD release and extract it to /openbsd.
  2. Install the following packages on the Debian server:
    • udhcpd, because it's way simpler than ISC dhcpd
    • atftpd, do NOT use the tftpd package, which is the implementation from NetKit, this is really broken and althrough supposed to be very simple I wasted hours on this
    • rarpd
    • bootparamd
    • nfs-kernel-server
  3. Here are the relevant bits for /etc/udhcpd.conf:
    start     10.0.0.100
    end       10.0.0.110
    interface eth2
    siaddr    10.0.0.1
    sname     debian
    bootfile  /tftpboot/pxeboot
    option    dns      8.8.8.8 8.8.4.4
    option    router   10.0.0.1
    option    domain   domain.local
    option    subnet   255.255.255.0
    option    lease    864000
    option    rootpath /openbsd
    
  4. Start it:
    udhcpd -f /etc/udhcpd.conf
  5. atftpd probably added a similar line to /etc/inetd.conf; keep everything as-is and change the serving path (last argument):
    tftp            dgram   udp4    wait    nobody /usr/sbin/tcpd /usr/sbin/in.tftpd --tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr 239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /openbsd
  6. Start or reload inetd.
  7. Now I advise to start a
    tcpdump -ev
    in one terminal t
    o see what's going on on the wire, and
    tail -f /var/log/daemon.log
    in an other. You can give a try to the Soekris: provided you have a serial cable on it (otherwise I don't even know why you're reading this), hit Ctrl-P at the boot and type:
    boot f0
    This won't work, but note the Soekris MAC address and the IP address which has been distributed to it.
  8. Now fill up /etc/ethers for rarpd (that's oldschool, isn't it? :)):
    00:00:24:14:11:80   10.0.0.102
  9. Now let's configure bootparamd by first telling the Soekris name in /etc/hosts:
    10.0.0.102   soekris
    and then giving NFS root in /etc/bootparams.conf (!)
    soekris root=10.0.0.1:/openbsd
  10. Export /openbsd in NFS in /etc/exports:
    /openbsd   10.0.0.0/24(rw,no_root_squash)
    and reload the list of exported filesystems:
    /etc/init.d/nfs-kernel-server restart
  11. Now configure a bit OpenBSD before booting it:
    1. /openbsd/etc/boot.conf (set the console to see the kernel boot and tell pxeboot where to find the kernel):
      stty com0 19200
      set tty com0
      boot tftp:/tftpboot/bsd
    2. /openbsd/etc/ttys (enable getty on the console and set the correct speed):
      console "/usr/libexec/getty std.19200"  vt220   on  secure
  12. Verify that everything is running: inetd, udhcpd, rarpd, all RPC daemons (rpcbind should be start before all of them), including rpc.bootparamd.
  13. You should now be able to boot, with root having no password!

Thursday, February 21, 2013

Poor man's browser sandboxing

Last update: Sat Feb 23 15:14:13 CET 2013

Nowadays I use my browser most of the time, as you probably do as well. And like me, you are probably quite annoyed to run this big software blob and its unavoidable bugs against so many websites. Yes, browsers are the prime attack vector now.

So why not sandbox it as another user so as to insulate it from you sensitive data (ssh keys/agent, files, ...)? You can do this in about 15 minutes. This is more a quick reminder for me than a full-fledged blog post, so the explanations and commands are a little terse. Adapt them to fit your need.

$ sudo adduser -m browser
$ sudo mkdir ~browser/.ssh
$ sudo cp ~/.ssh/id_rsa.pub ~browser/.ssh/authorized_keys

# You should shut down your browsers before doing this.
$ sudo mkdir ~browser/.config
$ sudo cp -Rp ~/.config/google-chrome ~browser/.config/
$ sudo cp -Rp ~/.mozilla ~browser/
$ sudo chown -R browser ~browser

$ cat > bin/runasbrowser.sh << EOF
#!/bin/sh
exec ssh -Xf browser@localhost "$@"
EOF
$ chmod +x bin/runasbrowser.sh

# Be sure that your ~/bin belongs to your $PATH.
$ runasbrowser Xephyr :1
$ runasbrowser google-chrome

Notes:

  • You will probably need to put user "browser" in a group to allow him to play music :).
  • I've got the idea of this here; you can see that the author did much more things than me to set this up. I'm not sure why it works for me with so few steps.

Tuesday, December 11, 2012

Producers/consumers timely queue in Python

For one of my projects (Push2mob), I had to implement a multiple producers / multiple consumers timely queue, that is a priority queue where the priority is a timestamp at which the item should be delivered.  After multiple more or less successful attempts I've finally come up with an implementation that was efficient and neat.  I've posted it on StackOverflow to ask for some review, and got a good one.

Here is the final result.  I would be glad if it could help someone.

# Copyright (c) 2012, Jeremie Le Hen 
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met: 
#
# 1. Redistributions of source code must retain the above copyright notice, this
#    list of conditions and the following disclaimer. 
# 2. Redistributions in binary form must reproduce the above copyright notice,
#    this list of conditions and the following disclaimer in the documentation
#    and/or other materials provided with the distribution. 
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

import collections
import heapq
import threading
import time

class TimelyQueue:
    """
    Implements a similar but stripped down interface of Queue which
    delivers items on time only.
    """

    def __init__(self, resolution=5):
        """
        `resolution' is an optimization to avoid wasting CPU cycles when
        something is about to happen in less than X ms.
        """
        self.timerthread = threading.Thread(target=self.__timer)
        self.timerthread.daemon = True
        self.resolution = float(resolution) / 1000
        self.queue = []
        self.triggered = collections.deque()
        self.putcond = threading.Condition()
        self.getcond = threading.Condition()
        # Optimization to avoid waking the thread uselessly.
        self.putwaketime = 0
        self.terminating = False
        self.timerthread.start()

    def put(self, when, item):
        """
        `when' is a Unix time from Epoch.
        """
        with self.putcond:
            heapq.heappush(self.queue, (when, item))
            if when < self.putwaketime or self.putwaketime == 0:
                self.putcond.notify()

    def get(self, timeout=None):
        """
        Timely return the next object on the queue.
        """
        with self.getcond:
            if len(self.triggered) > 0:
                when, item = self.triggered.popleft()
                return item
                self.getcond.wait(timeout)
            try:
                when, item = self.triggered.popleft()
            except IndexError:
                return None
            return item

    def qsize(self):
        """
        Self explanatory.
        """
        with self.putcond:
            return len(self.queue)

    def terminate(self):
        """
        Request the embedded thread to terminate.
        """
        with self.putcond:
            self.terminating = True
            self.putcond.notifyAll()

    def __timer(self):
        with self.putcond:
            maxwait = None
            while True:
                curtime = time.time()
                try:
                    when, item = self.queue[0]
                    maxwait = when - curtime
                    self.putwaketime = when
                except IndexError:
                    maxwait = None
                    self.putwaketime = 0
                self.putcond.wait(maxwait)
                if self.terminating:
                    return

                curtime = time.time()
                while True:
                    # Don't dequeue now, we are not sure to use it yet.
                    try:
                        when, item = self.queue[0]
                    except IndexError:
                        break
                    if when > curtime + self.resolution:
                        break

                    self.triggered.append(heapq.heappop(self.queue))
                if len(self.triggered) > 0:
                    with self.getcond:
                        self.getcond.notify(len(self.triggered))


if __name__ == "__main__":
    q = TimelyQueue()
    N = 100000
    t0 = time.time()
    for i in range(N):
        q.put(time.time() + 2, i)
    dt = time.time() - t0
    print "put done in %.3fs (%.2f put/sec)" % (dt, N / dt)
    t0 = time.time()
    i = 0
    while i < N:
        a = q.get(3)
        if i == 0:
            dt = time.time() - t0
            print "start get after %.3fs" % dt
            t0 = time.time()
        i += 1
    dt = time.time() - t0
    print "get done in %.3fs (%.2f get/sec)" % (dt, N / dt)
    q.terminate()
    # Give change to the thread to exit properly, otherwise we may get
    # a stray interpreter exception.
    time.sleep(0.1)

Friday, June 15, 2012

Using raw devices with VirtualBox run as user

Today I needed to run a virtual machine on my laptop. It is not very powerful so I wanted to avoid as much overhead as possible: skipping the host's VFS/filesystem layer is fairly easy, you just have to give access to a raw partition the your virtualization software instead of a file. After all, both are just seen as a big array of bytes. My hard drive is under LVM, so I created a dedicated logical volume that I wanted to be writable by me given I planned to run VirtualBox as a user.

I knew I could do this with udev(8) but skimming through the documentation and fumbling its rules would have been too long for my available time, so I tried Google with no luck and finally asked on IRC, where I found that someone already did this for the same reason.

The configuration line is quite easy:

root@r2d2# cat /etc/udev/rules.d/99-my.rules
SUBSYSTEM=="block",KERNEL=="dm-*",ACTION=="add|change",ENV{DM_NAME}=="*-vbox*",GROUP="jlh"

This rules basically tells that for any add or change of a block device named "dm-*" and matching "*-vbox*", change its group to "jlh" (note that == and = are different, as in many programming languages). One interesting thing to note is that ENV{DM_NAME}=="*-vbox*" is an helper environment variable that is set by udev(8) standards rules. Those stand in /lib/udev/rules/ on Debian and udev(8) merges the content of this directory with the standard configuration directory /etc/udev/rules.d/. Rules are applied by filename order, so be careful to be the last one. Initially I used "90-my.rules" but there is a rule in "91-permissions.rules" that overrode mine. You can easily debug by running udevd --debug, although the output is quite verbose.

The next step is to create a VMDK file for VirtualBox that will point to the raw device and then attach is to your VM's storage controller. This is quite well documented in the manual (Using a raw host hard disk from a guest).

Basically:

jlh@r2d2$ VBoxManage internalcommands createrawvmdk \
    -filename VirtualBox VMs/vm1/data.vmdk \
    -rawdisk /dev/mapper/vg0-vbox_vm1
jlh@r2d2$ VBoxManage showvminfo vm1 | grep 'Storage Controller Name'
Storage Controller Name (0):            IDE Controller
jlh@r2d2$ VBoxManage storageattach vm1 --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium /home/jlh/VirtualBox\ VMs/FreeBSD/data.vmdtk

Your mileage may vary if, for example, you have a different storage controller name, different port or device. The full "showvminfo" output will tell you which slot is available. Another solution is to add another storage controller, although VirtualBox will not permit you to have multiple IDE controller. You can add a S-ATA controller which allows you to plug up to 30 devices:

jlh@r2d2$ VBoxManage storagectl vm1 --name "SATA Controller" --add sata --controller IntelAHCI --bootable on
jlh@r2d2$ VBoxManage storageattach vm1 --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium /home/jlh/VirtualBox\ VMs/boot.vmdk
jlh@r2d2$ VBoxManage storageattach vm1 --storagectl "IDE Controller" --port 1 --device 0 --type hdd --medium /home/jlh/VirtualBox\ VMs/root.vmdk

VitualBox and VboxManage are pretty well documented.

Wednesday, March 28, 2012

A Varnish threads story

Varnish is a (now) well-known HTTP caching reverse-proxy. It has been written primarily by Poul-Henning Kamp, a famous FreeBSD developer. Varnish is very BSDish: simple, versatile and powerful.

(Yet, configuring it may be pretty tough because HTTP is a complex protocol with regard to caching (RFC 2616 mentions client-side proxies but not server-side ones). Besides, applications living on top of it are often written without any caching consideration in mind. For instance by default Varnish doesn't cache response from requests containing cookies, not it caches responses with a Set-Cookie header, for obvious reasons. Unfortunately PHP applications make heavy use of the PHPSESSID cookie simply because the session_start() function, which is part of the PHP library, is very handy for developers.

Varnish uses a pool of threads to serve requests, with a configurable minimum and maximum values as well as a timeout value (with the -w command-line option). Much like what Apache does with processes when used with the MPM prefork module. Additionally, Varnish enforces a configurable delay between thread creation (parameter's name is thread_pool_add_delay, you can configure it with the -p command-line option).

For some reason, one Varnish instance on a preproduction server here was configured with silly values regarding thread limits: only one thread at minimum. Given the server was often unused, threads timed out and were removed down to one. The problem was that when a developer wanted to test the websites, there was only one thread available and the aforementioned delay between thread creation prevented from spawning them all at one. Albeit being a very powerful server, the website was felt very sluggish.

It took me some time to find out this problem. When I modified the configuration, the website was really, really fast.

Tuesday, March 6, 2012

Portmaster options combo to upgrade FreeBSD ports

Some years ago I was using the famous Portupgrade to maintain my ports. This software is mature, very powerful and easy to use. Unfortunately its dependency on Ruby makes it really cumbersome, especially because I have many jails.

Therefore when Doug Barton began Portmaster, which is written is shell and does more or less the same thing (well, actually less, but I can live with it), I was quite eager to use it. One thing I didn't like from the beginning with Postmaster was that it is not able to work alone: it is constantly asking things. Of course there are options to disable this, but this leads to me the second problem: they are not intuitive! (at least for me...)

After some struggle, I finally managed to find the options I always want to use and I'm writing them as a reminder and in the hope to help someone else in the same hassle:

# portmaster -dBGm BATCH=1 --no-confirm --delete-packages -a

Here are the details:

  • I'm using portconf to configure the ports' build knobs, so I don't want to run the configuration or to be asked something about it. Just use the defaults unless I told otherwise: -G -m BATCH=1;
  • Don't create a backup package, I'm not running any financial application: -B;
  • Don't ask me if the distfiles must be cleaned, just do it: -d;
  • Don't ask me if I really want to upgrade my ports, I already executed the command proving it: --no-confirm;
  • Remove packages once installed: --delete-packages;
  • Upgrade everything: -a, but you might not want to ugprade everything at once so you can replace this with one or more port name.

Wednesday, November 9, 2011

Apache mod_rewrite evilness (with dynamic vhosts and .htaccess rewrite rules)

At my new $job, we have a SVN repository for the websites we are maintaining. We devised a workflow to work with it: each developer has one or more branches for himself. The merge of their features is done in the trunk. Once everything seems to work, we merge into the "preproduction" branch and finally in the "production" branch.

On the developement web server, I wanted them to be able access every branch with their web browser. Initially, there was a "svn.mywebsite.com" virtual host and each branch was accessible through an URL-path within it. Unfortuntaly, for "historical" reason, the web site doesn't work correctly if set in an URL sub-directory (and we are currently writing a new version of this website, so we actually don't want to spend time fixing it). I am therefore doomed to create a virtual host for each SVN trunk/tag/branch.

Here is the relevant part of the initial configuration I wrote:

<VirtualHost *:80>
ServerName svn.mywebsite.com
ServerAlias *.svn.mywebsite.com

DocumentRoot /var/empty

RewriteEngine on

RewriteCond %{HTTP_HOST} ^trunk\.svn\.mywebsite\.com$
RewriteRule $(.*) /home/www-data/svn/project/trunk$1 [L]

RewriteCond %{HTTP_HOST} ^([^.]+).branches\.svn\.mywebsite\.com$
RewriteRule $(.*) /home/www-data/svn/project/branches%1$1 [L]

RewriteCond %{HTTP_HOST} ^([^.]+).tags\.svn\.mywebsite\.com$
RewriteRule $(.*) /home/www-data/svn/project/tags%1$1 [L]

</VirtualHost>


So far it's easy and it would have worked if there wasn't the following RewriteRule in the .htaccess at the root of the project:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]


The problem with this rule is that if we request a "virtual URL" (which does not match a physical file in the hierarchy), index.php is called with the original URL in the query string, which results in an internal redirect within Apache.

I save you from the RewriteLog, but let's say you try: http://trunk.svn.mywebsite.com/virtual-url.


  • Initially the ${REQUEST_URI} is "/virtual-url". The vhost rewrite rules are applied, which redirect to the filesystem path: /home/www-data/svn/project/trunk/virtual-url.


  • Then we reach the per-directory (.htaccess) rewrite rules which, given the file doesn't exist, redirect to /home/www-data/svn/project/trunk/index.php with the following query string q=virtual-url.


  • Here is the first trap: an internal redirect is done within Apache, which restarts the rewrite rules evaluation from the beginning, with ${REQUEST_URI} set to /home/www-data/svn/project/trunk/index.php, while the ${HTTP_HOST} is still the same. So the directory will be prepended twice if we do not put a safeguard: basically checking that $REQUEST_URI doesn't contain /home/www-data/svn/project/.


  • But the true evilness is here: we cannot rewrite to a full filesystem path in a per-directory rewrite rule. The subsequent internal redirect will invariably think that this is an URL path, that is it will try to serve a page as if you had requested: http://trunk.svn.mywebsite.com/home/www-data/svn/project/trunk/index.php. Because of the safeguard above, the dynamic vhost magic will not apply, and Apache will try to reach this file from the vhost's DocumentRoot and you will get a 404.

    The workaround for this is trick Apache into thinking the content of ${REQUEST_URI} is a full filesystem path if the latter looks like a filesystem path :-). Contrary to the per-directory rewrite rules, the vhost rewrite rules are able to redirect to a full filesystem path. So just match the whole content and redirect to it.




<VirtualHost *:80>
ServerName svn.mywebsite.com
ServerAlias *.svn.mywebsite.com

DocumentRoot /var/empty

RewriteEngine on

RewriteRule ^(/home/www-data/svn/project/.*) $1 [L]

RewriteCond %{HTTP_HOST} ^trunk\.svn\.mywebsite\.com$
RewriteRule $(.*) /home/www-data/svn/project/trunk$1 [L]

RewriteCond %{HTTP_HOST} ^([^.]+).branches\.svn\.mywebsite\.com$
RewriteRule $(.*) /home/www-data/svn/project/branches%1$1 [L]

RewriteCond %{HTTP_HOST} ^([^.]+).tags\.svn\.mywebsite\.com$
RewriteRule $(.*) /home/www-data/svn/project/tags%1$1 [L]

</VirtualHost>


The first rewrite rule matches a ${REQUEST_URI} containing the a filesystem path. This is not exactly a rewrite, this is just a trick to trigger the mod_rewrite evaluation of the substitution.

Saturday, October 8, 2011

Installing full-ZFS server at OVH

OVH is a french web hosting service in France, that provides dedicated servers (and many other things). It is great because they offer an infrastructure which brings you low-cost but yet professional facilities for less than 20 euros a month. They provide Linux, FreeBSD and even Solaris: you can of course ask for your server to be installed with this but the real great thing is the netboot feature that will boot the same OS as the one which is installed on your server.

The FreeBSD installation is UFS based. It is nonetheless possible to migrate in on ZFS with little wizardry. It is best the do this with a fresh installation, but is should be possible to do so as long as you use less than the half of you hard drive. However, you have to move everything into the first physical half of the hard drive (it is easier when the server has just been installed, as you just have to keep the root partition).

The procedure is the following: You move all your data in a (small) transient partition at the end of the disk. Then a ZFS partition is created at the beginning of the disk, and again move your data there. You can then destroy the transient partition and create a physical swap partition in its stead. Indeed, although FreeBSD can use a ZFS vdev as swap, it cannot dump to it. Therefore this procedure creates a real partition for swap.

Your FreeBSD is booted. Go to the OVH manager and in the "Netboot" page and select "rescue-pro". Then reboot you server, wait for a while, you should receive a mail with the root password of your netbooted server.

Once connected on it, create a partition large enough to hold all your data at the end of the hard drive. We will copy them here in order to be able to install the ZFS partition at the beginning of the disk.


rescue-bsd# fdisk /dev/ad0
******* Working on device /dev/ad0 *******
[...]

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 488397105 (238475 Meg), flag 80 (active)
beg: cyl 1/ head 0/ sector 1;
end: cyl 655/ head 0/ sector 63
The data for partition 2 is:

The data for partition 3 is:

The data for partition 4 is:


rescue-bsd# bsdlabel /dev/ad0s1
# /dev/ad0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 20971520 0 4.2BSD 4096 16384 64
b: 2097152 20971520 swap
c: 488397105 0 unused 0 0 # "raw" part, don't edit
d: 465328433 23068672 4.2BSD 0 0 0

rescue-bsd# mount /dev/ad0s1a /mnt/
rescue-bsd# df -k /mnt/
Filesystem 1024-blocks Used Avail Capacity Mounted on
/dev/ad0s1a 10154158 495960 8845866 5% /mnt
rescue-bsd# umount /mnt/


We have a swap partition on /dev/ad0s1b and an empty filesystem on /dev/ad0s1d. The root partition only uses 500 MB. We are going to create a partition at the end of the disk to copy the content of it. Thus this partition must be large enough. But this partition should also large enough to hold the swap partition you want on your system eventually. In this example I want 1 GB of swap.

So let's create a 1 GB partition at the end of the disk. 1 GB is 1024*1024*1024/512 = 2097152 sectors. The disk is 488397105 sectors wide, so the partition would start at 488397105 - 2097152 = 486299953. A good practice is to align the partition to a 4 KB boundary: 486299953 % 4096 = 2353, so we will use 486299953 - 2353 = 486297600 for first sector of the partition. Given the end of the disk is 488397105, the partition size will be 488397105 - 486297600 = 2099505 sectors.


rescue-bsd# bsdlabel /dev/ad0s1 > /tmp/ad0.label
rescue-bsd# vi /tmp/ad0.label
[...]

rescue-bsd# cat /tmp/ad0.label
# /dev/ad0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 20971520 0 4.2BSD 4096 16384 64
b: 2099505 486297600 4.2BSD 4096 16384 64
c: 488397105 0 unused 0 0 # "raw" part, don't edit

rescue-bsd# bsdlabel -R /dev/ad0s1 /tmp/ad0.label
rescue-bsd# newfs /dev/ad0s1b
/dev/ad0s1b: 1025.1MB (2099504 sectors) block size 16384, fragment size 2048
using 6 cylinder groups of 183.77MB, 11761 blks, 23552 inodes.
super-block backups (for fsck -b #) at:
160, 376512, 752864, 1129216, 1505568, 1881920

rescue-bsd# mount /dev/ad0s1b /mnt
rescue-bsd# cd /mnt
rescue-bsd# dump -0af - /dev/ad0s1a | restore -rf -
DUMP: Date of this level 0 dump: Sat Oct 8 10:07:58 2011
DUMP: Date of last level 0 dump: the epoch
DUMP: Dumping /dev/ad0s1a to standard output
DUMP: mapping (Pass I) [regular files]
DUMP: mapping (Pass II) [directories]
DUMP: estimated 499089 tape blocks.
DUMP: dumping (Pass III) [directories]
DUMP: dumping (Pass IV) [regular files]
[...]
DUMP: finished in 131 seconds, throughput 3816 KBytes/sec
DUMP: DUMP IS DONE
rescue-bsd# cd
rescue-bsd# umount /mnt


Now we can remove the first partition and create a big ZFS partition spanning from the beginning of the disk to the beginning of the second partition we have just created.


rescue-bsd# gpart show ad0s1
=> 0 488397105 ad0s1 BSD (233G)
0 20971520 1 freebsd-ufs (10G)
20971520 465326080 - free - (222G)
486297600 2099505 2 freebsd-ufs (1.0G)

rescue-bsd# gpart delete -i 1 ad0s1
ad0s1a deleted
rescue-bsd# gpart show ad0s1
=> 0 488397105 ad0s1 BSD (233G)
0 486297600 - free - (232G)
486297600 2099505 2 freebsd-ufs (1.0G)

rescue-bsd# gpart add -s 486297600 -t freebsd-zfs ad0s1
ad0s1a added
rescue-bsd# gpart show ad0s1
=> 0 488397105 ad0s1 BSD (233G)
0 486297600 1 freebsd-zfs (232G)
486297600 2099505 2 freebsd-ufs (1.0G)


Now let's create the ZFS pool. But the OVH netboot only provides a read-only root filesystem, so we have to tell zpool(8) to put the cache file into /tmp (this file will be needed to import the pool at boot time). We must also to tell the ZFS layer to temporary mount the pool into /mnt, so it won't try to mount the root of the pool as /.


rescue-bsd# kldload opensolaris
rescue-bsd# kldload zfs
rescue-bsd# zpool create -o cachefile=/tmp/zpool.cache -o altroot=/mnt zroot /dev/ad0s1a
rescue-bsd# zpool export zroot


Install the various bootcodes. The MBR bootcode should already be there anyway. The ZFS bootcode is somewhat strange because it consists actually of two parts that must be written at different places (note that the first dd(1) uses /dev/ad0s1 while the second one uses /dev/ad0s1a):

rescue-bsd# gpart bootcode -b /boot/boot0 ad0
bootcode written to ad0
rescue-bsd# dd if=/boot/zfsboot of=/dev/ad0s1 count=1 bs=512
rescue-bsd# dd if=/boot/zfsboot of=/dev/ad0s1b skip=1 seek=1024 bs=512


Then re-import the pool with the same options used during its creation and create the datasets for the base filesystem (I'm using the same layout as described on the FreeBSD wiki):

rescue-bsd# zpool import -o cachefile=/tmp/zpool.cache -o altroot=/mnt zroot
rescue-bsd# zfs set checksum=fletcher4 zroot
rescue-bsd# zfs set mountpoint=none zroot
rescue-bsd# zfs create -o mountpoint=/ zroot/rootfs
rescue-bsd# zpool set bootfs=zroot/rootfs zroot
rescue-bsd# zfs create -o compression=on -o exec=on -o setuid=off zroot/rootfs/tmp
rescue-bsd# chmod 1777 /mnt/tmp/
rescue-bsd# zfs create zroot/rootfs/usr
rescue-bsd# zfs create zroot/rootfs/usr/home
rescue-bsd# ln -s /usr/home /mnt/home
rescue-bsd# zfs create -o compression=lzjb -o setuid=off zroot/rootfs/usr/ports
rescue-bsd# zfs create -o compression=off -o exec=off -o setuid=off zroot/rootfs/usr/ports/distfiles
rescue-bsd# zfs create -o compression=off -o exec=off -o setuid=off zroot/rootfs/usr/ports/packages
rescue-bsd# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/rootfs/usr/src
rescue-bsd# zfs create zroot/rootfs/var
rescue-bsd# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/rootfs/var/crash
rescue-bsd# zfs create -o exec=off -o setuid=off zroot/rootfs/var/db
rescue-bsd# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/rootfs/var/db/pkg
rescue-bsd# zfs create -o exec=off -o setuid=off zroot/rootfs/var/empty
rescue-bsd# zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/rootfs/var/log
rescue-bsd# zfs create -o compression=gzip -o exec=off -o setuid=off zroot/rootfs/var/mail
rescue-bsd# zfs create -o exec=off -o setuid=off zroot/rootfs/var/run
rescue-bsd# zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/rootfs/var/tmp
rescue-bsd# chmod 1777 /mnt/var/tmp


Note that I've activated compression on some datasets as in the FreeBSD wiki, but on a low-end box with little CPU power, I advise to turn it off.

Now let's copy our data to the ZFS partition.

rescue-bsd# mount /dev/ad0s1b /media
rescue-bsd# cd /media
rescue-bsd# find . | cpio -dump /mnt/
rescue-bsd# cd
rescue-bsd# umount /media
rescue-bsd# zfs set readonly=on zroot/rootfs/var/empty


Note that cpio(1) does not handle file flags set by chflags(8). Your system will be able to boot, but some security seatbelt won't be here until you perform an installworld.

Let's create the swap partition instead of the transient UFS filesystem:

rescue-bsd# gpart show ad0s1
=> 0 488397105 ad0s1 BSD (233G)
0 486297600 1 freebsd-zfs (232G)
486297600 2099505 2 freebsd-ufs (1.0G)

rescue-bsd# gpart delete -i 2 ad0s1
ad0s1b deleted
rescue-bsd# gpart add -t freebsd-swap ad0s1
ad0s1b added
rescue-bsd# gpart show ad0s1
=> 0 488397105 ad0s1 BSD (233G)
0 486297600 1 freebsd-zfs (232G)
486297600 2099505 2 freebsd-swap (1.0G)


Now we need to configure the system to be able to boot from ZFS:


rescue-bsd# echo 'zfs_load="YES"' > /mnt/boot/loader.conf
rescue-bsd# echo 'vfs.root.mountfrom="zfs:zroot/rootfs"' >> /mnt/boot/loader.conf
rescue-bsd# cp /tmp/zpool.cache /mnt/boot/zfs/
rescue-bsd# vi /mnt/etc/fstab
[...]
rescue-bsd# cat /mnt/etc/fstab
# Device Mountpoint FStype Options Dump Pass#
/dev/ad0s1b none swap sw 0 0
proc /proc procfs rw 0 0


Et voilĂ ! You can reboot your server (don't forget to deactivate netbooting from the OVH web interface).

Thursday, June 30, 2011

Configuring FreeBSD with dual console

This post is short as I intend to use it more as a reminder than a full-fledged article.

As an introduction for the un-educated reader, here is a simple paste of the boot(8) manpage:


By default, a three-stage bootstrap is employed, and control is automati-
cally passed from the boot blocks (bootstrap stages one and two) to a
separate third-stage bootstrap program, loader(8). This third stage pro-
vides more sophisticated control over the booting process than it is pos-
sible to achieve in the boot blocks, which are constrained by occupying
limited fixed space on a given disk or slice.


In summary: boot0 -> boot2 -> loader -> kernel

The first stage (boot0) cannot be configured, as the code as to fit in 512 bytes. It will simply use the default system console (the screen).

However the following things can be configured more or less independently:
- boot2 (stage 2);
- loader(8) (stage 3);
- the kernel;
- login(8).

Configuring boot2



boot2 is configured through /boot.config. This file contains the flags documented in boot(8), as though they were given on the boot2 prompt. Therefore if you want to see boot2 output on both your screen and the serial console, you have to put "-D" in it.


shell# cat /boot.config
-D


Configuring loader(8)



loader(8) is configured through /boot/loader.conf. The console variable can be set either "vidconsole", "comconsole" or "vidconsole,comconsole" to have both "comconsole,vidconsole" works too, we will see the difference later)


shell# grep ^console /boot/loader.conf
console="vidconsole,comconsole"


Configuring the kernel



/boot/loader.conf also contains variables that will set kenv variables, which will define the kernel behaviour. See this comment in /boot/defaults/loader.conf:


##############################################################
### Kernel settings ########################################
##############################################################

# The following boot_ variables are enabled by setting them to any value.
# Their presence in the kernel environment (see kenv(1)) has the same
# effect as setting the given boot flag (see boot(8)).

#boot_askname="" # -a: Prompt the user for the name of the root device
#boot_cdrom="" # -C: Attempt to mount root file system from CD-ROM
#boot_ddb="" # -d: Instructs the kernel to start in the DDB debugger
#boot_dfltroot="" # -r: Use the statically configured root file system
#boot_gdb="" # -g: Selects gdb-remote mode for the kernel debugger
#boot_multicons="" # -D: Use multiple consoles
#boot_mute="" # -m: Mute the console
#boot_pause="" # -p: Pause after each line during device probing
#boot_serial="" # -h: Use serial console
#boot_single="" # -s: Start system in single-user mode
#boot_verbose="" # -v: Causes extra debugging information to be printed
#init_path="/sbin/init:/sbin/oinit:/sbin/init.bak:/rescue/init:/stand/sysinstall"
# Sets the list of init candidates
#init_shell="/bin/sh" # The shell binary used by init(8).
#init_script="" # Initial script to run by init(8) before chrooting.
#init_chroot="" # Directory for init(8) to chroot into.



So basically, the kernel defaults to use the screen only, but you can override this by setting the boot_multicons variable:


shell# grep ^boot_multicons /boot/loader.conf
boot_multicons="YES"


How the whole stuff works



Actually when you configure one stage, subsequent stages will use the same settings unless configured to do differently. So in the end you just have to configure boot2.

Userland output



Contrary to the other parts, userland boot output can only be sent to one device at time. Even when configured with the above settings, the userland boot output will only appear on screen.

Actually, the kernel will pick the first entry from the console kenv variable to sent userland output to. So if you are not often behind the screen and you prefer to see the userland boot output on the serial console:


shell# grep ^console /boot/loader.conf
console="comconsole,vidconsole"


Configuring login(8)



Not seeing the userland boot output on one console or the other doesn't mean it is unusable. FreeBSD is configured by default to spawn a login: prompt on the screen. You can easily configure it to spawn another one one the serial console, as explained in this chapter on the handbook:


shell# grep ttyu0 /etc/ttys
ttyu0 "/usr/libexec/getty std.9600" dialup on secure



That's all.

Wednesday, June 2, 2010

Debian KVM console on a headless server

At work I use a Debian KVM with an encrypted root filesystem as a workstation (our physical workstations run Windows) running on a headless server. This means that I have to use the QEMU' VNC console to enter the password for the root filesystem very early in the boot process.

Unfortunately VNC is unsecure and anyway QEMU only binds VNC on 127.0.0.1. It would be easy to create an SSH tunnel, but this is administratively prohibited here and it is cumbersome to temporarily modify sshd_config(5) each time. So I tried a Netfilter DNAT rule as a workaround but Linux' network stack contains a very annoying line of code which checks that packets destined 127.0.0.1 comes from 127.0.0.1 as well. If you see some logs like this, you have probably been biten by it too:
Jun  2 18:14:20 srv kernel: martian destination 127.0.0.1 from 10.1.2.2, dev br0


So I gave up VNC and configured the KVM domain to use the serial port like any other headless server.

Supposedly your VM is already running so we will make the changes here first. There are three things to be told to use the serial console, which are in time-order:

  • the bootloader (GRUB here);

  • the kernel;

  • init(8) for the login prompt.



On Debian, the first two things can be done easily through /etc/default/grub.
# Bootloader part.
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=9600 --unit=0 --word=8 --parity=no --stop=1"

# Kernel command-line ("quiet" has no matter in our business):
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,9600n8 quiet"


Then regen the grub.cfg:
# upgrade-grub


If you do not use Debian, here is the relevant part of the generated /boot/grub/grub.cfg:
serial --speed=9600 --unit=0 --word=8 --parity=no --stop=1
if terminal_input serial ; then true ; else
# For backward compatibility with versions of terminal.mod that don't
# understand terminal_input
terminal serial
fi
if terminal_output serial ; then true ; else
# For backward compatibility with versions of terminal.mod that don't
# understand terminal_output
terminal serial
fi

menuentry "Linux 2.6.32-trunk-amd64" {
insmod ext2
set root='(hd0,1)'
search --no-floppy --fs-uuid --set 9245a9e3-8ea5-4170-a19b-17d10051c107
echo Loading Linux 2.6.32-trunk-amd64 ...
linux /vmlinuz-2.6.32-trunk-amd64 root=/dev/mapper/vg0-root ro console=tty0 console=ttyS0,9600n8 quiet
echo Loading initial ramdisk ...
initrd /initrd.img-2.6.32-trunk-amd64
}



Regarding the login prompt on serial console, edit /etc/inittab:
T0:23:respawn:/sbin/getty -L ttyS0 9600 vt100



Now your VM is configured, let's configure your KVM domain. Dump the configuration of your vm, and change the <serial> and <console> part to use a PTY (you can choose an arbitrary PTY, /dev/pts/24 here, as it seems to be redefined each time the VM is started). Other interfaces are possible, like TCP, pipe, stdio... (see the libvirt domain XML format) but I chose PTY because it can be easily attached using screen(1) and cannot be easily snooped:
# virsh dumpxml mykvm > mykvm.xml
# vi mykvm.xml
<serial type='pty'>
<source path='/dev/pts/24'/>
<target port='0'/>
</serial>
<console type='pty' tty='/dev/pts/24'>
<source path='/dev/pts/24'/>
<target port='0'/>
</console>


Then stop your VM, redefine your KVM domain and restart it:
# virsh shutdown mykvm      # or run shutdown(8) inside the VM
# virsh undefine mykvm
# virsh define mykvm.xml
# virsh start mykvm


You can attach the console using:
# virsh console mykvm

To detach, use Ctrl + $


If you attach quickly enough after starting it, you will even see the Grub menu!

Monday, May 24, 2010

Quick n' Dirty Linux WPA-PSK Wireless AP

On saturday evening, there was a party at home. One of the guests poured her glass of champagne on the ADSL modem lended by my ISP. Undoubtly it wasn't champagne-proof. I have about a week to wait before getting a new one. Fortunately I have my 3G connection but only one person can use it at a given time... and we are two at home. So I have created a very quick and dirty access-point to share my 3G connection. This post has two purpose: record how I did it and show how it eventually turned out to be really easy. Ironically, it was more difficult to configure a new wireless connection on Windows XP than creating the AP.

I am assuming you are running mac80211 wireless stack, which is standard from recent kernels 2.6.30+). You will need hostapd and ISC's DHCPd.

First set up your wireless interface as you would with any other wired interface:

# ifconfig wlan0 inet 192.168.10.1 netmask 0xffffff00 up


Next, configure /etc/hostapd/hostapd.conf:

driver=nl80211
interface=wlan0
channel=13
ssid=3g2wifi
auth_algs=1
wpa=1
wpa_passphrase=XXXXXXXX


And run it:

# hostapd /etc/hostapd/hostapd.conf


From now on, you can configure a smartphone or another computer with this wireless network and see DHCP traffic when using tcpdump -ni wlan0. You may enable debugging with hostapd's -d flag if it doesn't work.

Next step is to provide connectivity to the Internet through 3G (interface ppp0). We have to masquerade the computers behind the access-point (I assume there are no filtering rules):

# iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -o ppp0 -j MASQUERADE
# sysctl net.ipv4.conf.all.forwarding=1


Here you can manually configure the IPv4 layer on another computer, setting the DNS servers to the ones provided by you 3G provider, and it should work.

But the sugar on the cake would be to have a DHCP server, to minimize manual configuration. This is straightforward. Here is my /etc/dhcp3/dhcpd.conf (note that I used Google's open DNS resolvers for example purpose):

subnet 192.168.10.0 netmask 255.255.255.0 {
range 192.168.10.20 192.168.10.30;
option routers 192.168.10.1;
option domain-name-servers 4.4.4.4, 4.4.8.8;
}


Start the DHCP server:

# dhcpd3 -cf /etc/dhcp3/dhcpd.conf wlan0


And voila! Now good luck if you have to configure a Windows 7 computer to use this connection :).

Thursday, May 20, 2010

column.sh - columizator

It has been a long time since I've last written on this blog. I am indeed very busy by my work so I don't have much time to write. Nonetheless a positive aspect of this is that I happen to write some tools for me to alleviate some tasks.

I am pretty sure that many if not all of you have already been annoyed by the output format such as vmstat(8), iostat(8), ... They are great commands because they produce very valuable information but they are often very difficult to read, especially on busy servers when you need them most, because of the misalignment. We cannot blame them because it is the Unix way of doing thing: do one thing and do it well. It's not their job to pretty print the output. The "column" script exposed here will realign the output for you.

This command acts like a standard Unix command: it may be used alone or as a filter. It takes one mandatory argument, namely a keyword that will be used to recognize the «caption» line.


jlehen@warg:~$ ./column -h
Pretty-print columns, for iostat or vmstat.
OS: All.

Usage: column [-g]

Keyword is used to identify the caption line. This may be an
awk/nawk regex (thus don't forget to escape "/").
Options:
-g Change the output to what I called "giant mode".
This is useful for iostat with an awful lot of disks.


Let's take an example. Here is a classical vmstat(8) output on a busy Solaris server:

05:26:32 kthr memory page disk faults cpu
05:26:32 r b w swap free re mf pi po fr de sr m0 m1 m2 m1 in sy cs us sy id
05:26:32 0 29 0 139019936 26258128 4776 17739 38816 8 0 0 0 4 4 4 0 29969 129465 33184 21 25 54
05:26:33 0 40 0 138993232 26235120 5240 19492 47250 39 31 0 0 7 7 7 0 30665 133160 34489 23 25 51
05:26:34 0 28 0 139002648 26230848 4878 18717 36828 16 16 0 0 3 3 3 0 31368 144148 35850 22 24 54
05:26:35 1 26 0 138978136 26211456 4046 14760 30778 8 8 0 0 4 4 4 0 26295 158433 27845 20 20 61
05:26:36 0 29 0 138980472 26212968 4469 15525 33361 23 16 0 0 4 4 4 0 25826 118810 27653 21 19 60
05:26:37 0 22 0 139004424 26227824 4208 16734 29367 8 8 0 0 4 4 4 0 24879 122903 28154 23 19 58
05:26:38 0 18 0 139010608 26232864 3958 15324 27984 0 0 0 0 2 2 2 0 22422 105347 23563 17 17 66
05:26:39 0 25 0 139022192 26237584 4076 15124 27770 31 31 0 0 2 2 2 0 25046 115101 26467 17 17 65
05:26:40 0 21 0 139035416 26248720 5205 15215 38873 0 0 0 0 1 1 1 0 26961 131668 28818 22 18 60
05:26:41 0 17 0 139014056 26239552 5132 17257 31959 0 0 0 0 0 0 0 0 23780 116586 25043 18 19 63
05:26:42 0 13 0 139028280 26250352 3661 17215 17719 23 16 0 0 3 3 3 0 22048 110885 23717 15 17 67
05:26:43 0 11 0 138993472 26220968 3796 19241 16468 0 0 0 0 0 0 0 0 21640 115744 23173 18 15 66
05:26:44 0 12 0 138972904 26199192 2607 14008 15327 8 8 0 0 2 2 2 0 20428 120489 21064 18 15 67
05:26:45 0 11 0 138973008 26194376 2122 9228 21017 23 23 0 0 5 5 5 0 24543 123981 25423 20 16 64
05:26:46 2 12 0 138977112 26194960 1929 12010 23333 0 0 0 0 1 1 1 0 32145 144927 34596 26 16 59
05:26:47 1 21 0 139032632 26240256 2361 11192 32759 16 16 0 0 2 2 2 0 36018 204602 39325 27 21 51
05:26:49 1 25 0 139043840 26245952 2628 9895 35478 24 24 0 0 3 3 3 0 32645 136756 34903 26 20 54
05:26:50 0 23 0 139043552 26246304 1942 9952 21395 0 0 0 0 0 0 0 0 24948 121252 25918 21 14 66
05:26:51 0 19 0 139053752 26249176 1949 7929 26141 0 0 0 0 6 7 7 0 24490 113321 26220 19 14 67


This is difficult to read because of the large amount of memory and swap. Let's look at the same output now filtered through column (keyword used to match caption line is "swap"):

05:26:32 kthr memory page disk faults cpu
05:26:32 r b w swap free re mf pi po fr de sr m0 m1 m2 m1 in sy cs us sy id
05:26:32 0 29 0 139019936 26258128 4776 17739 38816 8 0 0 0 4 4 4 0 29969 129465 33184 21 25 54
05:26:33 0 40 0 138993232 26235120 5240 19492 47250 39 31 0 0 7 7 7 0 30665 133160 34489 23 25 51
05:26:34 0 28 0 139002648 26230848 4878 18717 36828 16 16 0 0 3 3 3 0 31368 144148 35850 22 24 54
05:26:35 1 26 0 138978136 26211456 4046 14760 30778 8 8 0 0 4 4 4 0 26295 158433 27845 20 20 61
05:26:36 0 29 0 138980472 26212968 4469 15525 33361 23 16 0 0 4 4 4 0 25826 118810 27653 21 19 60
05:26:37 0 22 0 139004424 26227824 4208 16734 29367 8 8 0 0 4 4 4 0 24879 122903 28154 23 19 58
05:26:38 0 18 0 139010608 26232864 3958 15324 27984 0 0 0 0 2 2 2 0 22422 105347 23563 17 17 66
05:26:39 0 25 0 139022192 26237584 4076 15124 27770 31 31 0 0 2 2 2 0 25046 115101 26467 17 17 65
05:26:40 0 21 0 139035416 26248720 5205 15215 38873 0 0 0 0 1 1 1 0 26961 131668 28818 22 18 60
05:26:41 0 17 0 139014056 26239552 5132 17257 31959 0 0 0 0 0 0 0 0 23780 116586 25043 18 19 63
05:26:42 0 13 0 139028280 26250352 3661 17215 17719 23 16 0 0 3 3 3 0 22048 110885 23717 15 17 67
05:26:43 0 11 0 138993472 26220968 3796 19241 16468 0 0 0 0 0 0 0 0 21640 115744 23173 18 15 66
05:26:44 0 12 0 138972904 26199192 2607 14008 15327 8 8 0 0 2 2 2 0 20428 120489 21064 18 15 67
05:26:45 0 11 0 138973008 26194376 2122 9228 21017 23 23 0 0 5 5 5 0 24543 123981 25423 20 16 64
05:26:46 2 12 0 138977112 26194960 1929 12010 23333 0 0 0 0 1 1 1 0 32145 144927 34596 26 16 59
05:26:47 1 21 0 139032632 26240256 2361 11192 32759 16 16 0 0 2 2 2 0 36018 204602 39325 27 21 51
05:26:49 1 25 0 139043840 26245952 2628 9895 35478 24 24 0 0 3 3 3 0 32645 136756 34903 26 20 54
05:26:50 0 23 0 139043552 26246304 1942 9952 21395 0 0 0 0 0 0 0 0 24948 121252 25918 21 14 66
05:26:51 0 19 0 139053752 26249176 1949 7929 26141 0 0 0 0 6 7 7 0 24490 113321 26220 19 14 67



The -g option is meant to be used when the output is giant. For instance, when a system has a lot of disks, the iostat(8) command prints so much lines that the caption line is swept out of your terminal immediately..

Let's take a sample output of iostat(8). I won't show a lot of disks because it is worthless, but keep in mind than when there are tenths or hundreds of disks, the caption line no longer appear on the screen:

extended device statistics
device r/s w/s kr/s kw/s wait actv svc_t %w %b
md0 5.0 8.1 36.3 155.5 0.0 0.1 6.7 0 6
md1 3.0 8.1 20.2 155.5 0.0 0.1 7.3 0 6
md2 2.0 8.1 16.2 155.5 0.0 0.1 5.1 0 4
md10 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md11 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md12 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md50 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md51 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md52 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md60 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md61 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
md62 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
ramdisk1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
sd0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
[...]


Filtering with column -g actv (using "device" would be useless as the very first line also contains it):

extended device statistics
device:md0 r/s:5.0 w/s:8.1 kr/s:36.3 kw/s:155.5 wait:0.0 actv:0.1 svc_t:6.7 %w:0 %b:6
device:md1 r/s:3.0 w/s:8.1 kr/s:20.2 kw/s:155.5 wait:0.0 actv:0.1 svc_t:7.3 %w:0 %b:6
device:md2 r/s:2.0 w/s:8.1 kr/s:16.2 kw/s:155.5 wait:0.0 actv:0.1 svc_t:5.1 %w:0 %b:4
device:md10 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md11 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md12 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md50 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md51 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md52 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md60 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md61 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:md62 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:ramdisk1 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
device:sd0 r/s:0.0 w/s:0.0 kr/s:0.0 kw/s:0.0 wait:0.0 actv:0.0 svc_t:0.0 %w:0 %b:0
[...]


This nice script is available here.
Beware that is used /bin/sh. If you want to use it on Solaris, use /bin/ksh instead. I have yet to find a shebang line that will work on *BSD, Linux and Solaris. If you know one, contact me on < jeremie le-hen org > (up to you to but the "@" and "." where you think it best fits...).

Wednesday, February 18, 2009

Bash prompt trick: cheap emulation of tcsh's pwd trailing component

I started Unix with Linux where Bash has always been the default shell, at least in my own reckoning. Hence I've always been using Bash as I never found the necessity nor the motivation to really switch to another shell.

When FreeBSD turned to be my favorite operating system, I had a chance to fiddle with tcsh. One thing I really liked was the "%c" prompt sequence, a.k.a. "trailing component of the current working directory". Basically, "%c02" in "/home/jlh/a/b" expands to something like "/<2>/a/b". Since then, I've always missed this sequence in Bash, as "\w" often expands to something far too long and "\W" is often not enough.

Therefore I implemented a function to mimic this behaviour in tcsh. I devised it to only use builtins so as to be cheap (read inexpensive, not lousy). Indeed, I think it would be really stupid to spawn many processes each time Enter is hit in my shell. Here it is:

traildir() {
local n=$1 dir=$2
local sl tildedir homelen shifted traildir

sl=/
tildedir=${dir#$HOME}
if ! [[ "$tildedir" == "$dir" ]]; then
tildedir="~$tildedir"
sl=
fi

set -- ${HOME//\// }
homelen=$#

shifted=0
set -- ${tildedir//\// }
if [[ $# -gt $n ]]; then
shifted=$(($# - $n))
shift $shifted
[[ -z "$sl" ]] && shifted=$(($shifted + $homelen - 1))
traildir="<$shifted>/$@"
else
traildir="$sl$@"
fi

echo ${traildir// /\/}
}


Then, you just have to set, for example:

TRAILPWD=2
export PS1='\u@\h:$(traildir $TRAILPWD $PWD) \#\$'


And that's it! Ok it still forks a process each time you hit enter, but it's not possible to achieve it less expensively with Bash anyway. Note that the example above uses a neat trick: if you need more trailing components in your prompt for your current task, just set TRAILPWD to the desired value.


Addendum on 2009/06/22


A smaller but less resilient version of this prompt is:

export PS1='\u@\h:$(set -- ${PWD//\// }; n=$(($# - $TRAILPWD)); s=; [[ $n -le 0 ]] && s=/ || shift $n; d="$@"; echo $s${d// /\/}) \#\$'


One advantage of this one is that is only relies on an environment variable, thus is inherited across forks. This is useful for instance if you call bash through sudo(8) and you wish to use the same prompt. This is impossible with the function-based prompt.

Wednesday, December 10, 2008

Get rid of Vodafone Mobile Connect Card driver for Linux

Back in August, I explained why I switched from shipped Xandros to the Debian Eee blend. I bought my EeePC with a 3G USB modem from Huawei:

Bus 001 Device 003: ID 12d1:1003 Huawei Technologies Co., Ltd. E220 HSDPA Modem / E270 HSDPA/HSUPA Modem

Thanks to Vodafone Mobile Connect Card driver for Linux, I could use it very easily. But compared to the Asus dialer, it is way slow! Between the time I click on the icon and the time I am connected, there is nearly two minutes because EeePC is not powerful enough to process quickly this bloated Python software.

While inspecting this program, you will notice that it uses wvdial behind the scene:

jlh 11626 11579 0 15:22 pts/5 00:00:00 /opt/vmc/bin/wvdial --config /tmp/VMC_uJJ0r/VMCYy5LXzwvdial.conf connect

Just copy the configuration file to wvdial.conf:

[Dialer Defaults]

Phone = *99***1#
Username = slsfr
Password = slsfr
Stupid Mode = 1
Dial Command = ATDT
Check Def Route = on
Dial Attempts = 3

[Dialer connect]

Modem = /dev/ttyUSB0
Baud = 460800
Init2 = ATZ
Init3 = ATQ0 V1 E0 S0=0 &C1 &D2 +FCLASS=0
Init4 = AT+CGDCONT=1,"IP","slsfr"
ISDN = 0
Modem Type = Analog Modem


But running wvdial with this configuration will only work if you have already entered the PIN code with Vodafone Mobile Connect Card driver for Linux. I needed to find a way to automate this. I quickly googled for a solution without luck, so I devised a way to do it myself.

What I basically did was to use strace(1) on Vodafone Mobile Connect Card driver for Linux just when I entered my PIN code. Then I looked for the PIN code itself in the output file and watched about so I could figure out the dialog with the USB modem to unlock it.

Let's say my PIN code is 5678. First check where my PIN code is used:

jlh@r2d2:~$ grep -n 5678 strace.vodafone
68172:12333 write(29, "AT+CPIN=5678\r\n"..., 14) = 14


Line 68172. The line is sent on file descriptor 29. Let's find where this file descriptor is opened before line 68172. It may be recycled during the execution of the program, so only take the last one:

jlh@r2d2:~$ cat -n strace.vodafone | head -n 68172 | grep 'open.*= 29' | tail -n 1
66643 12333 open("/dev/ttyUSB1", O_RDWR|O_NOCTTY|O_NONBLOCK|O_LARGEFILE) = 29


And where it's closed:

jlh@r2d2:~$ cat -n strace.vodafone | tail -n +66643 | grep 'close(29'
74856 12333 close(29) = 0


We just have to get read(2) and write(2) calls on file descriptor 29 between line 66643 and line 74856:

jlh@r2d2:~$ sed -n '66643,74856{ /\(read\|write\)(29/p }' strace.vodafone
12333 write(29, "AT+CSQ\r\n"..., 8) = 8
12333 read(29, "AT+CSQ\r\r\n+CME ERROR: SIM PIN requ"..., 8192) = 39
12333 write(29, "ATZ\r\n"..., 5) = 5
12333 read(29, "ATZ\r"..., 8192) = 4
12333 read(29, "\r\nOK\r\n"..., 8192) = 6
12333 write(29, "ATE0\r\n"..., 6) = 6
12333 read(29, "ATE0\r\r\nOK\r\n"..., 8192) = 11
12333 write(29, "AT+CPIN?\r\n"..., 10) = 10
12333 read(29, "\r\n+CPIN: SIM PIN\r\n\r\nOK\r\n"..., 8192) = 24
12333 read(29, "\r\n^BOOT:12633134,0,0,0,64\r\n"..., 8192) = 27
12333 write(29, "AT+CPIN=5678\r\n"..., 14) = 14
12333 read(29, "\r\nOK\r\n"..., 8192) = 6
12333 read(29, "\r\n^SIMST:1\r\n"..., 8192) = 12
12333 read(29, "\r\n^SRVST:2\r\n"..., 8192) = 12
12333 read(29, "\r\n^RSSI:13\r\n"..., 8192) = 12
12333 read(29, "\r\n^BOOT:12633134,0,0,0,64\r\n"..., 8192) = 27
...


The important thing is to send the PIN, so we will just stop after sending the PIN (if it doesn't work, you can still try to add a few lines). Let's translate this to a chat(8) script:

ECHO ON
TIMEOUT 1
'' AT+CSQ
'SIM PIN required' ATZ
OK ATE0
OK AT+CPIN?
OK AT+CGSN
OK AT+CPIN=5678


Then, just before running wvdial with the stolen configuration file, just use:

jlh@r2d2:~$ /usr/sbin/chat -f pin.chat < /dev/ttyUSB0 > /dev/ttyUSB0
[...]


Note that if you try to rerun this chat(8) script, it won't work because the modem won't return what is expected. Given there is no way to implement branches in chat(8), I used a very small timeout so it will exit very quickly if the modem is already unlocked.