Using raw devices with VirtualBox run as user

Today I needed to run a virtual machine on my laptop. It is not very powerful so I wanted to avoid as much overhead as possible: skipping the host's VFS/filesystem layer is fairly easy, you just have to give access to a raw partition the your virtualization software instead of a file. After all, both are just seen as a big array of bytes. My hard drive is under LVM, so I created a dedicated logical volume that I wanted to be writable by me given I planned to run VirtualBox as a user.

I knew I could do this with udev(8) but skimming through the documentation and fumbling its rules would have been too long for my available time, so I tried Google with no luck and finally asked on IRC, where I found that someone already did this for the same reason.

The configuration line is quite easy:

root@r2d2# cat /etc/udev/rules.d/99-my.rules
SUBSYSTEM=="block",KERNEL=="dm-*",ACTION=="add|change",ENV{DM_NAME}=="*-vbox*",GROUP="jlh"

This rules basically tells that for any add or change of a block device named "dm-*" and matching "*-vbox*", change its group to "jlh" (note that == and = are different, as in many programming languages). One interesting thing to note is that ENV{DM_NAME}=="*-vbox*" is an helper environment variable that is set by udev(8) standards rules. Those stand in /lib/udev/rules/ on Debian and udev(8) merges the content of this directory with the standard configuration directory /etc/udev/rules.d/. Rules are applied by filename order, so be careful to be the last one. Initially I used "90-my.rules" but there is a rule in "91-permissions.rules" that overrode mine. You can easily debug by running udevd --debug, although the output is quite verbose.

The next step is to create a VMDK file for VirtualBox that will point to the raw device and then attach is to your VM's storage controller. This is quite well documented in the manual (Using a raw host hard disk from a guest).

Basically:

jlh@r2d2$ VBoxManage internalcommands createrawvmdk \
    -filename VirtualBox VMs/vm1/data.vmdk \
    -rawdisk /dev/mapper/vg0-vbox_vm1
jlh@r2d2$ VBoxManage showvminfo vm1 | grep 'Storage Controller Name'
Storage Controller Name (0):            IDE Controller
jlh@r2d2$ VBoxManage storageattach vm1 --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium /home/jlh/VirtualBox\ VMs/FreeBSD/data.vmdtk

Your mileage may vary if, for example, you have a different storage controller name, different port or device. The full "showvminfo" output will tell you which slot is available. Another solution is to add another storage controller, although VirtualBox will not permit you to have multiple IDE controller. You can add a S-ATA controller which allows you to plug up to 30 devices:

jlh@r2d2$ VBoxManage storagectl vm1 --name "SATA Controller" --add sata --controller IntelAHCI --bootable on
jlh@r2d2$ VBoxManage storageattach vm1 --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium /home/jlh/VirtualBox\ VMs/boot.vmdk
jlh@r2d2$ VBoxManage storageattach vm1 --storagectl "IDE Controller" --port 1 --device 0 --type hdd --medium /home/jlh/VirtualBox\ VMs/root.vmdk

VitualBox and VboxManage are pretty well documented.

Debian KVM console on a headless server

At work I use a Debian KVM with an encrypted root filesystem as a workstation (our physical workstations run Windows) running on a headless server. This means that I have to use the QEMU' VNC console to enter the password for the root filesystem very early in the boot process.

Unfortunately VNC is unsecure and anyway QEMU only binds VNC on 127.0.0.1. It would be easy to create an SSH tunnel, but this is administratively prohibited here and it is cumbersome to temporarily modify sshd_config(5) each time. So I tried a Netfilter DNAT rule as a workaround but Linux' network stack contains a very annoying line of code which checks that packets destined 127.0.0.1 comes from 127.0.0.1 as well. If you see some logs like this, you have probably been biten by it too:

Jun  2 18:14:20 srv kernel: martian destination 127.0.0.1 from 10.1.2.2, dev br0

So I gave up VNC and configured the KVM domain to use the serial port like any other headless server.

Supposedly your VM is already running so we will make the changes here first. There are three things to be told to use the serial console, which are in time-order:

• the bootloader (GRUB here);


• the kernel;


• init(8) for the login prompt.

On Debian, the first two things can be done easily through /etc/default/grub.

# Bootloader part.
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=9600 --unit=0 --word=8 --parity=no --stop=1"

# Kernel command-line ("quiet" has no matter in our business):
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,9600n8 quiet"

Then regen the grub.cfg:

# upgrade-grub

If you do not use Debian, here is the relevant part of the generated /boot/grub/grub.cfg:

serial --speed=9600 --unit=0 --word=8 --parity=no --stop=1
if terminal_input serial ; then true ; else
  # For backward compatibility with versions of terminal.mod that don't
  # understand terminal_input
  terminal serial
fi
if terminal_output serial ; then true ; else
  # For backward compatibility with versions of terminal.mod that don't
  # understand terminal_output
  terminal serial
fi

menuentry "Linux 2.6.32-trunk-amd64" {
        insmod ext2
        set root='(hd0,1)'
        search --no-floppy --fs-uuid --set 9245a9e3-8ea5-4170-a19b-17d10051c107
        echo    Loading Linux 2.6.32-trunk-amd64 ...
        linux   /vmlinuz-2.6.32-trunk-amd64 root=/dev/mapper/vg0-root ro  console=tty0 console=ttyS0,9600n8 quiet
        echo    Loading initial ramdisk ...
        initrd  /initrd.img-2.6.32-trunk-amd64
}

Regarding the login prompt on serial console, edit /etc/inittab:

T0:23:respawn:/sbin/getty -L ttyS0 9600 vt100

Now your VM is configured, let's configure your KVM domain. Dump the configuration of your vm, and change the <serial> and <console> part to use a PTY (you can choose an arbitrary PTY, /dev/pts/24 here, as it seems to be redefined each time the VM is started). Other interfaces are possible, like TCP, pipe, stdio... (see the libvirt domain XML format) but I chose PTY because it can be easily attached using screen(1) and cannot be easily snooped:

# virsh dumpxml mykvm > mykvm.xml
# vi mykvm.xml
   <serial type='pty'>
     <source path='/dev/pts/24'/>
     <target port='0'/>
   </serial>
   <console type='pty' tty='/dev/pts/24'>
     <source path='/dev/pts/24'/>
     <target port='0'/>
   </console>

Then stop your VM, redefine your KVM domain and restart it:

# virsh shutdown mykvm      # or run shutdown(8) inside the VM
# virsh undefine mykvm
# virsh define mykvm.xml
# virsh start mykvm

You can attach the console using:

# virsh console mykvm

To detach, use Ctrl + $


If you attach quickly enough after starting it, you will even see the Grub menu!

Quick n' Dirty Linux WPA-PSK Wireless AP

On saturday evening, there was a party at home. One of the guests poured her glass of champagne on the ADSL modem lended by my ISP. Undoubtly it wasn't champagne-proof. I have about a week to wait before getting a new one. Fortunately I have my 3G connection but only one person can use it at a given time... and we are two at home. So I have created a very quick and dirty access-point to share my 3G connection. This post has two purpose: record how I did it and show how it eventually turned out to be really easy. Ironically, it was more difficult to configure a new wireless connection on Windows XP than creating the AP.

I am assuming you are running mac80211 wireless stack, which is standard from recent kernels 2.6.30+). You will need hostapd and ISC's DHCPd.

First set up your wireless interface as you would with any other wired interface:

# ifconfig wlan0 inet 192.168.10.1 netmask 0xffffff00 up


Next, configure /etc/hostapd/hostapd.conf:

driver=nl80211
interface=wlan0
channel=13
ssid=3g2wifi
auth_algs=1
wpa=1
wpa_passphrase=XXXXXXXX


And run it:

# hostapd /etc/hostapd/hostapd.conf


From now on, you can configure a smartphone or another computer with this wireless network and see DHCP traffic when using tcpdump -ni wlan0. You may enable debugging with hostapd's -d flag if it doesn't work.

Next step is to provide connectivity to the Internet through 3G (interface ppp0). We have to masquerade the computers behind the access-point (I assume there are no filtering rules):

# iptables -t nat -A POSTROUTING -s 192.168.10.0/24 -o ppp0 -j MASQUERADE
# sysctl net.ipv4.conf.all.forwarding=1


Here you can manually configure the IPv4 layer on another computer, setting the DNS servers to the ones provided by you 3G provider, and it should work.

But the sugar on the cake would be to have a DHCP server, to minimize manual configuration. This is straightforward. Here is my /etc/dhcp3/dhcpd.conf (note that I used Google's open DNS resolvers for example purpose):

subnet 192.168.10.0 netmask 255.255.255.0 {
range 192.168.10.20 192.168.10.30;
option routers 192.168.10.1;
option domain-name-servers 4.4.4.4, 4.4.8.8;
}


Start the DHCP server:

# dhcpd3 -cf /etc/dhcp3/dhcpd.conf wlan0


And voila! Now good luck if you have to configure a Windows 7 computer to use this connection :).

Get rid of Vodafone Mobile Connect Card driver for Linux

Back in August, I explained why I switched from shipped Xandros to the Debian Eee blend. I bought my EeePC with a 3G USB modem from Huawei:

Bus 001 Device 003: ID 12d1:1003 Huawei Technologies Co., Ltd. E220 HSDPA Modem / E270 HSDPA/HSUPA Modem

Thanks to Vodafone Mobile Connect Card driver for Linux, I could use it very easily. But compared to the Asus dialer, it is way slow! Between the time I click on the icon and the time I am connected, there is nearly two minutes because EeePC is not powerful enough to process quickly this bloated Python software.

While inspecting this program, you will notice that it uses wvdial behind the scene:

jlh 11626 11579 0 15:22 pts/5 00:00:00 /opt/vmc/bin/wvdial --config /tmp/VMC_uJJ0r/VMCYy5LXzwvdial.conf connect

Just copy the configuration file to wvdial.conf:

[Dialer Defaults]

Phone = *99***1#
Username = slsfr
Password = slsfr
Stupid Mode = 1
Dial Command = ATDT
Check Def Route = on
Dial Attempts = 3

[Dialer connect]

Modem = /dev/ttyUSB0
Baud = 460800
Init2 = ATZ
Init3 = ATQ0 V1 E0 S0=0 &C1 &D2 +FCLASS=0
Init4 = AT+CGDCONT=1,"IP","slsfr"
ISDN = 0
Modem Type = Analog Modem


But running wvdial with this configuration will only work if you have already entered the PIN code with Vodafone Mobile Connect Card driver for Linux. I needed to find a way to automate this. I quickly googled for a solution without luck, so I devised a way to do it myself.

What I basically did was to use strace(1) on Vodafone Mobile Connect Card driver for Linux just when I entered my PIN code. Then I looked for the PIN code itself in the output file and watched about so I could figure out the dialog with the USB modem to unlock it.

Let's say my PIN code is 5678. First check where my PIN code is used:

jlh@r2d2:~$ grep -n 5678 strace.vodafone
68172:12333 write(29, "AT+CPIN=5678\r\n"..., 14) = 14


Line 68172. The line is sent on file descriptor 29. Let's find where this file descriptor is opened before line 68172. It may be recycled during the execution of the program, so only take the last one:

jlh@r2d2:~$ cat -n strace.vodafone | head -n 68172 | grep 'open.*= 29' | tail -n 1
66643 12333 open("/dev/ttyUSB1", O_RDWR|O_NOCTTY|O_NONBLOCK|O_LARGEFILE) = 29


And where it's closed:

jlh@r2d2:~$ cat -n strace.vodafone | tail -n +66643 | grep 'close(29'
74856 12333 close(29) = 0


We just have to get read(2) and write(2) calls on file descriptor 29 between line 66643 and line 74856:

jlh@r2d2:~$ sed -n '66643,74856{ /\(read\|write\)(29/p }' strace.vodafone
12333 write(29, "AT+CSQ\r\n"..., 8) = 8
12333 read(29, "AT+CSQ\r\r\n+CME ERROR: SIM PIN requ"..., 8192) = 39
12333 write(29, "ATZ\r\n"..., 5) = 5
12333 read(29, "ATZ\r"..., 8192) = 4
12333 read(29, "\r\nOK\r\n"..., 8192) = 6
12333 write(29, "ATE0\r\n"..., 6) = 6
12333 read(29, "ATE0\r\r\nOK\r\n"..., 8192) = 11
12333 write(29, "AT+CPIN?\r\n"..., 10) = 10
12333 read(29, "\r\n+CPIN: SIM PIN\r\n\r\nOK\r\n"..., 8192) = 24
12333 read(29, "\r\n^BOOT:12633134,0,0,0,64\r\n"..., 8192) = 27
12333 write(29, "AT+CPIN=5678\r\n"..., 14) = 14
12333 read(29, "\r\nOK\r\n"..., 8192) = 6
12333 read(29, "\r\n^SIMST:1\r\n"..., 8192) = 12
12333 read(29, "\r\n^SRVST:2\r\n"..., 8192) = 12
12333 read(29, "\r\n^RSSI:13\r\n"..., 8192) = 12
12333 read(29, "\r\n^BOOT:12633134,0,0,0,64\r\n"..., 8192) = 27
...


The important thing is to send the PIN, so we will just stop after sending the PIN (if it doesn't work, you can still try to add a few lines). Let's translate this to a chat(8) script:

ECHO ON
TIMEOUT 1
'' AT+CSQ
'SIM PIN required' ATZ
OK ATE0
OK AT+CPIN?
OK AT+CGSN
OK AT+CPIN=5678


Then, just before running wvdial with the stolen configuration file, just use:

jlh@r2d2:~$ /usr/sbin/chat -f pin.chat < /dev/ttyUSB0 > /dev/ttyUSB0
[...]


Note that if you try to rerun this chat(8) script, it won't work because the modem won't return what is expected. Given there is no way to implement branches in chat(8), I used a very small timeout so it will exit very quickly if the modem is already unlocked.

Debian on Asus EeePC 701 with Huawei USB modem from SFR or Vodafone

A few month ago, I bought a neat bundle from SFR, a french mobile operator, containing Asus EeePC 701 and a subscription to Internet through 2G/3G/3G+. The connection is achieved thanks to an "E220 HSDPA Modem" USB key from Huawei Technologies. All this stuff has been working very well out of the box with the pre-installed Xandros-based distribution (based on Debian).

All I wanted from this netbook was to be able to surf the web and open terminals without wasting time administer the beast. And well, standard EeePC distribution achieves this very well, and use OpenOffice as a bonus. Of course, I was lacking some stuff like gcc, mplayer, screen... That's why I harvested a few unofficial package repositories to cram my sources.list(5). There was obviously some conflicts between the repositories, but I really didn't care (although usually I'm very keen to make my package manager happy): I had a handful of configuration files backup'd on a USB key and in case of unrecoverable failure within the package system, I just had to restore the original state by pressing F9 at startup (EeePC is shipped with a cunning disk setup: two partitions, the first one containing the original system and the second empty one being mounted on top of the former using unionfs, so restoring the system basically means blanking the second partition).

But as time went on, one thing was more and more upsetting me: no package updates from Asus. As you may already know, this EeePC is also shipped out-of-the-box with remote root exploit (through Samba)... This was very annoying for me because I sometimes connect to other boxes using SSH, so one could hack my EeePC to steal my passwords or perform even more subtle things. So I turned off everything I could because the kernel provided by Xandros doesn't contain IPTables. But honestly, I was still worry about security.

I finally decided to spare some time to install an other, more up to date, distribution when I noticed that I couldn't use Firefox 3 because most of the required libraries were not available. A friend of mine had tried Ubuntu EEE or EEE Ubuntu, whatever. At first, I thought it was a good choice because it could fit both the low administration and up-to-date-ness requirements. But he quickly told me that Ubuntu was far more too memory hungry. Moreover, I don't like these kind of bloated distributions; they somewhat remind me Windows where everything is done behind the scene without giving me any choice unless I really dig deep to understand how things work together. So I forgot Ubuntu and kept on with Xandros

Then I read that Asus was working with Debian in order to maybe replace Xandros on EeePC some day. This caught all my attention as this implied that Debian EeePC support should be very good. What finally decided me to give Debian Eee a shot, despite my disappointing experience with Debian on my girlfriend's laptop last year, was this post from the Debian Eee PC Team, which looked quite encouraging. Additionally, Debian is one of the cleanest distribution; or rather I should say this is one of the less messy ;-) (hey, it's Linux!).

I fetched the Debian Eee's WPA Installer and spread it on a USB key. I could install Debian flawlessly through my WPA/TKIP router. I went for a 256 MB swap partition and all the remaining space as one big partition, and asked for a desktop installation, hoping to find again the regretted Xandros' usability.

And I am very happy. Given this is based on Debian testing, all packages are fairly up to date: Firefox 3 is here! This is a great improvement because it can achieve true full screen like Opera; it's damn important because EeePC 701's screen is really small for web surfing. All devices seems to be supported, although I haven't tested all of them; some shortcut keys are working (contrast keys for example), but volume keys don't. But honestly it's not a big deal compared to what I won and hopefully it will be resolved soon.

The "hardest" task was to make the Huawey modem USB key work. The current kernel (2.6.25...something) is supposed to support it, but anyway you have to somehow manage to enter the PIN code because there is a SIM card in it. Fortunately, the Vodafone Mobile Connect Card driver for Linux (wow, what a name!) handle it perfectly: just beware to download the i386 installer, which is not available as a package at time of writing. You just have run it and tell that your user must belong to the "vmc" group. Then don't forget to logout so as to be in the "vmc" group effectively and run the "vodaphone-mobile-connect-card-driver-for-linux" (I'm not kidding). Edit the profile, and change username, password and APN host to "slsfr" and the DNS servers to "172.20.2.10" and "172.20.2.39" as noted in this french forum post. And voila! You can connect to Internet over 3G+ and even read the SMS and directory stored on the SIM card!

Ok, it's not a package and it spreads some files but... it works! And if you ever want to remove it, you could still follow the installation script to know what as been copied (/etc/udev/rules/ and /usr/bin mainly if I recall correctly). The graphical interface is heavy and I would have preferred a neat command-line tool, but I won't complain more. In open-source, if you want it, just code it! :-)

In summary, if you're fed up with your Xandros, go for Debian/Eee! (It's hard to say when you are a BSD guy ;p.)

Quick HOWTO for building Xen 3.2 on Debian/Ubuntu

I finally carried out "make world" with Xen 3.2 on Debian/sid after much struggle.

First, contrary to xen-3.1.0-src.tgz, xen-3.2.0.tar.gz doesn't come along with the linux-2.6-xen-sparse/ and patches/ directories which allow to build a «xen-infied» kernel from a vanilla kernel source. Thus it is impossible to use make world XEN_LINUX_SOURCE=tarball.

By default, make world will use Mercurial to pull down (or «clone» in Mercurial vocabulary) the xenified kernel from Xensource's Mercurial repository. Unfortunately, it seems that the current Mercurial version shipped with Debian/Ubuntu is outdated and cannot be used out-of-the-box. Nonetheless, it is possible to fetch the xenified kernel manually.

From my understanding, it is necessary to "make prep-kernels" in order to create the kernel build directory. Indeed if you put your .config file directory into the kernel tree itself, the kernel's build system will complain about the lack of cleanliness and will ask you to run "make mrproper". This is baffling but it appears that whenever the kernel is asked to store the object files in a separate directory (namely build-linux-2.6.18-xen_x86_32/), it makes sure you didn't create your .config file in the wrong directory. I suppose this is a safeguard.

So I devised with the following process to build Xen 3.2.

root# mkdir build
root# wget http://bits.xensource.com/oss-xen/release/3.2.0/xen-3.2.0.tar.gz
root# tar xzf xen-3.2.0.tar.gz
# Download the xenifid kernel tree manually, but NOT in xen-3.2.0/
# because the buildconfig/select-repository script would skip it.
# ! xen-3.2.0/ and linux-2.6.18-xen.hg/ must be at the same level !
root# hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg
root# cd xen-3.2.0
root# make prep-kernels
root# cp /boot/config-2.6.18-my build-linux-2.6.18-xen_x86_32/.config
# Using the world target will clean everything first.  Don't use it here.
root# make dist

Quick HOWTO for building Qumranet's KVM

This post really deserves the name scrawl, but I thought sharing my experience in building a Qumranet's KVM snapshot on Linux could helpful for others. Indeed it was not as straightforward as it seemed at first glance and it required some grope and wandering on Google.

You will need KVM (surprising, isn't it?) and Linux kernel sources corresponding to your kernel.

FWIW, KVM snapshots can be downloaded here.

First, let's prepare the kernel tree. This step is only important for people who use packaged kernel (or IOW who don't build their kernel themselves and have extracted the kernel source tree for the sole purpose of building KVM). For others, this step could be avoided because the targets we will use are performed implicitely when building the kernel.

root# tar xzf linux-2.6.23.tar.gz
root# cd linux-2.6.23.tar.gz
root# cp /boot/config-2.6.23 .config
root# make oldconfig prepare scripts
root# cd -

Next we will build KVM. Beware your GCC version! The current major branch is GCC 4 and it is shipped with almost if not all recent Linux distributions. Unfortunately, QEMU (on which KVM's userland is heavily based, not to say they've reused QEMU with little modifications) doesn't build with GCC 4. It requires GCC 3.2 or GCC 3.4 (a thorough explanation is provided in Fabrice Bellard's paper, "QEMU, a fast and portable dynamic translator", USENIX 2005). So you need to install this version of GCC as well. If you are running Debian for instance, this is pretty straightforward :

root# aptitude install gcc-3.4

And you will have a new compiler named gcc-3.4. On other distros, you will have to either find a package of GCC 3 or build it manually as described on Gentoo Wiki.

Finally you will simply have to build KVM with a few special arguments to the configure script:

root# tar xzf kvm-snapshot-20080117.tar.gz
root# cd kvm-snapshot-20080117
root# ./configure --qemu-cc=gcc-3.4 --kerneldir=$OLDPWD/linux-2.6.23 --prefix=/opt/kvm-snapshot-20080117
root# make all install

And this should work. :-)

As you may have noticed, I installed KVM in /opt in order to avoid messing with your tidy package management system.