Booting Ubuntu Karmic with grub2 and EFI

The past few days I've been tinkering with the TianoCore EFI and trying to get Ubuntu Karmic to boot with the EFI enabled grub2. To make debugging easier, I did all this inside a virtualised QEMU environment. A lot of this earlier hackery was blogged about in a previous article.

In trying to get this to work I hit a couple of issues; figuring out workarounds took some thought and time. Some of it is still "work in progress", but at least now I know what needs to be fixed.

Below is a video (grabbed using the recordmydesktop tool) showing Ubuntu Karmic booting:






First you see the TianoCore EFI starting and then loading grub2 - this then loads a very slimmed down grub.cfg and eventually boots the system.

I hit several issues; firstly the kernel oops'd when trying to call virt_efi_get_next_variable() - for some reason calling this EFI runtime service support function in the EFI is causing the kernel to oops. Other EFI support functions seem to work correctly, so my current belief is that the bug is not in the kernel EFI driver, but this needs a little more poking around to verify this assumption.

Second issue is that loading some grub2 modules and using some fundamental grub2 commands such as 'set' cause the EFI get caught in a loop. It took me a while to corner this - eventually I cut down the default grub config file down to a really minimal version which could at least boot the kernel! Anyhow, this grub2 issue is first on my list of bugs to fix.

Debugging these issues is a little tricky, but using QEMU helps as I can dump out debug from grub2 over a pseudo serial port which I can capture and view in real-time. Secondly, debugging the kernel using a serial console in QEMU (using the console=ttyS0 boot option) allows me to capture all kernel boot messages (including the offending kernel oops).

I still need to look into ACPI support and how to get the video initialised to get more than bog-standard VGA resolution, but these are currently lower priorities.

Without QEMU and Tristan Gingold's port of the TianoCore EFI to QEMU, this work would have been much harder.

QEMU and EFI BIOS hackery

Earlier this week I blogged about QEMU and EFI BIOS. Trying to debug a problem with grub2-efi-ia32 has given me a few little headaches but I'm finding ways to work around them all.

The first issue is getting a system installed with an EFI BIOS. My quick hack was to create a 4GB QEMU qcow2 disk image and then inside this create a small EFI FAT12 boot partition using fdisk - (partition type 0xef in the 1st primary partition). I then installed Ubuntu Karmic Desktop with ext4 and swap in primary partitions 2 and 3 by booting with the conventional BIOS. I then installed grub2-efi-ia32 in the EFI boot partition and then booted QEMU using the TianoCore EFI BIOS that has been ported to QEMU.

One problem is that the EFI BIOS does not scroll the screen, hence all output when it reaches the end of the screen just keeps over writing the last line, making debugging with printf() style prints nearly impossible. Then I found that the BIOS emits characters over a serial port, which QEMU can emulate. Unfortunately, the output contains VT control characters to do cursor positioning and pretty console colours, which makes reading the output a little painful. So I hacked up a simple tool to take the output from QEMU and strip out the VT control chars to make the text easier to read.

Now QEMU boot line is as follows:

qemu -bios bios-efi.bin -m 1024 karmic-efi-qcow2.img -serial stdio 2>&1 | ./parse-output

..and this dumps the output from the BIOS and grub2 to stdout in a more readable form.

The parse-output tool is a little hacky - but does the job. For reference, I've put it in my debug repository here.

Grub2 allows one to enable some level of debugging output by issuing the following command:

set debug=all

..which gives me some idea of what's working or broken at a fairly low enough level before I start attaching gdb. Fortunately debugging using gdb has been fairly well documented here - I just now need to shoe-horn in small patch to allow me to attach gdb to grub2 from outside QEMU - but that's for another debug session...

Upgrading to ext4

Ext4 support has been available as an installation option in Ubuntu Jaunty 9.04 and will be default in Ubuntu Karmic 9.10. I've been using ext4 now for quite a while on my servers (since the alpha releases of Jaunty) as it provides me with some extra benefits:

* fsck'ing a 500GB partition is so much faster - huge improvement!,
* removing entire kernel source trees takes a few seconds (very handy!),
* kernel builds are a little quicker (I've not quantified how much),
* block I/O on large files is quicker.

I've not experienced any problems with ext4; I use it daily to grind out kernels, and the filesystem gets a lot of exercise.

For those with existing ext2 or ext3 based file systems one can upgrade to ext4 using some relatively straight forward steps. Obviously please make sure you do this on an unmounted filesystem! One way of doing this is to boot your machine using a LiveCD and making sure the HDD is not mounted before proceeding...

Upgrading from ext2 to ext3, for example on /dev/sda1, do:

tune2fs -j /dev/sda1

..this basically enables the journal. One can then mount this as an ext3 filesystem.


Upgrading from ext3 to ext4, for example on /dev/sda5, do:

tune2fs -O extents,uninit_bg,dir_index /dev/sda5

Note that this renders the filesystem unmountable as an ext3 filesystem. You have been warned!

After this YOU MUST RUN fsck to fix some tune2fs modified filesystem structures:

e2fsck -fpDC0 /dev/sda5

New files created on this ext4 filesystem will use the extents format, however existing files will not be in extents format, so you won't see much of a speed improvement with existing files.

If you boot off the newly converted ext4 filesystem you need to re-install an ext4 capable version of grub using grub-install. The ext4 support for grub was introduced into Jaunty, thanks to a patch from Quentin Godfroy, and an inode tweak by me, and some help from Colin Watson. Maybe I should write about the fun and games I had in debugging this one day.. :-)

Any how, you may not get any immediate improvement from an ext3 to ext4 upgrade as existing files need to be re-written to get the full advantage of the ext4 extents. But over time with a lot of file activity you should get some improvement. Alternatively, back your data up, re-format to ext4 and restore the data and you get immediate ext4 goodness.

For more information about ext4, I recommend checking out the ext4 wiki.



GRUB 2 now default for new installations of Ubuntu

The default boot loader for Karmic Koala (Ubuntu 9.10) will be grub2, as announced by Colin Watson on Monday this week.

The existing boot loader (grub 0.97) will not be changed or upgraded on previous installations as changing the boot loader can be inherently risky.

If you would like to check if your BIOS works with grub2 I suggest following the instructions on the Grub2 testing wiki page.