Out of the loop for a while...

I'm heading into hospital for an operation on my neck later today and recuperation may take a while. Hence this blog is not going to be updated for quite a few weeks.

Meanwhile, while I'm away, please feel free to add comments on the kind of kernel/system/BIOS related issues you're interested in to help me focus blog entries in 2010...

UPDATE:

Operation was a success - now got a titanium disc in my neck and I've regained sense of touch in feet and hands. I can now walk again without the help of a walking stick. I'm keeping my keyboarding/desk activity to absolute minimum. Need to do a whole load of exercises to get neck more mobile and my immune system is working overtime which makes me really physically tired.

itop - look at top interrupt activity

When I want to see interrupt activity on a Linux box I normally use the following rune:

watch -n 1 cat /proc/interrupts

..which simply dumps out the interrupt information every second.  Another way is to use itop, which has the benefit that it outputs the interrupt rate which is lacking from my rune above.

To install, use:

sudo apt-get install itop

and to run, use:

itop

The output is refreshed every second, and outputs something like the following:

INT                NAME          RATE             MAX
  0 [PIC-edge      time]   154 Ints/s     (max:   154)
 14 [PIC-edge      ata_]    16 Ints/s     (max:    16)
 18 [PIC-fasteoi   ehci]     4 Ints/s     (max:     4)
 22 [PIC-fasteoi   ohci]     2 Ints/s     (max:     2)
 29 [MSI-edge      i915]    38 Ints/s     (max:    38)
 30 [MSI-edge      iwl3]     6 Ints/s     (max:     6)

Unfortunately itop does truncate the interrupt names, but I'm not so worried about this - I generally want to see very quickly if a machine is suffering from interrupt saturation or is missing interrupts, which I can get from itop easily.

To see all interrupts, run itop with:

itop -a

And to run for a number of iterations, run with the -n flag, e.g.

itop -n 10

Kernel Early Printk Messages

I've been messing around with the earlyprintk kernel options to allow me to get some form of debug out before the console drivers start later on in the kernel init phase. The earlyprintk kernel option supports debug output via the VGA, serial port and USB debug port.

The USB debug port is of interest - most modern systems seem to provide a debug port capability which allows one to send debug over USB to another machine. To check if your USB controller has this capability, use:

sudo lspci -vvv | grep "Debug port"

and look for a string such as "Capabilities: [58] Debug port: BAR=1 offset=00a0". You may have more than one of these on your system, so beware you use the correct one.

One selects this mode of earlyprintk debugging using:

earlyprintk=dbgp

for the default first port, or select the Nth debug enabled port using:

earlyprintk=dbgpN

One also needs to build a kernel with the following config option enabled:

CONFIG_EARLY_PRINTK_DBGP=y

On my debug set-up I used a NET20DC-USB Hi-Speed USB 2.0 Host-to-Host Debug Device connecting the target machine and a host with which I capture the USB debug using /dev/ttyUSB0 with minicom. So that I won't bore you with the details, this is all explained in the kernel documentation in Documentation/x86/earlyprintk.txt

As it was, I needed to tweak the earlyprintk driver to put in some delays in the EHCI probing and reset code to get it working on my fairly fast target laptop.
My experience with this approach wasn't great - I had to plug/unplug the debug device quite frequently for the earlyprintk EHCI reset and probe to work. Also, the EHCI USB driver initialisation later on in the kernel initialisation hung which wasn't useful.

Overall, I may have had problems with the host/target and/or the NET20DC-USB host-to-host device, but it did allow me to get some debug out, be it rather unreliably.

Probably an easier way to get debug out is just using the boot option:

earlyprintk=vga

however this has the problem that the messages are eventually overwritten by the real console.

Finally, for anyone with old legacy serial ports on their machine (which is quite unlikely nowadays with newer hardware), one can use:

earlyprintk=serial,ttySn,baudrate

where ttySn is the nth tty serial port.

One can also append the ",keep" option to not disable the earlyprintk once the real console is up and running.

So, with earlyprintk, there is some chance of being able to get some form of debug out to a device to allow one to debug kernel problems that occur early in the initialisation phase.

Results from the "Linux needs to have..." poll

Before I kick off another poll, here are the totally unscientific results of my "Linux needs to have..." poll:

Working Audio	16
Reliable suspend/resume	11
Better video support	10
Better battery life	9
Better wifi	8
Faster boot times	6
Nothing more - it rocks	2

It's based on a very small sample (even though we get thousands of hits a month), so I'd hate to draw many conclusions from the votes. However, it looks like audio is giving a lot of users grief and this needs fixing.  Making Suspend/Resume better does not surprise me as I see a lot of weird ACPI and driver suspend/resume issues on the hardware I work with.   It's interesting to see that faster boot times are low on the wish list,  but I does not surprise me that video and wifi support are higher priority (since they need to work!) compared to getting to boot faster.

And to finish with, not many users think Linux does not need improving - this spurs us on to get subsystems fixed to make it rock!