Perf tool now in Ubuntu Lucid.

A few months ago I blogged about the perf tool and how useful it is to drill down into kernel and user space applications to analyse performance bottle necks. Well, thanks to Andy Whitcroft, the perf tool is now available in Ubuntu Lucid. Perf needs to be in lock-step with the kernel, which complicated the packaging of this tool.

To install, use:

sudo apt-get install linux-tools

and this installs the perf, perf-stat, perf-top, perf-record, perf-report and perf-list tools.

The author of perf, Ingo Molnar has written some basic instructions on driving perf in the tools/perf/Documentation/example.txt file. They should give one a feel of how to drive these tools.

For example, to examine google-chrome:

perf record google-chrome

... run a test and then exit, and then generate a summary of activity:

perf report

One can examine specific events in the system too. To get a list of available events use:

perf list

For example, to measure CPU cycles, number of instructions, context switches and kmallocs on google-chrome one uses:

perf stat -e cpu-cycles -e instructions -e context-switches -e kmem:kmalloc google-chrome

Performance counter stats for 'google-chrome':

15576472832 cycles # 0.000 M/sec
13466330277 instructions # 0.865 IPC
40791 context-switches # 0.000 M/sec
38602 kmem:kmalloc # 0.000 M/sec

18.062301718 seconds time elapsed

One more quick example, this time we record a call-graph (stack chain/backtrace) of the dd command:

perf record -g dd if=/dev/zero of=/dev/null bs=1M count=4096
and get the call graph using:

# Samples: 12584 # # Overhead Command Shared Object Symbol # ........ ............... ............................... ...... # 95.96% dd [kernel] [k] __clear_user | |--99.69%-- read_zero | vfs_read | sys_read | system_call_fastpath | __read --0.31%-- [...]
1.37% dd [kernel] [k] read_zero 0.87% dd [kernel] [k] _cond_resched | |--50.91%-- read_zero | vfs_read | sys_read | system_call_fastpath | __read
..etc.

I recommend reading Ingo's example.txt and playing around with this tool. It is very powerful and allows one to drill down and examine system performance right down to the instruction level.

Reading the TSC from userspace

Recently I've been poking around looking at some Time Stamp Counter (TSC) anomalies when coming out of suspend. So what is the TSC? It's a 64 bit high resolution tick counter found on X86 processors (since Pentiums) and can be read using the rdtsc instruction.

It is intended to be a fast method of getting a high resolution timer. However it is known to problematic on multi-core and hyperthreaded CPUs - one needs to be locked to one CPU to get reliable results since the TSC may be different on each CPU. It is also known to reset when coming out of resume which means time can look like it goes backwards in a huge jump.

If the CPU speed is changed then the TSC rate can change too. If you have a more recent Intel CPU where the constant_tsc flag is set (see /proc/cpuinfo) then the TSC will run at a constant rate no matter the CPU speed - but this means that benchmarking with the constant TSC may make programs look like they use more CPU cycles than in reality!

Anyhow, getting the 64 bit TSC value is a simple case of using the rdtsc instruction. I've got some example code to do this here with the necessary inlined assembler magic to handle this correctly for 32 and 64 bit builds.

Tweaking the Gnome Window Button Layout

Call me old fashioned, but I kind of like my Gnome Window buttons to be on the right hand side of a Window, rather than on the left. Fortunately this can be easily configured using gconf-editor. Run gconf-editor and select: apps->metacity->general and then select the button_layout configuration setting and change it to:

:minimize,maximize,close

The leading colon sets the button icons on the right of a window and a trailing colon sets the button icons on the left of a window. Easy!



If you don't want to run the GUI tool, use the following rune:

gconftool-2 --type string --set /apps/metacity/general/button_layout :minimize,maximize,close

Atom Z530 identity crisis

Last week I peeked at /proc/cpuinfo on a Atom Z530 netbook and got the following model name information:

model name : Intel(R) Core (TM) CPU Z530 @ 1.60GHz

Is the kernel mistaken? It's not a Core CPU, it's an Atom! In fact the Z530 is mistaken. If one examines page 29 of http://download.intel.com/design/processor/specupdt/319536.pdf you will see errata AAE29:

"AAE29 CPUID Instruction Returns Incorrect Brand String

When a CPUID instruction is executed with EAX = 80000002H, 80000003H and 80000004H on an Intel® Atom(TM) processor, the return value contains the brand string Intel(R) Core(TM)2 CPU when it should have Intel(R) Atom(TM) CPU."

Doh! That's a rather poor mistake in the silicon.

Apparently this affects Intel® Atom(TM) processors Z550, Z540, Z530, Z520, Z515, Z510, and Z500 on 45-nm process technology. It is fixable with a microcode fix, which normally involves getting a BIOS upgrade.

The errata makes interesting reading - especially errata AAE44 and AAE46 - for older kernels I suggest booting with kernel boot option mem=nopentium to work around any bizarre kernel oopses caused by these particular processor bugs. In fact, I recommend this for any Atom processor as these bugs seem to also apply to the Atom Nxxx series to.