Making sense of PCIe ASPM

PCI Express based serial linked devices can be managed by Active State Power Management (ASPM) to extend battery life on mobile devices such as laptops and netbooks.  ASPM is a power management protocol that allows an operating system's power management to place the link physical layer into a low power mode and it has the ability to instruct other devices on the link to go into a lower power mode too.
The plus side is that we save power with ASPM, however, it will introduce some latency as the bus needs time to be woken up when in a low power state.
The PCIe specification (version 2.0) defines two power modes:

• L0s, which set low power mode in in direction on the link (usually from the physical link layer controller downstream)
• L1, which sets low power mode in both directions on the link, however there is greater wakeup latency.

Section 5.2.9.3 "IA-PC Boot Architecture Flags" of the version 4.0a of the ACPI specification for the PCIe ASPM Controls (bit 4 of IAPC_BOOT_ARCH field of the FADT) states:  "If set, indicates to OSPM that it must not enable OSPM ASPM control on this platform."

In the document ASPM implementation on Windows Vista the ASPM Controls are described as follows: "If the BIOS indicates that the platform does not properly support ASPM, the operating system disables ASPM. This can be conveyed in the ACPI fixed ACPI description table (FADT) table IAPC_BOOT_ARCH flags by setting the PCIe ASPM Controls bit to 1".

Unfortunately we may have PCIe devices that are unreliable under ASPM control, hence the ASPM Control state bit is set in the ACPI FADT to inform the operating system to not control ASPM.

The Linux ASPM driver implements the nitty-gritty details of ASPM - by default it reads the ASPM config from the PCI configuration space. The ASPM config is contained in one of the Capabilities List items, the head of this is pointed to by CapPntr (offset 0x34 in the Configuration Header).

Currently Linux forcibly clears ASPM state on the device if the Control state bit is set. A recent patch by Matthew Garrett refers to the "PCI Express In Depth for Windows Vista and Beyond" in which it seems to imply that Windows won't use the PCIe features (such as ASPM) unless the full control is granted via the _OSC control.    Garrett's patch only clears ASPM state if we have _OSC control and the ASPM Control state bit is set, and his test show that this seems to save several Watts on a ThinkPad X220.  We're currently crowd-sourcing a test of this patch here.

Linux also provides some ASPM driver kernel parameters to allow some level of tweakability.  The following kernel parameters can be used:

• "pcie_aspm=off" - disables ASPM
• "pcie_aspm=default" - use default firmware configuration as set in the PCI Express Capabalities list item with ID 0x10
• "pcie_aspm=performance"  - disables ASPM and clock power management
• "pcie_aspm=powersave" - highest power saving mode, enable ASPM and clock power management

"pcie_aspm=off" has seemed to help some users with some PCIe devices that case Linux hang at boot time.  I deeply suspect that the IACP_BOOT_ARCH flag may be telling Linux to do this but it's being ignored.
I measured "pcie_aspm=powersave" on one of my netbooks and I believe I get a small amount of power saving, powertop seemed to indicate about 0.2 Watts being saved, but this could be within the margin of error in my measurements.
Anyhow, if you want to tinker with more power savings, maybe overriding the firmware configured defaults with "pcie_aspm=powersave" may help. It's worth a try.

UPDATE:

Thanks to Alex Hung for pointing out some mistakes in my original blog post.