Presentation at Reversing the Broadcom NetExtreme's firmware

Sun 21 November 2010 by guillaume

I was giving a talk in October during last session. The presentation focuses on the roadmap taken to reverse engineer the Broadcom Ethernet NetExtreme firmware family: building a firmware debugger, instrumentation tools, to finally develop a customized network card firmware.

NetExtreme family cards are the standard range of PCI Ethernet cards from Broadcom. Broadcom released part of their soft specifications (inner workings, memory mappings, device register definitions...). However those specifications are incomplete and the firmware is distributed as a binary blob.

Given publicly available documentation (specifications, Linux open-source driver) and free open-source tools, I have built a set of tools to instrument the network card firmware. Those tools provided me a way to debug in real-time the MIPS CPU of the network card, as well as doing some advanced instrumentation on the firmware code (execution flow tracing, memory-accesses logging...). I have also reverse engineered the format of the EEPROM where firmware code is kept and the bootstrap process of the device leading to firmware execution. This way it is possible to develop a custom firmware code, flash the device and get execution on the CPU of the network card.

The main interest is developing a rootkit which will be residing inside the network card. A network card rootkit offers some very interesting features:

  • A very stealthy communication end-point over the Ethernet link. It can intercept and forge network frames without the operating system knowing about it.
  • A physical system memory access using DMA over the PCI link, leading to OS corruption.
  • No trace of the rootkit on the operating system, as it is being hidden inside the NIC.

The network card natively needs to perform DMA accesses, so that network frames can be exchanged between the driver and the device. From the firmware point of view, everything is operated using special dedicated device registers, some of them being non-documented. An attacker would then be able to communicate remotely with the rootkit in the network card and get access to the underlying operating system thanks to DMA.

Slides of my presentation are available here.

Here is a little demo of what can be done once you develop your own network card firmware: