Reverse Engineering the AM335x Boot ROM

It's probably been eighteen months since I first got my hands a few Beaglebone Black boards, saved from the dumpster. Unfortunately, the boards didn't work right away. Now, this was my first time working with one of these boards, or any single board computer for that matter, so I wasn't sure if the problem was something I was doing, or something wrong with the boards themselves (perhaps why they were headed for the dumpster in the first place). It has taken me quite a long time, and a large amount of effort, to get these boards to actually start booting, but damn it, I did it. And here's what I learned along the way.

NOTE: How to Use the Ghidra XML File

I've included some utilities in this repo, along with an xml file exported from Ghidra which has all the symbols I've obtained so far from reversing. I used this post to export without the actual firmware, to avoid any copyright issues, just in case. If you want to debug the boot ROM yourself, you'll already have the JTAG hooked up, so you can dump the boot ROM (from 0x20000 to 0x2BFFF) yourself.

To load the symbols:

1. Create a new Ghidra project. Import the binary (not the XML) into Ghidra: Use ARMv7 Little Endian, and make sure under Options you set the base address to 0x20000 and you can set the block name to bootrom.
2. Open this binary in CodeBrowser. DO NOT ANALYZE.
3. Go to File > Add program, and select the XML file. Defaults should be fine. You can now go through the reset handler, or jump to main(), or the MMC/SD card boot handler.

The Problem

To begin with, I knew these were custom versions of the standard beaglebone black, so early on I determined that it could be something missing on the board itself, like a board identifier. What I saw when booting a standard SD card formatted with balenaEtcher, was just nothing. I expected the LEDs on the board to start blinking, and I expected that connecting up a UART to USB cable would allow me to see the U-Boot process. However, the UART was quiet. If I removed the SD card, it would output the letter C over and over, which is expected behavior for a UART/serial boot. It was definitely trying to boot, and the SD card was altering this behavior, but I didn't have any more visibility. Most troubleshooting on the web took the U-Boot output as a starting point to diagnose problems. I guess I wasn't going to have that luxury.

I figured at this point that it would be worthwhile to connect up a debug probe. Unfortunately, I didn't have a matching header for the existing footprint come so I made my own.

The beaglebone board has a header with designator P2 which breaks out the JTAG connections. I connected some wires to this up to a female header so that I can talk to it through my J-Link.

Launching into Ozone (the Segger debugger) I configured the J-Link and started by just trying to find the entry point. I had thought that a reset-halt would put me where I needed to be, which was how I came to the (incorrect) assumption that the entry point was 0x2148a, although I certainly noticed that this wasn't consistent. Later, I realized that the AM335x boards don't really play well with the J-Link's reset-halt, so there was actually a delay of maybe a few hundred clock cycles, landing me somewhere inside a boot handler, indeterministically. (I eventually got around this by writing a GEL file for TI's Code Composer Studio which supports J-Link debugging - on reset, the PC register is set to the reset handler, the registers are cleared, and the instruction mode is forced to ARM.)

From a thread on the TI forums (AM335x: TI employees, where can I get the ROM Bootloader source code/symbols?) I picked up a couple debugging symbols: SPI Initialize at 0x231e0, SPI ReadSectors at 0x23230, and 0x24bfa is a routine performing a UART read. That's a nice bit of help I guess. I noticed that the boot failed by ending up in an infinite loop at 0x402f0440, a dead loop. Hmm, quite far off from the rest of the boot ROM, must be in RAM or something. It's probably time to go over to the technical reference manual (TRM)!

Chapter 26 of the TRM contains a ton of information on booting. We get the following view of the boot ROM:

Description:

The architecture of the Public ROM Code is shown in Figure 26-1. It is split into three main layers with a top-down approach: high-level, drivers, and hardware abstraction layer (HAL). One layer communicates with a lower level layer through a unified interface. The high level layer is in charge of the main tasks of the Public ROM Code: watchdog and clocks configuration and main booting routine. The drivers layer implements the logical and communication protocols for any booting device in accordance with the interface specification. Finally the HAL implements the lowest level code for interacting with the hardware infrastructure IPs. End booting devices are attached to device IO pads.

Figure 26-2 illustrates the high level flow for the Public ROM Code booting procedure. On this device the Public ROM Code starts upon completion of the secure startup (performed by the Secure ROM Code). The ROM Code then performs platform configuration and initialization as part of the public start-up procedure. The booting device list is created based on the SYSBOOT pins. A booting device can be a memory booting device (soldered flash memory or temporarily booting device like memory card) or a peripheral interface connected to a host. The main loop of the booting procedure goes through the booting device list and tries to search for an image from the currently selected booting device. This loop is exited if a valid booting image is found and successfully executed or upon watchdog expiration. The image authentication procedure is performed prior to image execution on an HS Device. Failure in authentication procedure leads to branching to a “dead loop” in Secure ROM (waiting for a watchdog reset).

Memory map! Exception vectors! Flow-charts! Lots of information in this section. My job just got a lot easier.

At this point I used the JTAG probe to download the firmware into a couple different files, and started loading things into Ghidra. There didn't seem to be any SVD files or other register mappings available in a convenient format, which is really unfortunate, because that means I need to define the memory regions and registers and everything manually. This was a tedious process, but after a while I had a python script I could use to load symbols into Ghidra for the the AM3358. One less thing to worry about!

Reversing

It seems like the files I have can be mapped as:

| Region | Start Address | Length | | ------------------------ | ------------- | -------- | | Boot ROM (Public) | 0x4002_0000 | 0xBFFF | | Boot ROM (Public, alias) | 0x0002_0000 | 0xBFFF | | SRAM Internal | 0x402F_0400 | 0xFC00 | | L3 OCM0 | 0x4030_0000 | 0x10000 |

It's interesting that the infinite loop at 0x402f_0440 is at the top of the "downloaded image" in the internal SRAM, while the exception vectors are stored elsewhere. Maybe this will be an important hint later...

On reset, the private boot ROM handles security stuff, and branches to 0x2 0000 which contains the reset vectors. The first instruction is a branch to 0x2 08d0 which must be the entry point. It's not a BX instruction so, presumably, we're still in Arm mode at that point.

Reset Handler

This is the first code run, which means it's not exactly a "function" with parameters, so much as a compiled-generated startup script. The first basic block:

	    ldr        r4,[->Peripherals::CM_PER]
	    mov        r0,#0x2c
		ldr        r6,[r4,r0]=>CM_PER.CM_PER_OCMCRAM_CLKCTRL
	    mov        r6,#0x2
	    str        r6,[r4,r0]=>CM_PER.CM_PER_OCMCRAM_CLKCTRL
	    mov        r0,#0x2c
poll:   ldr        r6,[r4,r0]=>CM_PER.CM_PER_OCMCRAM_CLKCTRL
	    cmp        r6,#0x2
	    bne        poll

This block sets the OCMC RAM clock to enabled:

1. Set CM_PER_OCMCRAM_CLKCTRL=0x2
2. Check if register was set; if not, keep polling The CM_PER_OCMCRAM_CLKCTRL register uses bits 0 and 1 for the MODULEMODE field, setting this =0x2 enables the clock to the OCMC RAM.

The next basic block:

	    ldr        r0,[PTR_control_status] 
	    ldr        r0,[r0,#0x0]=>control_status 
	    and        r0,r0,#0x700
	    mov        r0,r0, lsr #0x8
	    cmp        r0,#0x3
	    bne        skip
		ldr        r0,[PTR_control_status] 
	    ldr        r0,[r0,#0x0]=>control_status
	    cpy        r6,r0
	    and        r0,r0,#0x1f
	    cmp        r0,#0x1f
	    bleq       GPMIC_init 
skip:   ...

This block does the following:

1. Check if (control_status & 0x700) >> 8 == 0x3, skip if not
2. Check if control_status & 0x1f == 0x1f, if so, call function GPMC_init after loading control_status into r6

The next block sets up the coprocessor:

		msr        cpsr_c,#0xd3
	    ldr        r4,[->Exceptions::ROM_RESET_VECTOR] 
	    mcr        p15,0x0,r4,cr12,cr0,0x0
	    bl         LAB_00020934
	    bl         LAB_00020938
	    bl         LAB_0002093c
	    bl         LAB_00020940
	    bl         LAB_00020944
	    bl         LAB_00020948
	    bl         LAB_0002094c
	    bl         LAB_00020950
	    mrc        p15,0x0,r0,cr1,cr0,0x0
	    orr        r0,r0,#0x800
	    mcr        p15,0x0,r0,cr1,cr0,0x0
	    b          LAB_000207f0

Operations in this block:

1. Move 11010011b into the CPSR