libdlmalloc

libdlmalloc is a python script designed for use with GDB that can be used to analyse the Doug Lea's allocator, aka dlmalloc. It currently supports dlmalloc 2.8.x versions. Note that some parts can also be used independently GDB, for instance to do offline analysis of some snapshotted heap memory.

libdlmalloc was inspired by other gdb python scripts for analyzing heaps like libtalloc, unmask_jemalloc and libheap. Some basic functionality is almost identical to these projects.

Supported versions

libdlmalloc has been tested predominately on 32-bit and 64-bit Cisco ASA devices which use dlmalloc 2.8.3. It should work with other 2.8.x versions, however due to significant differences it will not work on earlier releases, such as <= 2.7.x.

If you successfully test libdlmalloc on some specific 2.8.x release or some specific device, please let the authors know and we will update the documents.

Installation

The script just requires a relatively modern version of GDB with python3 support. We have primarily tested on python3, so we expect it will break on python2.7 atm.

If you want to use the gdb commands you can use:

    (gdb) source libdlmalloc_28x.py

A bunch of the core logic is broken out into the dl_helper class, which allows you to directly import libdlmalloc and access certain important structures outside of a GDB session. This is useful if you want to analyze offline chunk/heap snapshots.

Usage

Most of the functionality is modelled after the approach in unmask_jemalloc and libtalloc where a separate GDB command is provided. Though we do also use a fair number of switches.

To see a full list of currently supported commands you can use the dlhelp command:

dlhelp

This is the main function to view the available commands. Each of the commands supports the -h option which allows you to obtain more detailed usage instructions.

(gdb) dlhelp
[libdlmalloc] dlmalloc commands for gdb
[libdlmalloc] dlchunk    : show one or more chunks metadata and contents
[libdlmalloc] dlmstate   : print mstate structure information. caches address after first use
[libdlmalloc] dlcallback : register a callback or query/modify callback status
[libdlmalloc] dlhelp     : this help message
[libdlmalloc] NOTE: Pass -h to any of these commands for more extensive usage. Eg: dlchunk -h

Chunk analysis

dlchunk can provide you with a summary of a chunk, or more verbose information of every field. You can also use it to list information about multiple chunks, search chunks, etc. Usage for dlchunk can be seen below:

(gdb) dlchunk -h
[libdlmalloc] usage: dlchunk [-v] [-f] [-x] [-c <count>] <addr>
[libdlmalloc]  <addr>  a dlmalloc chunk header
[libdlmalloc]  -v      use verbose output (multiples for more verbosity)
[libdlmalloc]  -f      use <addr> explicitly, rather than be smart
[libdlmalloc]  -x      hexdump the chunk contents
[libdlmalloc]  -m      max bytes to dump with -x
[libdlmalloc]  -c      number of chunks to print
[libdlmalloc]  -s      search pattern when print chunks
[libdlmalloc]  --depth depth to search inside chunk
[libdlmalloc]  -d      debug and force printing stuff
[libdlmalloc] Flag legend: C=CINUSE, P=PINUSE

Basic output looks like this:

(gdb) dlchunk 0xacff59d0
0xacff59d0 M sz:0x000f8 fl:CP

As you can see you want to give it the address of the actual dlmalloc metadata itself. To get more verbose output you can use -v.

(gdb) dlchunk -v 0xacff59d0
struct malloc_chunk @ 0xacff59d0 {
prev_foot   = 0x8140d4d0
size        = 0xf8 (CINUSE|PINUSE)

You can also list multiple adjacent chunks by using the -c <count> switch.

(gdb) dlchunk -c 2 0xacff59d0
0xacff59d0 M sz:0x000f8 fl:CP
0xacff5ac8 M sz:0x00270 fl:CP
(gdb) dlchunk -v -c 2 0xacff59d0
struct malloc_chunk @ 0xacff59d0 {
prev_foot   = 0x8140d4d0
size        = 0xf8 (CINUSE|PINUSE)
--
struct malloc_chunk @ 0xacff5ac8 {
prev_foot   = 0x8140d4d0
size        = 0x270 (CINUSE|PINUSE)

You can dump the hex contents of a chunk with -x and control how many bytes you want to dump with -m.

(gdb) dlchunk -v -x -m 16 -c 2 0xacff59d0
struct malloc_chunk @ 0xacff59d0 {
prev_foot   = 0x8140d4d0
size        = 0xf8 (CINUSE|PINUSE)
0x10 bytes of chunk data:
0xacff59d8:	0xa11c0123	0x000000cc	0x00000000	0x00000000
--
struct malloc_chunk @ 0xacff5ac8 {
prev_foot   = 0x8140d4d0
size        = 0x270 (CINUSE|PINUSE)
0x10 bytes of chunk data:
0xacff5ad0:	0xa11c0123	0x00000244	0x00000000	0x00000000

You can also search inside the chunks. Let's search 2 chunks for the value 0x00000244, which we see above is only in the second chunk.

(gdb) dlchunk -s 0x00000244 -c 2 0xacff59d0
0xacff59d0 M sz:0x000f8 fl:CP [NO MATCH]
0xacff5ac8 M sz:0x00270 fl:CP [MATCH]

All matches inside the number of chunks searched will be shown. Let's search for 0xa11c01123 which we saw above is present in both chunks:

(gdb) dlchunk -s 0xa11c0123 -c 2 0xacff59d0
0xacff59d0 M sz:0x000f8 fl:CP [MATCH]
0xacff5ac8 M sz:0x00270 fl:CP [MATCH]

dlmstate

The dlmstate command can be used for analyzing the mstate structure used to manage a discrete dlmalloc heap (aka mspace if compiled with MSPACES). You can see the usage of the command with the -h switch.

(gdb) dlmstate -h
[libdlmalloc] usage: dlmstate [-v] [-f] [-x] [-c <count>] <addr>
[libdlmalloc]  <addr>  a mstate struct addr. Optional if mstate cached
[libdlmalloc]  -v      use verbose output (multiples for more verbosity)
[libdlmalloc]  -c      print bin counts
[libdlmalloc]  --depth how deep to count each bin (default 10)
[libdlmalloc]  NOTE: Last defined mstate will be cached for future use

If you know the address holding the mstate, which is usually the first chunk inside of the first malloc asegment, you can pass it to dlmstate:

(gdb) dlmstate 0xa8400008
struct dl_mstate @ 0xa8400008 {
smallmap    = 0b000000000000010000011111111100
treemap     = 0b000000000000000000000000000111
dvsize      = 0x0
topsize     = 0x2ebdf040
least_addr  = 0xa8400000
dv          = 0x0
top         = 0xad020f90
trim_check  = 0x200000
magic       = 0x2900d4d8
smallbin[00] (sz 0x0)   = 0xa840002c, 0xa840002c [EMPTY]
smallbin[01] (sz 0x8)   = 0xa8400034, 0xa8400034 [EMPTY]
smallbin[02] (sz 0x10)  = 0xacbf7ad0, 0xa88647f0
smallbin[03] (sz 0x18)  = 0xa95059b8, 0xa9689a20
smallbin[04] (sz 0x20)  = 0xac79a028, 0xa87206f8
smallbin[05] (sz 0x28)  = 0xacff0120, 0xa948a0f8
smallbin[06] (sz 0x30)  = 0xac4e4af8, 0xacb56878
smallbin[07] (sz 0x38)  = 0xacfe3880, 0xacfe0df0
smallbin[08] (sz 0x40)  = 0xa9509b28, 0xa9509b28
smallbin[09] (sz 0x48)  = 0xa8a1dc80, 0xa8a1dc80
smallbin[10] (sz 0x50)  = 0xac782cb0, 0xac782cb0
smallbin[11] (sz 0x58)  = 0xacbf7a88, 0xacbf7a88 [EMPTY]
smallbin[12] (sz 0x60)  = 0xac782c00, 0xac782c00 [EMPTY]
smallbin[13] (sz 0x68)  = 0xacbf7a78, 0xacbf7a78 [EMPTY]
smallbin[14] (sz 0x70)  = 0xa89b9650, 0xa89b9650 [EMPTY]
smallbin[15] (sz 0x78)  = 0xac789828, 0xac789828 [EMPTY]
smallbin[16] (sz 0x80)  = 0xa89b9738, 0xa94af740
smallbin[17] (sz 0x88)  = 0xac4e5700, 0xac4e5700 [EMPTY]
smallbin[18] (sz 0x90)  = 0xac788030, 0xac788030 [EMPTY]
smallbin[19] (sz 0x98)  = 0xac782bc8, 0xac782bc8 [EMPTY]
smallbin[20] (sz 0xa0)  = 0xa89b9718, 0xa89b9718 [EMPTY]
smallbin[21] (sz 0xa8)  = 0xa8a1dc20, 0xa8a1dc20 [EMPTY]
smallbin[22] (sz 0xb0)  = 0xac782af8, 0xac782af8 [EMPTY]
smallbin[23] (sz 0xb8)  = 0xac789ed0, 0xac789ed0 [EMPTY]
smallbin[24] (sz 0xc0)  = 0xacbf7a20, 0xacbf7a20 [EMPTY]
smallbin[25] (sz 0xc8)  = 0xac789940, 0xac789940 [EMPTY]
smallbin[26] (sz 0xd0)  = 0xac789eb8, 0xac789eb8 [EMPTY]
smallbin[27] (sz 0xd8)  = 0xa94af6e8, 0xa94af6e8 [EMPTY]
smallbin[28] (sz 0xe0)  = 0xacbf78e8, 0xacbf78e8 [EMPTY]
smallbin[29] (sz 0xe8)  = 0xac4e4e68, 0xac4e4e68 [EMPTY]
smallbin[30] (sz 0xf0)  = 0xac4e5780, 0xac4e5780 [EMPTY]
smallbin[31] (sz 0xf8)  = 0xac7880b0, 0xac7880b0 [EMPTY]
treebin[00] (sz 0x180)      = 0xac783cb0
treebin[01] (sz 0x200)      = 0xac789dc0
treebin[02] (sz 0x300)      = 0xa883db48
treebin[03] (sz 0x400)      =        0x0 [EMPTY]
treebin[04] (sz 0x600)      =        0x0 [EMPTY]
treebin[05] (sz 0x800)      =        0x0 [EMPTY]
treebin[06] (sz 0xc00)      =        0x0 [EMPTY]
treebin[07] (sz 0x1000)     =        0x0 [EMPTY]
treebin[08] (sz 0x1800)     =        0x0 [EMPTY]
treebin[09] (sz 0x2000)     =        0x0 [EMPTY]
treebin[10] (sz 0x3000)     =        0x0 [EMPTY]
treebin[11] (sz 0x4000)     =        0x0 [EMPTY]
treebin[12] (sz 0x6000)     =        0x0 [EMPTY]
treebin[13] (sz 0x8000)     =        0x0 [EMPTY]
treebin[14] (sz 0xc000)     =        0x0 [EMPTY]
treebin[15] (sz 0x10000)    =        0x0 [EMPTY]
treebin[16] (sz 0x18000)    =        0x0 [EMPTY]
treebin[17] (sz 0x20000)    =        0x0 [EMPTY]
treebin[18] (sz 0x30000)    =        0x0 [EMPTY]
treebin[19] (sz 0x40000)    =        0x0 [EMPTY]
treebin[20] (sz 0x60000)    =        0x0 [EMPTY]
treebin[21] (sz 0x80000)    =        0x0 [EMPTY]
treebin[22] (sz 0xc0000)    =        0x0 [EMPTY]
treebin[23] (sz 0x100000)   =        0x0 [EMPTY]
treebin[24] (sz 0x180000)   =        0x0 [EMPTY]
treebin[25] (sz 0x200000)   =        0x0 [EMPTY]
treebin[26] (sz 0x300000)   =        0x0 [EMPTY]
treebin[27] (sz 0x400000)   =        0x0 [EMPTY]
treebin[28] (sz 0x600000)   =        0x0 [EMPTY]
treebin[29] (sz 0x800000)   =        0x0 [EMPTY]
treebin[30] (sz 0xc00000)   =        0x0 [EMPTY]
treebin[31] (sz 0xffffffff) =        0x0 [EMPTY]
footprint   = 0x33800000
max_footprint = 0x33800000
mflags      = 0x7
mutex       = 0x0,0x0,0x0,0x0,0xa8400000,
seg = struct malloc_segment @ 0xa84001d4 {
base        = 0xa8400000
size        = 0x33800000
next        = 0x0
sflags      = 0x8

To speed up output on slower devices we cache the last mstate data that we read. So if you just run dlmstate again, you will see the previously dumped output (which of course could be stale).

(gd