|
Description
|
libumem has a couple of problems which can lead to over-allocation of storage:
1. the sbrk backend, when hit by multiple threads at the same time, can
overallocate by a large margin.
2. The cache sizes chosen by umem can be sub-optimal, since common allocation
sizes (i.e. 4096 bytes), because of the malloc buffer tags, get rounded up (i.e.
to 8192)
The first problem requires some tweaking in the sbrk backend; the second, a
combination of tweaking the sizes, and letting the consumer further tweak them.
---
By writing two new dcmds, ::umem_malloc_info and ::umem_malloc_dist, we can gain
insight into the wastage experienced by the application:
> ::umem_malloc_info
BUFSZ MAXMAL BUFMAL AVG_MAL MALLOCED OVERHD
...
4096 4088 63 2835 178593 44.2%
8192 8184 123018 4172 513244710 96.1%
12288 12280 7 9127 63892 34.5%
16384 16376 0 0 0 0.0%
if we look at the details of the allocation sizes for the three high-overhead caches,
we see:
> 00096008::umem_malloc_info -b 8
CACHE BUFSZ MAXMAL BUFMALLC AVG_MAL MALLOCED OVERHD
00096008 4096 4088 63 2835 178593 44.2%
malloc size ------------------ Distribution ------------------ count
2681-2856 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 53
2857-3032 | 1
3033-3208 | 0
3209-3384 |@ 2
3385-3560 | 0
3561-3736 |@@@ 4
3737-3912 | 0
3913-4088 |@@ 3
> 00096588::umem_malloc_info -b 8
CACHE BUFSZ MAXMAL BUFMALLC AVG_MAL MALLOCED OVERHD
00096588 8192 8184 123018 4172 513244710 96.1%
malloc size ------------------ Distribution ------------------ count
4089-4600 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 123010
4601-5112 | 3
5113-5624 | 0
5625-6136 | 0
6137-6648 | 1
6649-7160 | 0
7161-7672 | 0
7673-8184 | 4
> 00096b08::umem_malloc_info -b 8
CACHE BUFSZ MAXMAL BUFMALLC AVG_MAL MALLOCED OVERHD
00096b08 12288 12280 7 9127 63892 34.5%
malloc size ------------------ Distribution ------------------ count
8185-8696 |@@@@@@@@@@@@@@ 2
8697-9208 |@@@@@@@@@@@@@@@@@@@@@ 3
9209-9720 | 0
9721-10232 |@@@@@@@ 1
10233-10744 |@@@@@@@ 1
10745-11256 | 0
11257-11768 | 0
11769-12280 | 0
In all cases, the vast majority of buffers are just slightly in the cache
in question. By consulting a table of allocation sizes and wastage, the following
three cache sizes will be added to give more coverage:
2304, 4544, 9216
In addition, we'll include the new dcmds so that customers can investigate the
overhead themselves, and add undocumented hooks to muck with the allocation size
table, so that it can be tweaked in the field.
|
