GNU ld: Linking .bss into .data to Ensure that Mem Size Equals File Size For Each LOAD Segment (.bss in a PROGBITS Section)
Update: I found another variant. I appended it to the end of the article.
Original post:
Some rudimentary ELF loaders require that p_memsz equals p_filesz for each LOAD segment in order to simplify the loading of the file. For example, the microkernels NOVA and Hedron have this restriction to bootstrap their roottasks (inital process). In this blog post, I’m going to provide a solution how this can be achieved when an ELF is assembled, or better to say linked, by object files generated from a typical higher level language, such as C. Or, in other words, I show you how you can place symbols that usually land in .bss in a section of type SHT_PROGBITS, which is the technical foundation that determines the file size of a LOAD segment.
Background
What is .bss?
Let’s start with the background. .bss, or block started by symbol, refers to a section where “typically only the length of the bss section, but no data is stored”, as Wikipedia says. A typical ELF under Linux, compiled from C source code and dynamically linked against the glibc, shows the following readelf -Wl output (shortened):
Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align ... LOAD 0x002df0 0x0000000000003df0 0x0000000000003df0 0x000224 0x000450 RW 0x1000 ... Section to Segment mapping: Segment Sections... ... 05 .init_array .fini_array .dynamic .got .data .bss ...
We can see that the .bss section is part of a LOAD segment that has read and write rights. Furthermore, .bss is placed at the end of the LOAD segment. Last but not least, we can see that file size and memory sizes do not equal for the LOAD segment. But why is this?
When does the Linker doesn’t include all the (zeroed) memory inside the ELF?
From the section headers, we find that
Section Headers: [Nr] Name Type Address Off Size ES [ 4] .bss NOBITS ffffffff88003000 003004 000204 00 WA 0 0 4096
which shows that the .bss section has the NOBITS type. If a section has the NOBITS type, the linker will use the p_memsz != p_filesz optimization as those symbols are expected to be zero and do not need to be inside the ELF. Hence, an ELF loader needs to allocate enough memory and zeroes it at the given link address of the corresponding LOAD segment. However, if p_memsz == p_filesz and the ELF file is loaded into physical memory, one can map it 1:1 into an address space of a user-space application and each LOAD segment can be mapped with one or multiple pages. No further allocations are required. This benefit is used by NOVA and Hedron, as mentioned earlier, to simplify bootstrapping an initial roottask/init process.
Trivia: More information about the type of a section can be found behind the identifier SHT_NOBITS in the ELF specification. SHT stands for section type.
From GNU ld’s source code, we can find that the .bss section is by default of type NOBITS:
// GNU binutils @ 658ba81aef5 > bfd/elf.c > line 2627
static const struct bfd_elf_special_section special_sections_b[] =
{
{ STRING_COMMA_LEN (".bss"), -2, SHT_NOBITS, SHF_ALLOC + SHF_WRITE },To summarize: The advantage of this optimization is that the file size of the ELF is reduced, but a ELF loader has more work to do.
Which Symbols Land in .bss?
From the default linker script of GNU ld (printable via ld --verbose or written in the source code in ld/scripttempl/elf.sc), we learn that *(COMMON) and *(.bss) symbols land in the .bss output section. Hence, we need to do something with those symbols.
Trivia: COMMON symbols only exist in object files and not in final ELF executables or shared libs. If you want to learn more about their difference, please look here.
Putting .bss into a Section of Type PROGBITS.
Since GNU binutils 2.39, linker scripts allow setting the type of the output section (doc). However, this doesn’t allow switching the type from Update, they replied to me! Check out the link and Variant 3 at the bottom of the page.NOBITS to PROGBITS. I believe this is either a bug or misleading documentation. I try to clarify this and update this blog post accordingly.
Another option is to link *(COMMON) and *(.bss) into .data. It has the same permissions (readable and writeable) but the.data section is of type PROGBITS by default, as the following excerpt from GNU ld’s source shows:
// GNU binutils @ 658ba81aef5 > bfd/elf.c > line 2640
static const struct bfd_elf_special_section special_sections_d[] =
{
{ STRING_COMMA_LEN (".data"), -2, SHT_PROGBITS, SHF_ALLOC + SHF_WRITE },Putting Everything Together in a Minimal Example
This example is also on my GitHub page.
Now, we create a ELF file with the following readelf output where .bss is part of the .data section. We do not see .bss here as we merge it into .data, as you will see in a couple of seconds:
Elf file type is EXEC (Executable file) Entry point 0xffffffff88000000 There are 3 program headers, starting at offset 64 Section Headers: [Nr] Name Type Address Off Size ES ... [ 1] .text PROGBITS 0000000000400000 001000 000020 00 AX 0 0 4096 [ 2] .rodata PROGBITS 0000000000401000 002000 000004 00 A 0 0 4096 [ 3] .data PROGBITS 0000000000402000 003000 000240 00 WA 0 0 4096 ... Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align LOAD 0x001000 0x0000000000400000 0x0000000000400000 0x000020 0x000020 R E 0x1000 LOAD 0x002000 0x0000000000401000 0x0000000000401000 0x000004 0x000004 R 0x1000 LOAD 0x003000 0x0000000000402000 0x0000000000402000 0x000240 0x000240 RW 0x1000 Section to Segment mapping: Segment Sections... 00 .text 01 .rodata 02 .data
Note that file size equals memory size for each section and that all are of type PROGBITS!
Let’s take the following, minimal C-program:
/*
* This is the code for a minimal, freestanding C program that shows how the
* GCC puts certain code constructs into specific sections.
*
* The linker will use the linker file to rename some of those sections
* and place it into LOAD segments.
*/
// GCC will make this a "COMMON" symbol.
char global_buffer_uninitialized[512];
// GCC will place this in the .data section.
char global_buffer_initialized_rw[4] = { 1, 2, 3, 4};
// GCC will place this in the .rodata section.
const char global_buffer_initialized_ro[4] = { 1, 2, 3, 4};
// GCC will place this in the .bss section.
int flag = 0;
// A simple function with no inputs and no return value. This program can be
// executed under Linux but will stuck in an endless loop.
void start() {
// All variables are marked as volatile so that the compiler
// does not discard them.
// This string will land in the .rodata section.
volatile char * msg = "Hello World!\n";
// Values will land on the stack.
volatile int a = 0xdeadbeef;
volatile int b = 0x1337;
volatile int c = a + b;
while (1) {}
__builtin_unreachable();
}We can compile it with $ gcc -ffreestanding -nostdlib -c -o main.o main.c. To link it, we need the following invocation $ ld -M -o main -Tlink.ld main.o and this linker script
ENTRY(start)
/* Program headers. Also called segments. */
PHDRS
{
/* PT_LOAD FLAGS(x): The flags of an ELF program header/segment. Always 32 bit long,
even for 64-bit ELFs. Also called "Segment Permissions" in ELF
specification or "p_flags". Helps loaders of ELF files to set
the right page table bits. */
rx PT_LOAD FLAGS(5); /* 0b101 - read + execute */
ro PT_LOAD FLAGS(4); /* 0b100 - read */
rw PT_LOAD FLAGS(6); /* 0b110 - read + write */
}
SECTIONS {
.text 4M : ALIGN(4K)
{
*(.text .text.*)
} : rx
/* ALIGN(4K): The linker parses this file from top to bottom and automatically increases
link and load addresses. */
.rodata ALIGN(4K) : ALIGN(4K)
{
*(.rodata .rodata.*)
} : ro
/* Section for .data and .bss. i.e., all data that needs read and write permissions. */
.data ALIGN(4K) : ALIGN(4K)
{
*(.data .data.*)
/* We place the .bss section in .data as .bss is of type SHT_NOBITS by default but we
need its symbols to be in a section of type SHT_PROGBITS so that FILESIZE equals
MEMSIZE for each LOAD segment. */
/* The .bss output section of an ELF executable (or shared lib) actually consists of
symbols that are either in the COMMON section or the `.bss` section of object files.
This can also be verified by looking at the standard linker script for Linux
programs. */
*(COMMON)
*(.bss .bss.*)
} : rw
/DISCARD/ :
{
*(.note.*)
*(.comment .comment.*)
*(.eh_frame*)
*(.got .got.*)
}
}I only used the -M flag in above’s GNU ld invocation so that ld prints out verbose information where it puts certain sections into. This helps to check that everything works as expected.
Now, as all symbols that typically land in .bss are part of .data, we ensured that all default zeroed buffers land “as they are” statically allocated in the ELF file. The ELF file is of course larger this way, but it contains all the memory that the program needs.
The example can also be found on my GitHub page.
PS: There is no command line parameter for GNU ld that fulfills the desired goal. Hence, we need a dedicated linker script.
Update: Variant 2
I found another variant. In my subsequent blog post, I dived deeper into the reasons when and why the file-size saving optimization is done for the .bss section. We can use the code from above but with this linker script modification:
/*
* Section .bss is usually of type SHT_NOBITS. However, here I want to guarantee that
* the file size equals the memory size for each LOAD segment. So we make sure that
* .bss is not the last section in the "rw" segment.
*
* More info: https://phip1611.de/blog/how-does-the-file-size-is-smaller-than-mem-size-optimization-work-in-gnu-ld/
*/
.bss ALIGN(4K) : ALIGN(4K)
{
/*
* The .bss output section of an ELF executable (or shared lib) actually consists
* of symbols that are either in the COMMON section or the `.bss` section of
* object files.
*
* This can also be verified by looking at the standard linker script for Linux
* programs.
*/
*(COMMON)
*(.bss .bss.*)
} : rw
.data ALIGN(4K) : ALIGN(4K)
{
*(.data .data.*)
} : rwThe explanation is: The optimization is only done if both conditions are true:
- section type must be
SHT_NOBITS - the section must be the last section of the LOAD segment
With variant 2, the readelf output will look like this instead:
Program Headers: Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align LOAD 0x001000 0x0000000000400000 0x0000000000400000 0x000020 0x000020 R E 0x1000 LOAD 0x002000 0x0000000000401000 0x0000000000401000 0x000004 0x000004 R 0x1000 LOAD 0x003000 0x0000000000402000 0x0000000000402000 0x000240 0x000240 RW 0x1000 Section to Segment mapping: Segment Sections... 00 .text 01 .rodata 02 .bss .data
Notice that the last LOAD segment now contains two sections, and the file size still equals the memory size. .bss is not the last section of the LOAD segment. If we change the order, the optimization will be back.
Variant 3
Update: I added Variant 3. It only works with GNU binutils 2.39 or newer. In your linker script, do:
.bss ALIGN(4K) (TYPE=SHT_PROGBITS) : ALIGN(4K)
{
// Some untyped data is required to that the section type can be overwritten.
// See https://sourceware.org/bugzilla/show_bug.cgi?id=29861
BYTE(1)
*(COMMON)
*(.bss .bss.*)
} : rw
0 Comments