OSD: Executable and relocatable file formats

Executable file sections

Executable files typically contain a file header at or near the start of the file. This header contains 'magic numbers' that identify the file type. Beyond this header, executable files are typically divided into sections. Each section is characterized by name, permissions (RWX), size, file offset, and virtual address (VMA).

Section in executable file	Read/Write/eXecute	C storage class
.text or _TEXT	R-X	code, `const` globals, 'large' literals
.rodata	R--	`const` globals, large literals
.data or _DATA	RW-	initialized globals, initialized static locals
.bss or _BSS	RW-	uninitialized globals, uninitialized static locals

Non-static C locals, whether initialized or not, are stored on the stack. Space for these variables is allocated at run-time. Memory for C++ objects usually comes from the heap, an area of memory just beyond the BSS. Heap memory is also allocated at run-time.

The ELF file format has one or more .rodata sections for read-only (const) data. Older file formats such as COFF put such data into .text

'Large' literals are constant values too large to be handled conveniently with immediate addressing, such as string literals and constant structures. GNU C will put literal strings in .data instead of .text or .rodata if you use the -fwritable-strings command-line option. (-fwritable-strings is deprecated in GCC 3.4)

The C language requires that the BSS be zeroed before main() is called. Because every variable in the BSS has the same initial value (zero), only the BSS size is stored in the executable file.

Binary files

There are many problems with binary files. Most of these are due to the lack of file headers:

No 'magic values' in header that let you verify the executable file format
No entry point in header. Entry point must be start of file.
No section addresses in header. The binary file must have an ORG value that agrees with the code that loads it.
No BSS size in header. The bootloader has no way of knowing how big the kernel actually is, so it can't load anything after the kernel (e.g. modules).
Can't objdump the kernel to see if it's linked properly
The MinGW version of ld crashes or complains ("PE operations on non PE file.") when asked to make a binary kernel.

DJGPP COFF file format

Authoritative docs: http://www.delorie.com/djgpp/doc/coff

Concise summary of DJGPP COFF file format

A loader for DJGPP COFF executables should find and load the .text, .data, and .bss sections:

C:\tmp>objdump --headers coff-os.x

coff-os.x:     file format coff-go32

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00003000  00100000  00100000  00000100  2**4
                  CONTENTS, ALLOC, LOAD, CODE
  1 .data         00001000  00103000  00103000  00003100  2**4
                  CONTENTS, ALLOC, LOAD, DATA
  2 .bss          00003000  00104000  00104000  00000000  2**2
                  ALLOC
  3 .stab         00006a50  00107000  00107000  00004100  2**2
                  CONTENTS, DEBUGGING
  4 .stabstr      000025b7  0010da50  0010da50  0000ab50  2**0
                  CONTENTS, DEBUGGING

Other COFF variants

coff-m68k (Motorola 680x0 CPU):
- All multi-byte values are BIG ENDIAN instead of little
- COFF file magic value is 0x150 instead of 0x14C
- Relocatable files contain only type 17 relocations, instead of type 6 or type 20
sh-coff (Hitachi SuperH CPU):
- All multi-byte values are BIG ENDIAN instead of little
- COFF file magic value is 0x500 instead of 0x14C
- Relocatable files contain only type 0 relocations, instead of type 6 or type 20
- Relocation records are 16 bytes instead of 10

ELF file format

Authoritative docs: http://www.acm.uiuc.edu/sigops/rsrc/pfmt11.pdf

Concise summary of ELF file format (incomplete)

A loader for ELF executables should find and load the program headers (also called segments), not the sections. Each program header has two different size values: size-on-disk (file size) and size-in-memory (memory size). If the memory size is greater than the file size, the program header contains the BSS, and the extra memory should be zeroed:

C:\tmp>objdump-elf --private-headers elf-os.x

elf-os.x:     file format elf32-i386

Program Header:
    LOAD off    0x00001000 vaddr 0x00100000 paddr 0x00100000 align 2**12
         filesz 0x0001e440 memsz 0x0001e440 flags r-x
    LOAD off    0x0001f440 vaddr 0x0011f440 paddr 0x0011f440 align 2**12
         filesz 0x000031f4 memsz 0x00007940 flags rw-

The readelf utility, which is now part of GNU binutils (including DJGPP binutils) can examine ELF files in great detail. readelf --segments ... will display the program headers and the corresponding sections:

C:\tmp>readelf --segments elf-os.x

Elf file type is EXEC (Executable file)
Entry point 0x100000
There are 2 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x001000 0x00100000 0x00100000 0x1e440 0x1e440 R E 0x1000
  LOAD           0x01f440 0x0011f440 0x0011f440 0x031f4 0x07940 RW  0x1000

 Section to Segment mapping:
  Segment Sections...
   00     .text .rodata
   01     .data .bss

Win32 PE COFF file format

Authoritative docs: http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx

Concise summary of Win32 PE COFF file format

A loader for Win32 PE COFF executables should find and load the .text and .data sections from the file. As with ELF program headers, the sections of Win32 PE COFF executables have separate size-on-disk (raw data size) and size-in-memory (virtual size) values. For PE files built with commercial compilers such as Borland C 5.5, the BSS size is encoded into the difference between the two size values for the .data section. For PE files built with CygWin or MinGW, there is a separate .bss section:

C:\tmp>objdump --headers pe-os.x

pe-os.x:     file format pei-i386

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00004000  c0000000  00004000  00002000  2**2
                  CONTENTS, ALLOC, LOAD, CODE
  1 .data         00001000  c0004000  0000c000  00006000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .edata        00001000  c0010000  00001000  00007000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  3 .rsrc         00001000  c0011000  00001000  00008000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  4 .reloc        00001000  c0012000  00001000  00009000  2**2
                  CONTENTS, ALLOC, LOAD, DATA

The 'LMA' value shown here is actually the section virtual size.

MinGW based on GCC 2.95.2 stores the BSS size in the wrong field of the section headers. Because of this bug, you can't link old MinGW object files with object files generated by other tool chains (including NASM). This bug has been fixed in the newest versions of MinGW, available from SourceForge. This bug may also be present in older versions of CygWin.

a.out file formats

DJGPP a.out (a.out-i386) and Linux a.out (a.out-i386-linux) are different file formats.

This file format is useful if you want to mix 16- and 32-bit code, but is otherwise obsolete.

Code snippets

Load and run DJGPP COFF, Win32 PE COFF, or ELF .o file (demo relocation code).

Create task from DJGPP COFF executable

Create task from statically-linked Win32 PE COFF executable

Create task from statically-linked ELF executable

Links

http://www.nondot.org/sabre/os/articles/ExecutableFileFormats/
File formats: http://www.wotsit.org
Dynamic linking and loading: http://www.iecc.com/linker/linker10.html
Making tiny (45 bytes!) ELF executables with NASM under Linux: http://www.muppetlabs.com/~breadbox/software/tiny/teensy.html

TO DO

- finish summaries of file formats
- add summary of .EXE file format
- loading, relocating, running 16-bit .EXE file
- loading a DLL and calling code in it, or letting it call us
        ELF     Win32 PE COFF
- loading and running a dynamically-linked executable
	ELF     Win32 PE COFF
- dlopen(), dlerror(), dlsym(), dlclose()