The Hidden World of Firmware: Exploring Your Computer's Boot-Up Process

A lot of people are interested in how the computer starts up. This is where the magic begins and continues as long as the device is on. In this article, we will take an overview of the process, including its various stages, the key components involved, and the challenges faced during the process. boot While our primary focus will be on the (the most widely used), other architectures would have many similarities in their boot process. I hope this article would be a valuable resource for anyone looking to deepen their knowledge in this field. Here we go! x86 architecture Table of Contents: BOOT ROM Non-Volatile Memory One Time Programmable Field Programmable eXecute In Place (XIP) Cache-As-Ram (CAR) Layout & Memory Mapping Non-Descriptor Mode Intel Flash Descriptor / Descriptor mode Intel Firmware Interface Table (FIT) AMD Embedded Firmware Structure Silicon Initialization Intel Firmware Support Package (FSP) AMD Generic Encapsulated Software Architecture (AGESA) Autonomous subsystems Intel ME AMD PSP Hardware Power Sequences BOOT ROM An (chip) that is located on the motherboard and stores the firmware code responsible for booting the computer is called . This name is not standardized, so other developers often call it , , , , etc (such names are given to them because of the technology, interface, and purpose names). Not to worry, these terms are interchangeable. The firmware code in the is executed first when the computer is powered on. It performs basic tests, initializes the hardware, and then loads the OS loader from a bootable device, such as a hard drive or a USB drive, into memory. This chip is made from . integrated circuit BOOT ROM FLASH ROM BIOS FLASH BOOT FLASH SPI FLASH BOOT ROM non-volatile memory (NVM) Non-Volatile Memory Non-volatile memory is a type of computer memory that . It makes this type of memory ideal for storing important data that needs to be retained even when the computer is powered off. Furthermore, the discussion will be focused only on the memory that holds the firmware code. We will not talk about storage such as Hard Disk Drives (HDD), Solid State Drives (SSD), floppy disks, and so on. retains its contents even when the power is turned off Basically, we can categorize this type of memory into the following groups. One Time Programmable Masked ROM: The contents are determined at the time of manufacture and cannot be changed after that. Programmable ROM (PROM): Unlike masked ROM, this type of memory can be programmed after manufacture. But still only once. Field programmable Erasable Programmable ROM (EPROM): Сan be programmed multiple times, but its contents can be erased and reprogrammed using . ultraviolet light Electrically Erasable Programmable (EEPROM): Can be reprogrammed multiple times using . electrical signals : Architecturally arranged in blocks where data is erased at the and can be read or written at the . NOR memory is directly accessible using a standard interface such as byte parallel, I2C, or SPI. NOR Flash memory block level byte level In the industry, there is a convention to reserve the term to byte-wise erasable memories compared to block-wise erasable memories. EEPROM flash Programmable memory comes with one rule - . In such memory, writing new data is more complicated because the data is stored as a charge on a floating gate (the reason most of it just lies in the physics of the memory cells). The amount of charge on the gate determines whether the cell stores "0" or "1". erase before write When you erase a flash memory chip, you set all the bits of data stored on it to a known (default) state, typically a logic "1". This allows you to start with a clean slate, so to speak, and program new data onto the chip without having any remnants of the old data still stored on it. When new data is written to the chip, the state of individual bits is changed from "1" to "0" to represent the new data. If you simply write new data to the chip without first erasing it, the new data would be combined with the old data, resulting in unpredictable results. For example, consider a flash memory chip that has 8 bits of memory storing the value "0110 0010". If you write new data "1100 1001" to the chip without first erasing it, the resulting state of the chip would be "0100 0000", which may not be what you intended. The main confusion is related to the word which stands for . The term "Read-Only Memory" has been historically used to refer to memory that is permanent and cannot be altered by the user. However, as technology has advanced, the definition of ROM has changed, and now it is often used to refer to memory that is pre-programmed at the factory and cannot be changed by the end user. But if the user has desired skills and specialized equipment (for example, a programmer), the person can reprogram the chip. The name ROM has remained, even though the definition has changed, as a historical reference to the original purpose of the memory. ROM Read Only Memory easily By applying write protection, some types of reprogrammable ROMs may temporarily become read-only memory. These are NOT ALL existing types of non-volatile memory, but most of the popular ones you might hear about just by chance. Nowadays, on most motherboards, these chips are made by using . NOR Flash technology eXecute In Place (XIP) Execute in Place (XIP) is a method that allows the processor to execute code directly from the flash memory without copying it into the volatile memory (such as ) first. This is achieved by mapping the flash memory into the processor's address space, so that code execution can be performed directly from the flash. So, the system is able to start executing code as soon as possible, without having to wait for the RAM to be initialized first. RAM Wait... the CPU can communicate with BOOT ROM via SPI/Parallel/etc protocol? Of course not, it is just fetching instructions from system memory, requests to this memory region are redirected to the Intel Direct Media Interface (DMI or AMD / Unified Media Interface (UMI) (predecessor). It's the link between the CPU and the chipset on the motherboard. At this point, decoding of the address is performed via decoders located in the chipset, and data from the chip is returned to the processor. ) Infinity Fabric (IF) When the chip is made from NOR flash memory, which supports random access reads, but not random access writes came one problem. As far as writable memory isn't available, all computations have to be performed within the processor registers. At this point, the code can be written only in assembly language and it tends to set up the environment for high-level language (typically, for ). The reason for this is that memory initialization has become so complex, that it would be difficult to write purely in assembly. Since such languages require at least a heap and stack, we need writable memory. Some processors have SRAM embedded in the chip itself, but a more modern approach is to use the onboard . C language Cache memory As RAM (CAR) Cache-As-Ram (CAR) The is a high-speed memory that stores a copy of frequently used data and instructions from the main memory. A cache is located closer to the processor and organized into multiple levels (L1, L2, L3, ...), with each level being larger and slower than the previous one. CPU cache If the data is in the cache, the CPU can retrieve the requested data from the cache (it's called a ). When the CPU cache is unable to find the required data, it results in a . This can occur either because the data was never stored in the cache, or because the data was previously stored but has been from the cache. Anyway, the processor has to go all the way to the main memory to access the data and copy it into the cache. cache hit cache miss evicted is the process of removing data from the cache to free up space for new data. The eviction of data can be initiated either by the caching system (typically when a cache is full and new data needs to be stored, or when the data's time-to-live policy has expired) or by explicit request. Cache eviction However, if we want to use the CPU , we need to set up the cache to operate in , also called . This technique prevents due to a . Instead, the cache is treated as a regular SRAM, and all accesses (read/write) will hit the cache and will hit the main memory. The mode can be activated using vendor-specific CPU instructions. Cache as RAM Non-Eviction Mode No-Fill Mode eviction cache miss not Layout & Memory Mapping In reality, the BOOT ROM contains several types of firmware. Once a bunch of firmware is stored in the BOOT ROM, it needs to be organized somehow to distinguish between them. Let's find out how it is done. Non-Descriptor Mode Originally, the chipset does a straight mapping of the entire BOOT ROM contents to memory (from 4GB to 4GB - 16MB). Typically, if the BOOT ROM is less than 16 MB, the contents are mapped repeatedly. The CPU and the firmware can read/write to the flash without any restrictions. Non-Descriptor Mode is not supported anymore at new chipsets. Intel Flash Descriptor / Descriptor mode Eventually, in the ICH8, Intel introduces a special layout for BOOT ROM. The flash is divided into the following regions: Flash Descriptor (FD) - this data structure must be located at the beginning of the device with offset. It is made up of eleven sections as shown in the figure below: 0x10 The has pointers to the other regions and the size of each as well. Descriptor MAP The section has information about the flash(s) in the system (number of components, the density of each, invalid instructions, and so on). Component The section defines the read/write permissions for regions. As far as read/write is concerned, permissions must be set to Read-Only, the information stored in this region can only be written during the manufacturing process. Masters’ BIOS - only this region is mapped into memory. Converged Security and Management Engine (CSME / ME) - the firmware to support different Intel technologies and ME. Intel Gigabit Ethernet (GbE) - can only be directly accessed by the Gigabit Ethernet controller. Platform data Embedded Controller (EC) The and are the only required regions. Flash Descriptor Intel ME Intel Firmware Interface Table (FIT) The FIT is a data structure inside the and contains various entries that describe the platform configuration. Each entry in the table is 16 bytes in size. The first is called the , the other is called the . It's located by a at a physical address (4GB - 0x40). BIOS region FIT header FIT entry FIT pointer 0xFFFFFFC0 These components must be processed before executing the first CPU instruction from the . Entries include CPU microcode updates, Startup ACM, Platform Boot/TPM/BIOS/TXT policies, and other stuff. But at least the FIT should include and entries. So, the common usage of FIT is to update the microcode before executing the . reset vector FIT Header Microcode Update reset vector Here's what the memory map looks like: AMD Embedded Firmware Structure Unfortunately, there is much less information, I couldn't find any leaked AMD chipset documentation with details about their layout. So I can't tell you better than the . It is written based on documentation which is only available under NDA. coreboot documentation says AMD Actually, it will be enough to know that the AMD analog of Flash Descriptor is and it contains pointers to the , and other firmware. Embedded Firmware Structure PSP Directory Table BIOS Directory Table, Silicon Initialization If you desired to take a look at how exactly modern memory and CPU are initialized, then I have to upset you. Intel and AMD are not in hurry to release the Silicon Initialization Code to the community. As far as such information is not publicly available, they offer binary distribution of necessary silicon initialization code. This is to be considered a library for firmware developers and contains binary code to initialize the memory controller, the chipset, the CPU, and other different parts of the system. Intel Firmware Support Package (FSP) That binary can be split up into 4 components: FSP-T: Setting up the early execution environment ("Temporary RAM") in which C code can be executed. In practice, this binary sets up CAR, but also does some early hardware initialization like setting up PCIe memory-mapped configuration space. FSP-M: Initializing the permanent memory (such as DRAM). FSP-S: Completing the silicon initialization including CPU and IO controller initialization. FSP-O: Optional component that provides OEM devices initialization. Here’s a posted by Intel that you can find on their GitHub. can be obtained from the Intel website. Repository of Intel FSP binaries FSP Specification v2.1 AMD Generic Encapsulated Software Architecture (AGESA) AGESA for products earlier than Family 17h is known as or . At that time, AGESA was open-sourced and the code was available in the (it was deprecated after release 4.18). can be found on the AMD website. v5 Arch2008 coreboot repository The specification for Arch2008 With the introduction of (Zen microarchitecture) products, AMD has not published AGESA source code, only prebuilt binary solutions. Such a successor is called and supports Family 17h and later. Family 17h AGESA v9 openSIL There is no detailed information available, only . news Autonomous Subsystems An integral part of the modern x86 boot process, without which the x86 cores would never be activated. Therefore it is them. These technologies are responsible for the initialization of hardware, verifying system integrity, power management, and launch of the CPU. The firmware for these subsystems is loaded and executed the main processor starts executing its own firmware. A code on such systems runs independently from the CPU cores of the platform. impossible to completely disable before As long as many hardware companies have incorporated the principle of , neither the source code nor the documentation for these subsystems is available. Fortunately, we know how it affects the boot process - see . security through obscurity Hardware Power Sequences We won't go into detail, because there are already extensive articles on the Internet from researchers all over the world. But I will just give you a brief description of what it is. Intel Management Engine (ME) Intel ME is a separate microprocessor integrated into the Intel chipset (PCH) since 2008. It has its own RAM, built-in ROM, bus bridges to all of the buses inside the chipset (as result, it can access the network and even the main RAM on the CPU), and so on. Runs a custom OS based on MINIX. i486/80486 AMD Platform Security Processor (PSP) AMD PSP is an core relying on Trustzone extension, which is inserted into the CPU die as a coprocessor. This chip has been integrated into most AMD platforms since 2013. Runs an undocumented and proprietary OS. ARM Hardware Power Sequences This process, also knows as or , provides a number of derived voltage levels and/or rails in a particular order needed on the platform. In more simple terms, it powers up a number of platform components in a specific order. The process varies depending on the system or platform design, but typically a standard PC includes the following steps: Power On Sequence Power Sequencing power supply You press the . But wait... this button is on the that is not a necessary part of a computer. Usually, the power button is a cable. We have a button on one side and a that we put on on the other side. When we press the button these prongs are connected so the electricity can go through them. Watch the video below on how to turn on a computer without a power button if you're interested. power button computer case switch two metal prongs on the motherboard https://www.youtube.com/watch?v=G4FOBL1c3pA&embedable=true The sends a signal to the power supply unit (PSU). motherboard receives the signal, provides the proper amount of electricity, and sends a signal back to the motherboard. The power supply Once the motherboard receives the power good signal, it powers up platform components such as the core, clocks, chipset, memory, different controllers, and so on. A variety of subsystems, including autonomous subsystems (discussed above), may starts prior to the main processor. AMD-based systems (for Family 17h and later) PSP executes -chip BOOT ROM. on PSP locates the in the -chip BOOT ROM and executes PSP firmware. Embedded Firmware Table off PSP parses the to find ABL stages and executes them. PSP Directory Table ABL stages initialize main memory, locate the BIOS image in the BOOT ROM, and loads it into DRAM (decompresses if the image is compressed). This platform has no reason to use CAR because DRAM is already available and PSP loads the firmware image into it. Intel-based systems Chipset (ICH/PCH) finds the in the BOOT ROM. Intel Flash Descriptor Chipset copies CSME firmware into internal memory where Intel ME can access it and the last one starts executing it. Chipset maps the to the memory. BIOS region Microcode updates located in the are loaded into the CPU. They have to be . Firmware Interface Table applied at each system boot (optional) If Authenticated Code Modules (ACM) are found, then that entry is executed. All of this time, the is asserted to prevent the CPU from starting before other parts of the system are ready. When the platform is ready, the is de-asserted. In a multi-processor or multi-core system one CPU is dynamically chosen to be the that runs all of the firmware initialization code. The remaining processors, called at this point, remain halted until later on when they are explicitly activated by the firmware/kernel. CPU reset signal CPU reset line bootstrap processor (BSP) application processors (AP) After the CPU is first powered on, it operates in . Most of the registers have , including the Instruction Pointer (IP), the Code Segment (CS), and the , which is a of each within the processor to allow fast access to segment memory. real mode well-defined values Descriptor Cache copy segment descriptor The is an entry in the and contains the base address, segment limit, and access information (this part is ignored because the real mode doesn't have access control like the protected mode). Instead of accessing the GDT (which is located in the memory) for every memory access, the information is stored in a descriptor cache. Segment Descriptor Global Descriptor Table (GDT) However, the GDT is not involved in real mode, so the processor generates entries internally. The CS selector register, used to access the segment descriptor, is loaded with . The CS base address is initialized to . IP is initialized to . 0xF000 0xFFFF_0000 0xFFF0 Therefore, the processor begins from memory located at physical-address ( + ). The first instruction executed at that address is called the . fetching instructions 0xFFFF_FFF0 0xFFFF_0000 0x0000_FFF0 reset vector NOTE: This trick gives you access to the high address space, however, you cannot access the code below address. The CS base address remains at this initial value until the CS selector register is loaded by the firmware. It can be done by executing a . 0xFFFF_0000 far jump At this point, the best decision is to switch to protected mode with 4 GB of addressability. If the firmware doesn't do it, then in order for the real mode to work, the chipset must be able to alias a range of memory below 1 MB to an equivalent range just below 4 GB. Certain chipsets don't have this aliasing and may require a switch to another operating mode before the first long jump is performed. The address is in a section of , so the CPU uses the Execute In Place (XIP) method. Although if it's an AMD-based system, you're probably reading from main memory. non-volatile memory The CPU executes a firmware code. I recommend that you watch the video below about the power-on sequence, which explains the process using the ASUS P9X79 motherboard as an example. Despite the fact that it's in the Russian language, you'll be able to understand everything if you turn on the auto-generated English subtitles. https://www.youtube.com/watch?v=lAH7eSjR1d4&embedable=true This article has provided a lot of theoretical information related to how booting works. However, to truly understand this process, we need to take a closer look at the source code and architecture of existing firmware. In the next article, we will delve deeper into , and to examine them in detail. BIOS UEFI, coreboot Resources , web archive Booting an Intel Architecture System, Part I: Early Initialization How Computers Boot Up ROM, EPROM, AND EEPROM TECHNOLOGY Differences Between EEPROM and NOR Flash A gentle introduction to Cache-as-Ram on X86 Pictures about non-descriptor and descriptor mode are derived from , which was derived from Xeno Kovah's 'Architecture 4001: x86-64 Intel Firmware Attack & Defense' class John Butterworth & Xeno Kovah's 'Advanced Intel x86: BIOS and SMM' class