FPGA Bitstream Explained¶
Version History
| Date | Description |
|---|---|
| Sep 18, 2020 | add github link and usenix paper |
| Dec 20, 2019 | Update |
| Oct 24, 2019 | Created |
The proof-of-concept code to decode Xilinx bitstream is here: https://github.com/lastweek/fpga_decode_bitstream.
USENIX Security 2020 has a paper on decrypting Xilinx bitstream. They find a vulnerability in the 7-series chip and in turn able to decrypt a fully encrypted bitstream. WHAT A HACK!
Introduction¶
An FPGA bitstream can configure an FPGA. A bitstream includes the description of the hardware logic, routing, and initial values for both registers and on-chip memory (e.g., LUT). The common believe is that a bitstream has vendor-specific format thus cannot be reversed or understood. This is partially true.
A bitstream file is more than the bits to configure an FPGA, it also has certain human-readable fields to describe those bits. In fact, it has an assembly-like instruction set to describe the FPGA configuration process. This note is trying to walk through this.
At a high-level, a bitstream file is similar to an executable program. Analogous to the ELF format, a bistream has its own format to describe the contents. Note, the file format is publicly documented 1. Thus, you can analyze the contents of a bitstream file, meaning you can understand the steps taken to configure the FPGA. The un-documented part is the bits mapping: the format of the configuration bits, especially how the bitstream bits map to specific on-chip LUTs, wires etc. Think this way: given some assembly instructions, you can simply understand that, say some assembly instructions are doing Addition on certain registers, however, the instructions do not specify which registers they operate on.
As a normal FPGA user, you mostly do not need to understand neither of these. You only need to understand this if you are planning to do bitstream readback, preemption scheduling, or similar stuff.
After reading this note, I hope you could understand that a bitstream file is just a sequence of instructions and data. Nothing fancy.
The FPGA chip usually has a simple state machine module to accept and parse the bitstream, then configure the chip (ICAP in Xilinx). As we mentioned earlier, the bistream file format is partially public, the mapping between the bitstream configuration bits and the actual physical resource is undocumented.
Bistream Related Files¶
In a normal flow, Vivado only generates a simple .bit file.
When you click “Program Device”, Vivado will use this file to configure your FPGA.
In addition to generating this file, Vivado is capable of generating a bunch other files.
You can find a complete coverage in this link.
We give a high level summary here.
Most of the files have the same content and have similar file size.
For instance, the difference between a .rbt and a .bit is that the former one is in ASCII format while the latter is in binary format,
but they have the same contents. As for a .bit and a .bin file, the latter does not have some ASCII headers at the beginning of the file.
.ll, the logical link file, is very interesting.
It tells you the mapping between user logic and the actual bit offset in the bistream file data section.
This file can be used to aid preemption scheduling.
However, note that, this file only documents a very small part of the mapping.
To the best of my knowledge, I think only the registers, on-chip memory are documented, but the routing
information is missing. Thus, this file can help reserve engineer bitstream data section to some extend, but not full of it.
Prjxray is an open source project working on cracking everything on 7-series FPGA.
Details¶
We use .rbt and .bit to demonstrate the file format.
Note that they are essentially the same thing, except the former in human-readable ASCII format.
The target board is VCU118, the one used by many cloud vendors.
The following snippt is the first few lines of the .rpt file.
The first few lines are human-readable ASCII contents describing some general information
about the bitstream. Starting from line 8 is the actual bitstream file contents.
Note that the .bin file starts directly from line 8, no general header info is attached.
The interesting part is the 1s and 0s.
Unless otherwise noted, when we refer to bitstream format, we focus on the 1s and 0s only
and omit any general ASICC information headers.
Xilinx ASCII Bitstream
Created by Bitstream 2018.3 SW Build 2405991 on Thu Dec 6 23:36:41 MST 2018
Design name: base_mb_wrapper;UserID=0XFFFFFFFF;Version=2018.3
Architecture: virtexuplus
Part: xcvu9p-flga2104-2L-e
Date: Wed Nov 20 04:13:05 2019
Bits: 641272864
11111111111111111111111111111111
11111111111111111111111111111111
11111111111111111111111111111111
...
Note that each line has 32 bits, thus 4 bytes. In Xilinx bistream format, each four bytes is a packet (analogous to CPU instruction). Each packet has certain format, it could be a special header packet, or a normal data packet. The header packet follows a simple assembly-like instruction set to dictate the configuration process. The bitstream file is a sequence of these four bytes packets.
Why it sounds so complicated, a sequence of instructions?! I think the short answer is that configuraing FPGA is not an easy task, and any wrong doings may permanently harm the chip. Natually, the designer would have a on-chip state machine to control the configuration process, not only to control the whole process but also to ensure safety.
Each Xilinx FPGA has an on-chip configuration packet processor. All configuration methods such as JTAG, SelectMAP, ICAP merge into this final narrow bridge to carry out the configuration. The configuration packet processor has many internal registers (similar to x86 RAX, CRn, MSR registers). The bitstream usually interact with one of the registers at a time to do one thing. For a more detailed explanation, check out this blog, and UG570 chapter 9.
To this end, a bitstream consits of three parts:
- 1) Header packets to prepare the configuration process.
- 2) The actual configuration bits in a contiguous sequence of data packets.
AN write to the
FDRIregister marks the beginning of this section. The length of this section is described by the packet following the FDRI header packet. - 3) Header packets to clean up the configuration process.
The actual configuration bits are the ones determine the FPGA functionality. Note that if you are using an SSI Xilinx device like VCU118, the bitstream format is a bit more complicated. Basically, each die has the above three parts. If an chip has N dies, it will have N above triplet. I have complained about this is not well documented here and here.
I wrote a simple C program
to parse the .rbt file and associate a human-reable syntax with each line.
I didn’t have a complete coverage of the header packet format.
The following snippt shows a parsed .rbt file with header removed.
Here, 0xffffffff has no effect, like a NOP.
0x000000bb and 0x11220044 are special bus detect words.
0xaa995566 is another special work marking the synchronization status.
The last few lines mark the beginning of the configuration bits section.
Parsed from base_mb_wrapper.rbt
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
ffffffff
000000bb Bus Width Sync
11220044 Bus Width Detect
ffffffff
ffffffff
aa995566 SYNC
20000000
20000000
30022001 Write to regs 17
00000000
30020001 Write to regs 16
00000000
30008001 Write to CMD
00000000
20000000
30008001 Write to CMD
00000007
20000000
20000000
30002001 Write to FAR
00000000
30026001 Write to regs 19
00000000
30012001 Write to regs 9
38003fe5 Write to regs 1
3001c001 Write to regs 14
00400000
30018001 Write to IDCODE
04b31093 IDCODE=4b31093
30008001 Write to CMD
00000009
20000000
3000c001 Write to regs 6
00000001
3000a001 Write to regs 5
00000101
3000c001 Write to regs 6
00000000
30030001 Write to regs 24
00000000
20000000
20000000
20000000
20000000
20000000
20000000
20000000
20000000
30002001 Write to FAR
00000000
30008001 Write to CMD
00000001
20000000
30004000 Write to FDRI
5065eadc <- The length of configuration bits, follows a certain format
00000000 <- The first 4 bytes of the configuration bits!
Thank you for reading. Hope you enjoyed this post.