Skip to content

All

Source Code Study

Version History
Date Description
Sep 13, 2020 some notes for python; add tcpstat
Jul 26, 2020 Add OpenJDK! Hinted by Hacker News :)
Jun 2, 2020 Add librcu
Apr 26, 2020 Add wayland, X, gnome, gtk etc
Apr 10, 2020 add graphics section
Apr 6, 2020 add verbs perftes
Mar 3, 2020 add FreeBSD, some fpga stuff
Feb 4, 2020 add io_uring, firecracker
Jan 31, 2020 Add some good stuff
Jan 18, 2020 Initial

Beautiful code is art.

Nutrition

Projects supporting our day-to-day work.

  • GNU glibc: libc, elf, and dynamic linker
    • It is the default C library used by almost everyone
    • It includes ld.so, the dynamic linker
    • I wrote some notes about GOT/PLT and explains what has happend before main() is called.
  • GNU binutils: gas, static linker, and more
    • This repo has a lot commands like as, ld, objdump, nm and so on
    • ld is static linker and I like the magic of its linker script
    • I guess another useful repo is elfutils
  • Standard C Library
    • GNU glibc used by major Linux distributions
    • musl libc is a small libc impl used by Alpine Linux. Clean code.
    • uClibc is a small libc targeting embedded cases
    • bionic is Android’s C library, math library, and dynamic linker
  • strace
    • System call tracer at userspace
    • I’ve designed one for LegoOS in kernel space
  • Unix Commands
    • Of course almost all other listed repos in this section have some sort of commands. But they are not essential. The following repos have the essential UNIX commands like ls, cat. It’s not possible to go through all of them. But rather, I think they serve as references when we want to know how certain things are implemented (e.g., how dmesg get kernel log).
    • BusyBox
    • GNU Coreutils
    • util-linux
    • FreeBSD and its friends
  • Tools
  • Editors
  • C for life
    • Some small and useful C projects
    • cJSON: A lightweight JSON parser in C.
    • userspace-rcu: A userspace RCU implementation library.
  • Outliers
    • CRIU: Checkpoint and Restore in Userspace
      • The reason I love this repo is because it has so many interesting pieces on how to interact with kernel, save states, and restore them. In addition, it shows how to properly use many less well known syscalls.
    • GRUB2: bootloader
      • Learn how modern bootloader works.
      • Detailed analysis of Linux booting sequence (how it transit from real-mode to protected mode, and finally to 64-bit mode, how to navigate Linux source code etc.)
    • FFmpeg
      • FFmpeg project is famous for its clean and neat C code.
      • Besides, this project is used by a lot online video service companies
    • io uring

Operating Systems

image_unix_timeline (Image source: https://commons.wikimedia.org/wiki/File:Unix_timeline.en.svg)

Network

  • iperf3 is a TCP, UDP, and SCTP network bandwidth measurement tool
  • tcpdump
  • OpenSSH is our ssh!
  • scapy: Python-based interactive packet manipulation program & library. Very neat
  • tcpstat: C-based simple tool that could dump network traffic. Seems using pcap interface, the one used by tcpdump?
  • Also checkout FreeBSD as it has tools like ifconfig, if.

Virtualization

Also see: http://lastweek.io/notes/source_code/virt/.

Compilers

  • Clang, LLVM, in C++
    • This is a collection of projects. Clang is the frontend, compiles C/C++ code into LLVM’s own IR format. The the backend LLVM will take multiple Passes to optimize the IR and the finally generate the assembly.
    • The beauty of Clang and LLVM is that they can be used as libraries, and we could invoke them to manipulate the compilation results, to do source-to-source transforms, modify Pass’s IR etc. I found this super interesting!
    • To get started, I strongly recommend LLVM for Grad Students
  • OpenJDK
    • JRE = JVM + Runtime Classes => JVM is the one parsing the bytecode, along with some extra classes/libraries, they form JRE.
    • JDK = JRE + Development Tools => JDK as in Development Kit therefore consists of some tools in addition to JRE.
    • JDK is a monster collection of resources in one place. The JVM here is called HotSpot, a reference JVM implementation written in C++, Since JDK also has so many runtime support, it has a lot Java code.
    • Personally I haven’t written Java since 2013 or so. Although I’m not using it anytime soon, I’m curious how it performs nowadays.
  • Python, in C
  • GNU GCC
  • Rustc, in Rust
  • PHP, in C
  • Google V8, in C++
  • Apple Swift, in C++
  • TCL, in C
  • Perl 5, in C
  • Scala
  • SpinalHDL

Firmware

See here: http://lastweek.io/notes/source_code/20200506-on-firmware-softwares/.

The open-source firmware landscape: 20200506-on-firmware-landscape.png

FPGA

Web Servers

Key Value Stores

Point of interests: 1) in-memory, and can it extend to use disk/ssd? 2) persistence support 3) network support

Databases

RDMA and More

See here: http://lastweek.io/notes/source_code/rdma/

Graphics

More here: http://lastweek.io/notes/source_code/20200501-on-graphic-softwares/

  • X Server and Wayland
    • X is being replaced by Wayland now..
    • Wayland code seems clean
  • xvnc
    • xvnc and its friends, are sitting on top of display manager (i.e., X/Wayland). They are clients of X/Wayland, but they act as X/Wayland servers for upper layer application such as GTK/Qt.
    • It’s a middleman, bringing network between X and GTK.
    • TigerVNC, TurboVNC and so on.
  • GNOME Shell and GTK
    • GTK’s default backend is X.
    • GNOME shell is a layer on top of GTK+. Similar for KDE/Qt.
  • xRDP, an RDP server. In C
  • FreeRDP, client and server. In C
    • Took a brief read of the code, it’s super neat. Should take a serious look sometime.

This figure shows the software landscape: 20200501-on-graphic-softwares-img3.png 20200501-on-graphic-softwares-img3.png


Last update: September 17, 2020

Comments