Resea

Resea [ríːseə] is a microkernel-based operating system written from scratch. It aims to provide an attractive developer experience and be hackable: intuitive to understand the whole design, easy to customize the system, and fun to extend the functionality.

Features

  • A pure microkernel with x86_64 (SMP) support
  • TCP/IP server
  • Some device drivers (PS/2 keyboard, e1000 network card, and text-mode screen)

License

CC0 or MIT. Choose whichever you prefer.

Getting Started

Prerequisites

  • LLVM/Clang/LLD version 8.x or higher
  • Python version 3.7 or higher
  • QEMU
  • Bochs (optional)
  • GRUB and xorriso (optional)
  • Docker (optional: Linux ABI emulation uses Docker to build packages)

macOS

$ brew install llvm python3 qemu bochs i386-elf-grub xorriso mtools sparse
$ brew cask install gcc-arm-embedded
$ pip3 install -r tools/requirements.txt

Ubuntu 20.04

$ apt install llvm-8 lld-8 clang-8 qemu bochs grub2 xorriso mtools \
    python3 python3-dev python3-pip python3-setuptools \
    gcc-arm-none-eabi
$ pip3 install --user -r tools/requirements.txt
$ cargo install cargo-xbuild

Rust

If you'd like to work on Rust applications, install Rust toolchain:

  1. Install rustup.
  2. Add rust-src component: rustup component add rust-src --toolchain nightly.

Building Resea

$ make build               # Build a kernel executable.
$ make iso                 # Build an ISO image.
$ make build BUILD=release # Build a kernel executable (release build).
$ make build V=1           # Build with verbose command output.
$ make clean               # Remove generated files.

Running Resea

$ make run GUI=1     # Run on QEMU.
$ make run           # Run on QEMU with -nographic.
$ make bochs         # Run on Bochs.

Design

In this section, we'll walk you through the design of Resea.

Microkernel Design

Microkernel, the key concept of Resea, is a well-studied operating system kernel design which extracts and isolates traditional kernel features such as device drivers and TCP/IP protocol stack into userspace.

Pros and Cons

The microkernel design improves flexibility and security by isolating components.

However, it has been said that microkernels are slow compared to monolithic kernels because communication between components involves Inter Process Communication (IPC) via kernel.

Improving IPC performance has been a key research topic: see L4 microkernel familly and SkyBridge if you're interested in.

Microkernel-based Design in Resea

Resea's microkernel (Resea Kernel) aims to be pure: the kernel does not implement features that can be done in userspace.

In Resea, like other microkernel-based operating systems, traditional kernel features such as file system, TCP/IP, and device drivers are implemented as standalone userland programs (called servers).

Similar to the "Everything is a file" philosophy in Unix, Resea has a philosophy: "Everything is a message passing". Reading a file, receiving a keyboard event, and all other operations and events are represented as messages.

Tasks

A task is a unit of execution just like process in other operating systems. It contains a CPU context (registers) and its own virtual address space.

The biggest difference is that Resea does not have kernel-level threading: in other words, only single thread can exist in a task 1.

Server

Server is a task which provides services like device driver, file system, TCP/IP, etc. While we use the term server in documentation and code comments, the kernel does not distinguish between server tasks and client (non-server) tasks.

Pager

Each tasks (except the very first task created by the kernel) is associated a pager, a task which is responsible for handling exceptions occurred in the task. When the following events (called exceptions) occur, the kernel sends a message to the associated pager to handle them:

  • Page fault: The pager task is responsible for mapping a memory page by the map system call (or kills the task if it's a so-called segmentation fault). Specifically, a pager allocate a physical memory page for the task, copy the file contents into the page, map the page, and reply the message to resume the task.
  • When a task exits: Because of invalid opcode exception, divide by zero, etc.
  • ABI Emulation Hook: If ABI emulation is enabled for the task, the kernel asks the pager to handle system calls, etc.

This pager mechanism is introduced for achieving the separation of mechanism and policy and it suprisingly improves the flexibility of the operating system.

1

Note that you can still implement threads in Resea by simply mapping same physical memory pages in your pager. I suppose the size of page table is negligible.

Message Passing

The message passing is the only way provided by the kernel to communicate with other tasks.

The IPC operations, sending and receiving a message, are synchronous (i.e. blocking). The desitnation of a message is specified by task ID and Resea does not provide the notion of channel or port of IPC for simplicity.

For more details, see IPC API.

Why not asynchronous?

Synchronous IPC is good for performance and for the separation of mechanism and policy. Survey L4 microkernels if you're interested in.

Futhermore, syncronous IPC siginificantly simplifies the complexity of a multi-server OS like Resea and makes easy to debug your code.

Message

A message is fixed sized and consists of the following fields:

  • The type of message (e.g. FS_READ_MSG).
  • The sender task ID.
  • The fixed-sized payload (less than 256 bytes, depends on arch).
  • The out-of-line payload for large data.

We use our own Interface Definition Language to generate message definitions.

Notifications

While the sending and receiving a message are blocking operations and it works pretty well in most cases, sometimes you will need a asynchronous way to send a message.

Notification is asynchronous (non-blocking) IPC mechanism which simply updates (bitwise-ORing) the notifications field of the destination task. When a task tries to receive a message, the kernel first checks pending (unreceived) notifications, and if exists, the kernel returns them as a message.

Note that notification IPC is just a bitfield update (just like signals in UNIX): it's impossible to determine how many times a same notification has been notified.

Why we need notifications?

Let's say that you're desiging a TCP/IP server and underlying device drivers:

              send a message
             when packet arrives
   Device  ----------------------->  TCP/IP
   Driver                            Server
     ^                                |
     |                                |
     +--------------------------------+
       send a message to emit packets

It looks an intuitive approach, however, what if the device driver tries sending a network packet when the TCP/IP server is trying to send a message to the driver? It would cause a deadlock because IPC operations are blocking.

The most important thing when you're writing a Resea application is: never have two task send messages each other as described in QNX's IPC documentation.

How can we avoid deadlocks in Resea? This is where notifications comes into play.

Notify & Pull Pattern

Instead of sending message each other, when TCP/IP sends some data to the driver, it notifies the driver asynchronously that there's pending data. When the device driver receives the notification, it pulls the pending data via message passing:

            2. request and receive
             the sending packet
   Device  ----------------------->  TCP/IP
   Driver                            Server
     ^                                .
     .                                .
     ..................................
         1. notify that new data is
          available asynchronously

See Asynchronous IPC for more detials.

Memory Management

Resea Kernel does not allocate memory for userland programs. Instead, an userland server task manages memory pages. The kernel only reserves small chunk of memory for its internal use (e.g. page tables).

How Page Faults are Handled

  1. Page fault occurrs in a task.
  2. The kernel sends a PAGE_FAULT_MSG to its pager task on behalf of the task.
  3. The pager task (e.g. vm server) allocates a memory page and maps it into the task's virtual memory by the map system call. Lastly, the pager task replies PAGE_FAULT_REPLY_MSG.
  4. The kernel resumes the task.

Servers

Virtual Memory Manager (vm)

The virtual memory manager, called vm, is responsible for allocating physical memory pages and managing tasks (as a pager task), etc.

Once the kernel initializes itself, it loads and starts vm as the first task.

Features

  • Allocating and mapping physical memory pages. In other words, the kernel does not allocate memory pages at all. The responsibility is delegated to vm.
  • Launching tasks and handling their exceptions (e.g. page faults) as their pager task.
  • Service discovery (ipc_lookup API).
  • Out-of-Line payload transmitting.

Source Location

servers/vm

Device Manager (dm)

The device manager, called dm, is responsible for managing devices on the computer and their device drivers.

Source Location

servers/dm

TCP/IP Protocol Stack

A TCP/IP server (tcpip) implements popular internet protocols.

Supported Protocols

  • IPv4
  • TCP
  • UDP
  • DHCP client
  • ARP
  • ICMP (Echo Request only)
  • DNS client

Source Location

servers/tcpip

MinLin - A binary compatibility layer for Linux programs (experimental)

MinLin is twofold: MinLin server and MinLin Linux distribution.

MinLin server is responsible for providing ABI compability (especially Linux system calls) for unmodified Linux binaries that run natively on Resea.

MinLin Linux is a minimalistic Linux distribution for testing this compatibility layer. Of course it runs on Linux kernel as well.

Source Location

servers/minlin

Apps

Shell

TODO:

Source Location

servers/apps/shell

Benchmark

The benchmark app is for benchmarking basic operations on Resea.

Source Location

servers/apps/benchmark

Integrated Tests

The test app implements integrated tests for Resea Kernel and the standard library (libs/resea).

Source Location

servers/apps/test

Web API Server

TODO:

Source Location

servers/apps/webapi

File System Drivers

FAT File System

The fatfs server implements the FAT file system.

Supported FAT versions

  • FAT16

References

Source Location

servers/fs/fatfs

Tar File System

The tarfs server implements the tarfs file system, a volatile in-memory file system which loads files from a tarball.

Usually, tarfs is used for debugging.

Source Location

servers/fs/tarfs

Drivers

virtio-net

A network device driver for virtio-net version 1.x (so-called modern device) and 0.x (legacy device).

References

Source Location

servers/drivers/net/virtio_net

e1000

A device driver for Intel's e1000 network cards.

References

  • 8254x Family of Gigabit Ethernet Controllers Software Developer's Manual.

Source Location

servers/drivers/net/e1000

ide

A device driver for hard disks that support the ATA PIO mode.

References

Source Location

servers/drivers/blk/ide

ramdisk

A pseudo device driver which provides in-memory virtual disk.

Usually it is used for debugging the file system drivers.

Source Location

servers/drivers/blk/ramdisk

Userspace Development

In the following pages, let's develop a server called rand, which generates a (pseudo) random numbers!

A server is a userspace program which provides services such as:

  • Device driver
  • File system
  • TCP/IP protocol stack
  • ...

Mainloop

First, let's write the mainloop! You can use the template in servers/example.

// main.c
#include <resea/ipc.h>
#include <resea/printf.h>
#include <string.h>

void main(void) {
    TRACE("starting...");

    INFO("ready");
    while (true) {
        struct message m;
        bzero(&m, sizeof(m));
        ASSERT_OK(ipc_recv(IPC_ANY, &m));

        switch (m.type) {
            case BENCHMARK_NOP_MSG:
                m.type = BENCHMARK_NOP_REPLY_MSG;
                m.benchmark_nop_reply.value = 123456789;
                ipc_reply(m.src, &m);
                break;
            default:
                TRACE("unknown message %d", m.type);
        }
    }
}

From this short snippet, we can learn how servers (and applications) are written in Resea:

  • In userspace programming, we'll use Resea Standard Library (libs/resea). In C, they're available in <resea/*.h> header files.
    • <resea/ipc.h>: Message passing APIs such as ipc_recv.
    • <resea/printf.h>: Print/assertion macros such as INFO, TRACE, and ASSERT_OK.
  • The message passing APIs (ipc_recv and ipc_reply) are blocking and the message is fixed-sized (32-256 bytes, depends on arch).
  • If you re-use the message buffer m to reply a message to the client, clear it with bzero to prevent information leak.
  • We recommend to write a program in single-threaded event-driven programming, that is, receive a message, handle it, reply, and then wait for a new one, ...

Build Files

Add build.mk in the server's directory:

# build.mk

# The server name. It must satisfy /[a-zA-Z_][a-zA-Z0-9_]*/
name := ps2_keyboard
# A short description.
description := PS/2 Keyboard Driver
# Object files.
objs-y += main.o
# Library dependencies.
libs-y += driver

If you'd like to add build configuration, add Kconfig file to the directory:

menu "ps2_keyboard - PS/2 Keyboard Driver"
    # PS2_KEYBOARD_SERVER is set if this `ps2_keyboard` is enabled in the config.
	depends on PS2_KEYBOARD_SERVER

    config PRINT_PERIODICALLY
        bool "Print a message every second"
endmenu

See Kconfig Language for details.

Interface Definiton Language (IDL)

Since we plan to support multiple programming languages (C and Rust), for interoperability, we use our own Interface Definition Language (IDL).

Message definitions are automatically generated from the IDL files.

Let's a take a look at an example:

namespace fs {
    rpc open(path: str) -> (handle: handle);
    rpc close(handle: handle) -> ();
    rpc read(handle: handle, offset: offset, len: size) -> (data: bytes);
}

IPC

Header File

#include <resea/ipc.h>

Message Structure

A message is fixed-sized. It contains the message type (or an error), the sender task ID, and the message payload (arbitrary bytes, defined by IDL).

/// Message.
struct message {
    /// The type of message. If it's negative, this field represents an error
    /// (error_t).
    int type;
    /// The sender task of this message.
    task_t src;
    /// The message contents. Note that it's a union, not struct!
    union {
        // The message contents as raw bytes.
        uint8_t raw[MESSAGE_SIZE - sizeof(int) - sizeof(task_t)];

        // The common header of message fields.
        struct {
            /// The ool pointer to be sent. Used if MSG_OOL is set.
            void *ool_ptr;
            /// The size of ool payload in bytes.
            size_t ool_len;
        };

        // Auto-generated message fields:
        //
        //     struct { notifcations_t data; } notifcations;
        //     struct { task_t task; ... } page_fault;
        //     struct { paddr_t paddr; } page_reply_fault;
        //     ...
        //
        IDL_MESSAGE_FIELDS
   };
};

Sending a Message

In Resea, IPC operations are sychronous. The destination is specified by a task ID. For simplicity, we don't provide indirect IPC mechanism so-called channel.

error_t ipc_send(task_t dst, struct message *m);
error_t ipc_send_err(task_t dst, error_t error);
error_t ipc_send_noblock(task_t dst, struct message *m);

ipc_send_err is a wrapper function which sets error to m.type and then sends the error message.

ipc_send_noblock tries to send a message. If the desitnation task is not ready for receiving a message, it immediately returns ERR_WOULD_BLOCK instead of blocking the sender task.

Receiving a Message

On a receive operation, you have two options, open receive and closed receive:

  • open receive (when src == IPC_ANY): accepts a message from any tasks.
  • closed receive (otherwise): accepts a message from the specific task (src). Other sender tasks are blocked.
error_t ipc_recv(task_t src, struct message *m);

Replying a Message from a Server

In case the sender task does not wait for a reply message, use the following wrapper functions (they wrap ipc_send_noblock). If the client calls the server with ipc_call, these APIs should success.

void ipc_reply(task_t dst, struct message *m);
void ipc_reply_err(task_t dst, error_t error);

Sending Notifications

Notifications is a asynchronous IPC like signals in UNIX. Each task has its own notifications bitfield. When a task tries to receive a meesage and pending notifications exist (i.e. the bitfield is not zero), the kernel constructs and returns NOTIFICATIONS_MSG with the notifications.

error_t ipc_notify(task_t dst, notifications_t notifications);

ipc_notify does bitwise-OR operation on the destination task's notifications bitfield and the given bits, i.e. dst->notifications |= notifications.

Send and Receive a Message at once

error_t ipc_call(task_t dst, struct message *m);
error_t ipc_replyrecv(task_t dst, struct message *m);

ipc_call is same as ipc_send(dst, m) and then ipc_recv(dst, m). Clients should use this API instead of calling those two APIs or ipc_reply from the server may fail.

Both APIs overwrite the message buffer m with the received message.

ipc_replyrecv is same as ipc_reply(dst, m) and then ipc_recv(IPC_ANY, m). With this API, you can reduce the number of system calls in the server.

Out-of-Line Payload

Since a message is fixed-sized and the size is very small (typically 256 bytes), we need another way to send large data (e.g. file contents).

Out-of-Line payload (OoL in short) is a feature implemented by vm server for that purpose.

OoL Types

OoL supports the following payload types:

TypeDescription
bytesan arbitrary data
stra string terminated with \0

Caveats

  • It's slow for now since it needs some IPC calls with vm.
  • Only single OoL payload is supported per message.
  • The maximum size of an OoL payload is configureable in the build config.

Sending a OoL Payload

OoL is integrated with the IDL and userspace library. Let's take a look at an example:

namespace fs {
    rpc open(path: str) -> (handle: handle);
    rpc create(path: str, exist_ok: bool) -> (handle: handle);
    rpc close(handle: handle) -> ();
    rpc read(handle: handle, offset: offset, len: size) -> (data: bytes);
    rpc write(handle: handle, offset: offset, data: bytes) -> ();
    rpc stat(path: str) -> (size: size);
}

In the fs interface definition shown above, as you can see, some methods use OoL payloads. fs.open uses str payload to send a path name to be opened, and fs.read returns a bytes payload for the read file contents.

To send a bytes, set a pointer to data and the length of data:

static void reply_file_contents(task_t client, uint8_t *data, size_t len) {
    struct message m;
    m.type = FS_READ_REPLY_MSG;
    m.fs_read_reply.data = data;
    m.fs_read_reply.data_len = len;
    ipc_call(client, &m);
}

To send a str, set a pointer to a null-terminated ASCII string to as str payload. The IPC library computes the length of the OoL payload by strlen and transparently copies it into the destination task (fs server):

struct message m;
m.type = FS_OPEN_MSG;
m.fs_open.path = "/hello.txt";
error_t err = ipc_call(fs_server, &m);

Receiving an OoL Payload

In the fs server, the IPC library sets a valid pointer to the OoL payload field. For str payloads, it is guaranteed that the string is terminalted by \0.

struct message m;
ipc_recv(IPC_ANY, &m);
if (m.type == FS_OPEN_MSG) {
    DBG("path = %s", m.fs_open.path);
}

For bytes payload, use <name>_len to determine the size of the payload:

ipc_recv(fs_server, &m);
if (m.type == FS_READ_REPLY_MSG) {
    HEXDUMP(m.fs_read_reply.data, m.fs_read_reply.data_len);
}

A memory buffer for received OoL payload is dynamically allocated by malloc. Don't forget to free!

How It Works

+--------+  3. ool.send  +------+  2. ool.recv  +----------+
| sender |  -----------> |  vm  | <------------ | receiver |
|  task  |               |      |               |   task   |
|        |               |      |  4. copy OoL  |          |
|        |               |      | ------------> |          |
|        |               |      |               |          |
|        |               |      | 5. send msg   |          |
|        | -----------------------------------> |          |
|        |               |      |               |          |
|        |               |      | 6. ool.verify |          |
|        |               |      | <------------ |          |
+--------+               +------+               +----------+
  1. For messages with a ool payload, IPC stub generator adds MSG_OOL to the message type field (i.e. (m.type & MSG_OOL) != 0 is true).
  2. In ipc_recv API, the receiver task sends a ool.recv message to tell the location of the OoL receive buffer (allocated by malloc) to the vm server.
  3. When a sender task ipc_send API, if MSG_OOL bit is set, it calls ool.send to the vm server before sending the message.
  4. The vm server looks for an unsed OoL buffer in the desitnation task, copies the OoL payload into the buffer, and returns the pointer to buffer in the receiver's address space.
  5. The sender tasks overwrites the OoL field with the receiver's pointer and sends the message.
  6. Once receiver task received a message with OoL, it calls vm's ool.verify to check if the received pointer and the length is valid.
  7. ipc_recv returns.

Why not Implement OoL in Kernel?

In fact, OoL is initially implemented in the kernel and is removed later because it turned out that page fault handling makes the kernel complicated.

Since we can now map memory pages in the userspace through the map system call, I believe that it is a better idea to implement a more efficient message passing with OoL support within userspace using shared memory.

Asynchronous IPC

Despite the asynchronous IPC works well in most cases, asynchronous message passing is sometimes convinient.

The Resea Standard Library provides an asynchronous message pasing on top of the sychronous message passing and notifications. See examples below for more details.

#include <resea/async.h>

error_t async_send(task_t dst, struct message *m);
error_t async_recv(task_t src, struct message *m);
error_t async_reply(task_t dst);

In a nutshell, async library manages message queues. An async message is enqueued and the destination task is notified that there's a pending async message. The message will be delivered when the clients sends a pull request (ASYNC_MSG).

Sending a Asynchronous Message

Enqueue a message by async_send and handle message pull requests (ASYNC_MSG) by async_reply:

// my_server.c

void somewhere(void) {
    // `async_send` enqueues the message and notifies the destination task with
    // the notification `NOTIFY_ASYNC`.
    m.type = BENCHMARK_NOP_MSG;
    async_send(dst, &m);
}

void main(void) {
    while (true) {
        struct message m;
        bzero(&m, sizeof(m));
        ASSERT_OK(ipc_recv(IPC_ANY, &m));

        switch (m.type) {
            case ASYNC_MSG:
                // Handle a request from the client's async_recv().
                async_reply(m.src);
                break;
        }
    }
}

Receiving a Asynchronous Message

Wait for NOTIFY_ASYNC notification and the use ipc_recv to receive the pending async message:

// my_client.c

void main(void) {
    while (true) {
        struct message m;
        bzero(&m, sizeof(m));
        ASSERT_OK(ipc_recv(IPC_ANY, &m));

        switch (m.type) {
            case NOTIFICATIONS_MSG:
                if (m.notifications.data & NOTIFY_ASYNC) {
                    // Pull a pending asynchronous message from the server.
                    // As you can see, you have to know which server would
                    // send an async message in advance: you cannot determine
                    // which task has notified NOTIFY_ASYNC!
                    async_recv(my_server, &m);
                    switch (m.type) {
                        case BENCHMARK_NOP_MSG:
                            INFO("received a async message!");
                    }
                }
        }
    }
}

Service Discovery

The vm server implements service discovery, it allows looking for services by their names.

Registering a Service

#include <resea/ipc.h>

error_t ipc_serve(const char *name);

Looking for a Service

#include <resea/ipc.h>

task_t ipc_lookup(const char *name);

This function blocks until the server with the given name has been registered, and then returns the server's task ID.

Memory Allocation (malloc)

We provide some dynamic memory alllocation APIs.

#include <resea/malloc.h>

void *malloc(size_t size);
void *realloc(void *ptr, size_t size);
void free(void *ptr);

See a man page in UNIX for details.

Debugging

We don't have rich debugging features yet. Use printf macros. Good luck!

How to deal with dead locks in IPC

Even if you don't use locks like mutex (note that we don't provide such a thing), your program could be blocked forever by an IPC operation.

The common case is that both your program and the destination task are trying to send a message to each other. You can check it by ps command in the kernel debugger:

kdebug> ps
#1 vm: state=blocked, src=0
#2 display: state=blocked, src=0
#3 e1000: state=blocked, src=6
  senders:
    - #6 tcpip
#4 ps2kbd: state=blocked, src=0
#5 shell: state=blocked, src=0
#6 tcpip: state=blocked, src=3
  senders:
    - #3 e1000
#7 webapi: state=blocked, src=0

Notice that e1000 and tcpip are blocked and they're sending to the other server.

Unit Testing

libs/unittest provides a very primitive unit testing framework for libraries (in libs) and userspace applications (in servers). It's useful if you're writing a somewhat complicated function.

This framework enables some attractive characteristics and features:

  • The framework compiles tests into a normal userspace application for the your development environment (e.g. macOS).
  • Since the testing program is a native application, it is super-fast and you can use your favorite debugging tools like LLDB!
  • Undefined Behavior Sanitizer and Address Santizer are enabled by default.

Caveats

  • You can't use system calls (e.g. message passing) from the testing environment.
  • main() in your Resea application won't be called.

Writing Tests

#include <unittest.h>

int add(int a, int b) {
    return a + b;
}

TEST("1 + 1 equals to 2") {
    TEST_EXPECT_EQ(add(1, 1), 2);
}

Macros

  • TEST("description"): Use this macro to define a unit testing function.
  • TEST_EXPECT_EQ(a, b): Checks if a == b holds.
  • TEST_EXPECT_NE(a, b): Checks if a != b holds.
  • TEST_EXPECT_LT(a, b): Checks if a < b holds.
  • TEST_EXPECT_LE(a, b): Checks if a <= b holds.
  • TEST_EXPECT_GT(a, b): Checks if a > b holds.
  • TEST_EXPECT_GE(a, b): Checks if a >= b holds.

How to Run Tests

$ make unittest TARGET=servers/apps/test

Writing a Library

TODO:

Kernel Development

In this section, we'll walk you through the Resea Kernel developement.

Kernel Debugging

Resea Kernel is written in C. While some people say "C is a bad language! Rewrite everything in Rust!", C is a pretty good chioce for writing kernel because it makes easy to understand what happens.

That said, debugging C code (especially in the kernel world) is really painful. In this page, we'll walk you through some useful features for kernel debugging.

printk

Use the following macros:

  • TRACE(fmt, ...)
    • A trace message. Disabled on release build.
  • DEBUG(fmt, ...)
    • A debug message. Disabled on release build.
  • INFO(fmt, ...)
    • A info message.
  • WARN(fmt, ...)
    • A warning message.
  • OOPS(fmt, ...)
    • Same as WARN but it also prints a backtrace.
  • OOPS(expr)
    • Prints an oops message if expr != OK.
  • PANIC(fmt, ...)
    • Kernel panic. It prints the message and halts the CPU.
  • BUG(fmt, ...)
    • An unexpected situation occurred in the kernel (a bug). It prints the message and halts the CPU.

Backtrace

  • backtrace()
    • Prints a backtrace like the following output. We recommend to use OOPS macro instead.
[kernel] WARN: Backtrace:
[kernel] WARN:     #0: ffff80000034a7e0 backtrace()+0x3c
[kernel] WARN:     #1: ffff800000113de7 kernel_ipc()+0x77
[kernel] WARN:     #2: ffff80000010f7bb mainloop()+0x6b
[kernel] WARN:     #3: ffff80000010f4a9 kernel_server_main()+0x149
[kernel] WARN:     #4: ffff8000001027d6 x64_start_kernel_thread(+0xa

Kernel Debugger

Kernel debugger is available only in the debug build. You can use it over the serial port. Implemented commands are:

  • ps
    • List processes and threads. It's useful for debugging dead locks.

Runtime Checkers

In the debug build, the following runtime checkers are enabled.

Porting to a CPU Architecture

Porting to a new CPU architecture (arch in short) is pretty easy if you're already familiar with the architecture:

  1. Scaffold your new port using the example arch: libs/common/arch/example, kernel/arch/example, and libs/common/arch/example.
  2. Define arch-specific types and build settings in the common library.
  3. Implement Hardware Abstraction Layer (HAL) for kernel.
  4. Implement arch-specific part in the resea library.
  5. Add the architecture in Kconfig.

Implementing common library

The common library (libs/common) is responsible for providing standalone libraries (e.g. doubly-linked list) and types for both kernel and userspace programs. You'll need to implement the following files.

  • libs/common/arch/<arch-name>/arch.mk
    • Build options for the arch: $CFLAGS, run command, etc.
  • libs/common/arch/<arch-name>/arch_types.h
    • Arch-specific #defines and typedefs.

Porting the kernel

For portability, the kernel separates the arch-specific layer (Hardware Abstraction Layer) into kernel/arch.

Roughly speaking, you'll need to implement:

  • CPU initialization
  • Serial port driver (for print functions)
  • Context switching
  • Virtual memory management (updating and switching page tables)
    • Resea Kernel also supports NOMMU mode for CPUs that don't implement virtual memory.
  • Interrupt/exception/system call handlers
  • The linker script for the kernel executable (kernel/arch/<arch-name>/kernel.ld)
  • Multi-Processor support (optional)

Implementing resea library

The resea library is the standard library for userspace Resea applications. You'll need to implement:

  • The syscall() function.
  • The linker script for userspace programs (libs/resea/arch/<arch-name>/user.ld).
  • The entry point of the program: initialize stack pointer and then call resea_init().
  • Bootfs support. Bootfs is a simple file system image (similar to tar file) for Resea. Resea starts the first userspace programs from that file. You need to embed the bootfs header to make room for the bootfs header. See libs/resea/arch/x64/start.S for a concrete example.

Common Library (libs/common)

Boot Sequence

  1. The bootloader (e.g. GRUB) loads the kernel image.
  2. Arch-specific boot code initializes the CPU and essential peripherals.
  3. Kernel initializes subsystems: debugging, memory, process, thread, etc. (kernel/1. boot.c)
  4. Kernel creates the very first userland process from bootfs.
  5. The first userland process (typically vm) spawns servers from bootfs.

Build System

Boot FS

Bootfs is a simple file system embedded in the kernel executable. It contains the first userland process image (typically bootstrap server) and some essential servers to boot the system. It's equivalent to initramfs in Linux.

On-Disk Format

File System Header

+---------------------------------------------+
|                 Jump Code                   |
|    (The entrypoint of bootstrap server)     |
+---------------------------------------------+
|                  Version                    |
+---------------------------------------------+
|              File System Size               |
|          (excluding this header)            |
+---------------------------------------------+
|              Number of Files                |
+---------------------------------------------+
|              Reserved (padding)             |
+---------------------------------------------+
|                  File #1                    |
+---------------------------------------------+
|                  File #2                    |
+---------------------------------------------+
|                    ...                      |
+---------------------------------------------+

File

+---------------------------------------------+
|                 File Path                   |
|             (terminated by NUL)             |
+---------------------------------------------+
|                 File Size                   |
+---------------------------------------------+
|                Padding Len                  |
+---------------------------------------------+
|                 Reserved                    |
+---------------------------------------------+
|               File Content                  |
+---------------------------------------------+
|                 Padding                     |
+---------------------------------------------+

Change Log

v0.5.0 (Oct 3, 2020)

  • Support bare-metal Raspberry Pi 3B+. Resea now boots on real Raspberry Pi!
  • Support Google Compute Engine: A HTTP server (servers/apps/webapi) on Resea works in the cloud!
  • Add the virtio-net device driver. It supports both modern and legacy devices.
  • tcpip: Support sending ICMP echo request.
  • tcpip: Fix some bugs in the DHCP client.
  • Some other bug fixes and improvements.

v0.4.0 (Aug 21, 2020)

  • shell: Use the serial port driver in kernel for the shell access.
  • Support command-line arguments.
  • libs/resea: Add parsing library <resea/cmdline.h>.
  • tcpip: Implement TCP active open.
  • Add command-line utilities application named utils.
  • Remove display and ps2kbd device drivers.
  • kernel: Deny kernel memory access from the userspace by default.
  • kernel: Reorganize internal interfaces.
  • Introduce sparse, a static analyzer for C.

v0.3.0 (Aug 2, 2020)

  • Reorganized system calls into: exec, ipc, listen, map, print, and kdebug.
  • Removed kernel heap for separation of mechanism and policy. Task data structures are now statically allocated in the kernel's .data section, and page table structures are now allocated from the userland (through kpage parameter in map system call).
  • Started implementing Rust support. Currently, there's only a "Hello World" sample app. I'll add APIs once the C API gets stabilized (hopefully September).
  • shell: Add log command to print the kernel log.
  • Many bug fixes and other improvements.

v0.2.0 (June 14, 2020)

  • Add experimental support for Micro:bit (ARMv6-M).
  • Add experimental support for Raspberry Pi 3 (AArch64 in ARMv8-A).
  • Add experimental (still in the early stage) Linux ABI emulation layer: run your Linux binary as it is on Resea!
  • A new build system.
  • Bunch of breaking changes, bug fixes, and improvements.

Coding Style Guides

Please follow this style guide for consistency. Because a well-written coding style is boring to read, we are not strict with the coding styles.

Line Length

80 characters.

C Style Guides

Indentation

4 spaces.

Functions

void handle_message(struct message *m) {
}

static inline struct toooooooooo_loooooooong *function(int a, int b, int c,
                                                       int d, int e) {
}

static void func(
    const struct very_long_name *a,
    const struct very_long_name *b
) {
}

Switch

switch (m.type) {
    case FS_READ_MSG:
        return_value = fs_read();
        break;
    case FS_WRITE_MSG: {
        const void *data = m.fs_write.data;
        fs_write(data);
        break;
    }
}

Conditions

if (very_looooooooong_variable1 == very_looooooooong_variable2
    || very_looooooooong_variable1 != very_looooooooong_variable3) {
    stmt();
}

Function Calls

do_something(a, b, c, d,
             e, f, g, h);

Too Long Expressions

int result = very_looooooooong_variable1 + very_looooooooong_variable2
             + very_looooooooong_variable1;

or

int result = very_looooooooong_variable1 + very_looooooooong_variable2
    + very_looooooooong_variable1;

Macros

#define MAX(a, b)                                                              \
    ({                                                                         \
        __typeof__(a) __a = (a);                                               \
        __typeof__(b) __b = (b);                                               \
        (__a > __b) ? __a : __b;                                               \
    })

Don't omit curly braces for single-line bodies in if, while, ...

This rule prevents some nasty bugs (and sometimes security vulnerabilities).

// Bad
if (expr)
    stmt();

// OK
if (expr) {
    stmt();
}

Python Style Guides

Please follow PIP-8.

Rust Style Guides

Use rustfmt.