Subscribe with RSS to keep up with the latest changes.
Exploring C++20 coroutines for embedded and bare-metal development on RISC-V platforms
Can C++20 coroutines build efficient, real-time embedded applications on bare-metal RISC-V platforms — no RTOS required?
November 24, 2024|Articles (C++,baremetal,coroutines)
Introduction
This post is about using C++ coroutines to suspend and resume functions in real time. The objective is a simple way of building real time tasks using only C++, without the need for an RTOS or operating system kernel.
Coroutines are functions that can be suspended and resumed, using the keywords co_await
, co_yield
and
co_return
. The C++20 standard introduced coroutines to the language.
C++ standardized the keywords and type concepts for coroutines, but it did not standardize a runtime1. The lack of a standard runtime has made them hard to use them “out of the box”, but the implementation of coroutines is very adaptable to different use cases.
Here I use a simple runtime implementing C++20
coroutines on bare metal (no operating system) for RISC-V, using the
co_await
keyword. This is done by passing the real time scheduler and resume time condition as the argument to the asynchronous wait operator.
The runtime is described in detail in this post.
This story is also published on Medium.
-
The C++23 standard library provides a limited runtime for coroutines generators. ↩
Building a header-only C++20 coroutine runtime for bare-metal RISC-V
Designing a lightweight coroutine runtime for real-time tasks without an OS, featuring awaitable timers and static memory allocation
November 24, 2024|Articles (C++,baremetal,coroutines)
Creating the coroutines runtime infrastructure
A simple coroutine example was presented in “C++20 coroutines, header only, without an OS”. This post describes the runtime used for that example in detail.
This story is also published on Medium.
Summary of the runtime files
The runtime for this example is a set of include files in include/coro
. These files are used:
nop_task.hpp
: Task structure includingpromise_type
to conform the C++ coroutines task concept.scheduler.hpp
: Generic scheduler class that can manage a set ofstd::coroutine_handle
to determine when they should resume and implement the resumption.awaitable_timer.hpp
: An “awaitable” class that can be used withco_await
to schedule a coroutines to wake up after a givenstd::chono
delay.static_list.hpp
: An alternative tostd::list
that uses custom memory allocation from a static region to avoid heap usage.awaitable_priority.hpp
: An alternative “awaitable” class for tasks to be scheduled to wake according to priority.
NOTE: All classes here are designed to not use the heap for allocation. They will allocate all memory from statically declared buffers.
Update to Docker
June 15, 2024|Toolchain (toolchain,vcd,simulation,debugging,tracing,baremetal,C,gtkwave)
Docker Images
The reference docker images for building and simulating examples from this blog have been updated:
https://github.com/five-embeddev/build-and-verify/tree/main/docker
Target | Description |
---|---|
riscv-tool-build | Base image for compiling tools |
riscv-spike | RISC-V ISA Simulator. |
riscv-openocd | OpenOCD JTAG debugger interface |
riscv-xpack-gcc | X-PACK packaged GCC |
riscv-spike-debug-sim | RISC-V Spike ISA configured to run with OpenOCD |
riscv-spike-debug-gdb | RISC-V Spike ISA configured to run with OpenOCD & GDB |
riscv-rust | Rust for RISC-V |
riscv-gnu-toolchain | Build GCC for RISC-V |
riscv-gnu-toolchain-2 | Build GCC for RISC-V (using docker compose to conserve resources) |
Docker Examples
Examples of usage:
https://github.com/five-embeddev/build-and-verify/tree/main/examples
Example | Description |
---|---|
build-c/ | Build a simple C test program for host and target |
build-rust/ | Build a small rust program |
build-run-sim/ | Build a full baremetal example and simulate with Spike, debug with GDB |
test-code-c/ | Small example program |
Build with Cmake
e.g. https://github.com/five-embeddev/build-and-verify/tree/main/examples/build-run-sim
docker run \
--rm \
-v .:/project \
-v /home/five/five-embeddev-wsl/build-and-verify/examples/build-run-sim/../test-code-c:/project/test_code \
fiveembeddev/riscv_xpack_gcc_dev_env:latest \
cmake \
-S test_code \
-B build \
-G "Unix Makefiles" \
-DCMAKE_TOOLCHAIN_FILE=../test_code/riscv.cmake
docker run \
--rm \
-v .:/project \
-v /home/five/five-embeddev-wsl/build-and-verify/examples/build-run-sim/../test-code-c:/project/test_code \
fiveembeddev/riscv_xpack_gcc_dev_env:latest \
make \
VERBOSE=1 \
-C build
GTKWave Filters for Debugging VCD Traces of RISC-V Software
April 21, 2024|Code (toolchain,vcd,simulation,debugging,tracing,baremetal,C,gtkwave)
GTKWave can be used for firmware debugging. This is obviously useful for co-simulation, but is also useful for debugging timing and integration problems.
In a previous post I showed how to create a vcd trace of a firmware run via a fork of the RISC-V ISA simulator.
In this post using https://github.com/five-embeddev/riscv-gtkwave I add:
- Instruction Decoding according to the Spike disassembler.
- Bus Address decoding according to an elf file.
- CSR register decoding.
There are a few methods to decode via GTKWave. The decoders in https://github.com/five-embeddev/riscv-gtkwave are using 3 methods:
- Generated Translate filter file.
- Translate Filter process.
- Signal Grouping and Vector Expansion and Combination
RISC-V ISA Instruction Quick Reference Update
March 03, 2024|Updates (riscv.org,spec,html)
The Instructions Quickref has been updated to match the changes made for Compiler Explorer.
The opcode references are generated from opcodes.yaml which is generated by convert_opcodes.rb in https://github.com/five-embeddev/riscv-docs-html/tree/gh_pages.
RISC-V ISA Reference for Compiler Explorer
November 15, 2023|Updates (riscv.org,spec,html)
Compiler explorer now includes RISC-V opcode references for the rv64 targets, following my pull pull request and initial work by Siyao.
The updated files are:
- etc/scripts/docenizers/docenizer-riscv64.py: Generator, converts
yaml
tots
. - lib/asm-docs/generated/asm-docs-riscv64.ts: The generated opcode documentation.
The opcode references are generated from opcodes.yaml which is generated by convert_opcodes.rb in https://github.com/five-embeddev/riscv-docs-html/tree/gh_pages.
The opcode decriptions are automatically extracted from the ISA user manual in HTML format. For example the description for sd is extracted from the Load and Store Instructions section.
To generate the data run these commands in the compiler explorer repo:
cd compiler-explorer/etc/scripts/docenizers/
./docenizer-riscv64.py \
-i https://five-embeddev.github.io/riscv-docs-html/opcodes.yaml \
-o ../../../lib/asm-docs/generated/asm-docs-riscv64.ts
RISC-V ISA Manuals to HTML
July 31, 2023|Updates (riscv.org,spec,html,docker)
The User ISA and Privileged ISA have been are converted to HTML from TEX on the upstream repo.
The latest upstream versions have recently been converted to asciidoc, but the legacy versions that were used to generate HTML on this site are converted from the older TEX.
In theory converting tex should be as simple as running a conversion tool such as pandoc or latex2html but without some optimization that gives quite ugly results - in particular for tables and diagrams.
The set of scripts that this blog uses to optimize the pandoc output has been uploaded to https://github.com/five-embeddev/riscv-docs-html. A Dockerfile is used to configure a build environment with pandoc and asciidoctor and the other tools used to do the conversion.
A set of static html github pages have been exported here, https://five-embeddev.github.io/riscv-docs-html/. They HTML is identical, but it lacks the side navigation menu and css style used by this site.
These scripts are also used to create the opcodes.yaml and csr.yaml data files used by the Instruction Quick Reference and CSR Quick Reference on this site.
More info on that will be posted later.
RISC-V Interrupts/Exceptions Quick Reference
May 11, 2023|Updates (interrupts,quickref)
A fix has been made to the Interrupts/Exceptions Quick Reference to clarify that mstatus.mpp is set to the the least-privileged supported mode on mret. The local version of Machine-Level ISA has been updated to the Priv-v1.12 source as this is the current version on https://riscv.org/technical/specifications/. References to the depreciated user mode interrupts and the n extension were also removed.
CSR Quick Reference Update
May 04, 2023|Updates (registers,csr,quickref)
The CSR quick reference list was updated:
- Cross references to manuals.
- Details for some fields.
RISC-V ISA Manual Update
May 03, 2023|Updates (riscv.org,spec)
The User ISA and Privileged ISA have been updated to tag Priv-1.12 in the upstream repo and re-generated as HTML.
- Updated to official release branch (Priv-1.12)
- Added links to source documents, source version.
The upstream changelog from git is:
RISC-V Debug Spec Update
April 29, 2023|Updates (riscv.org,spec)
The generated html for the “RISC-V External Debug Support” spec has been updated.
- Reverted to official release branch (0.13.2)
- Many tex->html conversion errors fixed.
- Added links to source documents, source version.
Direct Hardware Access in C
A RISC-V Example
March 20, 2023|Articles (baremetal,C,articles,interrupts,timer)
This article was also posed to Medium.
The C programming language provides a thin hardware abstraction that allows us to create low-level systems programs. However, there are still many hardware features that aren’t exposed by the programming language. How do we access hardware while programming in C?
This article covers some tricks used to write low-level code in C and build a simple bare-metal run-time environment. The target hardware is a RISC-V RV32I in machine mode. The RISC-V is a simple architecture well suited to understanding low-level programming, and a small 32-bit core is ideal for applications that benefit from bare-metal programming.
Rust CSR Access Macros
December 28, 2022|Code (toolchain,baremetal,rust)
Added CSR access macros for RISC-V to the https://github.com/five-embeddev/riscv-csr-access respository.
- riscv-csr-access:rs/riscv_csr_macros/src/riscv_csr_macros.rs : Access macros
- riscv-csr-access:templates/riscv_csr_macros.rs: Generator Template
There is a standard crate for RISC-V https://docs.rs/riscv/0.1.4/riscv/index.html . that has a more canonical implementation for accessing CSRS. The purpose of this code is a to have a template generated implementation that closely matches the C header used for other examples riscv-csr-access:include/riscv-csr.h.
Rust CSR Access Macros
December 27, 2022|Code (toolchain,baremetal,rust)
Added CSR access macros for RISC-V to the https://github.com/five-embeddev/riscv-csr-access respository.
- riscv-csr-access:rs/riscv_csr_macros/src/riscv_csr_macros.rs : Access macros
- riscv-csr-access:templates/riscv_csr_macros.rs: Generator Template
There is a standard crate for RISC-V https://docs.rs/riscv/0.1.4/riscv/index.html . that has a more canonical implementation for accessing CSRS. The purpose of this code is a to have a template generated implementation that closely matches the C header used for other examples riscv-csr-access:include/riscv-csr.h.
Testing Startup Constructors
November 19, 2022|Code (toolchain,baremetal,C,C++)
There were a few bugs in the code examples that have been fixed.
The loop in startup.c
function _startup()
that calling each entry
from __init_array_start
to __init_array_end
was incorrect in
Startup Code in C and Startup Code in
C++. It has been fixed and tested.
To test it the main.c
routine was updated to demonstrate the operation of the
contructors called in _startup()
before main()
is called.
Example for VCD Tracing of RISC-V Software
September 04, 2022|Code (toolchain,vcd,simulation,debugging,tracing,baremetal,C)
In the previous post I added a fork of the RISC-V ISA simulator. In this post you can see how it can be used.
The source code for this example is on github, here: https://github.com/five-embeddev/riscv-scratchpad/tree/master/baremetal-vcd-trace.
This small program handles the mie
, msi
, mti
and trap interrupts and updates some global variables when interrupts occur.
RISC-V Software Tracing with VCD and Spike
September 02, 2022|Toolchain (toolchain,vcd,simulation,debugging,tracing)
Viewing hardware and software interaction is much easier with a logic analyzer style trace. In almost every job I’ve had I’ve added a VCD tracer to the instruction set simulator. Both the “big picture” state flow can be seen, and the “little details” of bus/register/interrupt/io interaction can be seen.
For that purpose I’ve forked the RISC-V ISA Simulator, https://github.com/five-embeddev/riscv-isa-sim/tree/vcd_trace.
About the fork
This fork adds a few features.
- Tracing of registers and memory bus.
- Time limit for simulation.
- New commands to trace variables.
- Two way symbol <-> address lookup.
This is an example of a firmware writing to the mtimecmp
register:
Baremetal Example Updates
September 01, 2022|Code (toolchain,baremetal,C,csr,registers,cmake)
Updates have been made to the baremetal examples.
CMake Updates
- The CMake file can now find
riscv-none-elf-gcc
as the xpack GCC executable has been renamed toriscv-none-elf-*
. - Recent GCCs/binutils require the
zicsr
extension to be specified in-march
to use CSR instructions.
Code Updates
- Updated CSR access templates to test for
zicsr
extension. - Updated the CSR access header for the Baremetal C examples.
- Fixed the
CMakeLists.txt
files to include-march=rv32imac_zicsr
. - Updated the
timer.h
to allow USEC/MSEC intervals and different timer frequencies. - Added the
baremetal-vcd-trace
example.
RISC-V ISA Update
August 18, 2022|Updates (riscv.org,spec,quickref)
A few upates:
- Update ISA & extensions to many recent updated extensions.
- Update to latest Vector Spec and Bit-manipulation Spec.
Simulating and Debugging with a Docker Containerized Toolchain
March 25, 2022|Toolchain (toolchain,docker,simulation,debugging)
Containerized Development
In a previous post I used docker to containerize building for a RISC-V target.
In this post I’ll simulate that target file in a containerized RISC-V ISA Simulator.
Spike ISA Sim
The official reference ISA simulator for RISC-V is spike. There are other more functional and performant open source and commercial simulators , however the aim of this post is to describe a solution for quick and easy debugging and simulation of the low level examples on this blog.
Spike implements a functional model of the RISC-V hart(s) and the debug interface. GDB must connect to OpenOCD, which will in turn connect to Spike’s model of the RISC-V serial debug interface. Spike’s Readme file describes how to do this, but it’s pretty tedious to load an example program.
The container example downloads and builds the appropriate source code and manages running the independent tools.
Docker Images
The docker files are in https://github.com/five-embeddev/build-and-verify. Assume it’s checked out to build-and-verify
.
Stack and Global Pointer Quick Reference
March 06, 2022|Isa (registers,quickref)
Added some notes on the global pointer and stack pointer.
Example Bugs!
March 06, 2022|Code (toolchain,baremetal,C)
There were a few bugs in the code examples that have been fixed.
- Global pointer needs to be written with
norelax
option set. (See quickref: Global Pointer) - For C++ the argument order to
std::copy()
was incorrect. (it matchedmemcpy()
… target, source)
Building with a Docker container Toolchain
March 01, 2022|Toolchain (toolchain,docker)
Containerized Development
Modern development is moving towards packaging tools in containers. There are several benefits to containerizing development tools:
- A simpler tool deployment process, the binary image with all dependencies can simply be pulled from a server and run.
- An automated and consistent deployment process that can be shared between development machines and cloud based build servers.
- A consistent set of tools for all team members.
- The ability to build with General Purpose IDE’s rather than whatever tools a device vendor may provide.
- Isolation of dependencies used for different devices and build targets.
For RISC-V there are a few more benefits:
- A “recipe” to consistently build the vendor neutral tools at, such as https://github.com/riscv-collab/riscv-gnu-toolchain.
- New versions of the tools with support for new extensions etc can be easily deployed.
I’ve put together a set of Docker images and Docker Compose build and run configurations with the aim of deploying them with GitHub’s workflows.
They are located here: https://github.com/five-embeddev/build-and-verify
Extending PlatformIO
November 05, 2021|Toolchain (articles,toolchain,C)
Information about extending cross compilation with PlatformIO has been added.
Baremetal Vectored Interrupts in C for RISC-V
September 25, 2021|Code (articles,baremetal,C)
An example of using vectored interrupts in C.
Baremetal Timer Driver in C for RISC-V
September 24, 2021|Code (articles,baremetal,C)
An example of a timer driver in C has been added.
Baremetal Startup in C for RISC-V
September 24, 2021|Code (articles,baremetal,C)
An example of a startup code in C.
Machine Readable Specification Data
September 10, 2021|Toolchain (registers,spec,interrupts,opcodes)
As RISC-V is a new architecture so there will be new development at all layers of the software and hardware stack. Rather than write code based on human language specifications from scratch, a smarter way to work can be to translate a machine readable specification to code.
“Machine Readable” does not need to be an all encompassing formal model of the architecture, there are many convenient formats such as csv, yaml, xml and json that can be parsed and transformed using the packages available in most scripting languages.
Baremetal Timer Driver in C++ for RISC-V
August 13, 2021|Code (articles,baremetal,C++)
An example of a timer driver.
Baremetal Interrupt in C++ for RISC-V
August 13, 2021|Code (articles,baremetal,C++)
An example of using interrupts.
Baremetal Programming in C++ for RISC-V
July 18, 2021|Code (toolchain,medium,baremetal,C++)
I’ve put together a series of posts on Medium covering bare-metal RISC-V development and C++.
- An overview of a simple bare-metal blinky application for the SiFive HiFive Rev-B
- Intro to PlatformIO for RISC-V
- Zero to
main()
for RISC-V, in (almost) pure C++. - The CSR system registers of RISC-V and zero cost C++ abstractions.
- The RISC-V machine mode timer and timing keeping using the C++ std::chrono library.
- The basics of RISC-V interrupt handling, and C++ lambda functions.
(These are also included in this blog.)
A Baremetal Introduction using C++. Interrupt Handling.
May 06, 2021|Articles (baremetal,C++,interrupts)
What are the basics of interrupt handing in RISC-V? Can we utilize modern C++ to simplify the interrupt handling?
A Baremetal Introduction using C++. Machine Mode Timer.
May 05, 2021|Articles (baremetal,C++,timer)
This is the sixth post in a
series,
about the RISC-V machine mode timer and timing keeping using the C++
std::chrono
library.
A Baremetal Introduction using C++. System Registers.
May 03, 2021|Articles (baremetal,C++,csr)
What are system registers in RISC-V? How can we access them with modern C++?
System registers require special instructions to access, so unlike memory mapped registers (MMIO) we can’t just cast a pointer to memory to get access them in C++.
A Baremetal Introduction using C++. Startup.
May 03, 2021|Articles (baremetal,C++,startup)
In the last post, we set up the development environment. This post is about how the RISC-V core executes our program.
How do we go from reset to entering the main()
function in C++ in
RISC-V? Startup code is generally not something you need to worry about,
however, it is of interest when bringing up a device from scratch.
A Baremetal Introduction using C++. Development Environment
April 30, 2021|Articles (baremetal,C++,toolchain)
For this series of posts, my platform is a SiFive HiFive1 Rev B development board. It’s equipped with a 320MHz RV32IMAC (FE310 core). For software build and debug the choice is Platform IO, an IDE integrated into VS Code.
A Baremetal Introduction using C++. Overview.
April 30, 2021|Articles (baremetal,C++)
As described in Part1, a simple C++ application to blink an LED, what does this look like with no operating system?
A Baremetal Introduction using C++. Introduction.
April 30, 2021|Articles (baremetal,C++)
This is a series of posts where I’d like to combine those topics for embedded systems programming. RISC-V and C++ have been evolving rapidly, and you will see modern C++ is a great way to explore RISC-V and develop embedded systems.
Baremetal Startup Code in C++ for RISC-V
April 13, 2021|Code (toolchain,baremetal,C++)
An example of baremetal bootstrap code for RISC-V in C++.
CMake Cross Compilation for RISC-V Targets
March 20, 2021|Toolchain (articles,toolchain,baremetal,cmake)
An example of cross compiling a baremetal program to RISC-V with CMake.
RISC-V Compile Targets, GCC
February 09, 2021|Toolchain (gcc,base_isa,extensions,abi)
Note to self: When compiling the riscv-toolchain for embedded systems, set the configure options!
The toolchain can be cloned from the RISC-V official github. Once the dependencies are installed it’s straight forward to compile.
RISC-V CSR Access
November 18, 2020|Code (registers,code,baremetal,quickref)
For baremetal programming I’ll often need to access CSRs, e.g. mstatus.mie for critical sections, mcause in interrupts handlers, etc. Defining function wrappers for accessing these registers creates easier to understand code, however writing these wrappers is pretty tedious.
The quick reference on this blog is generated from a YAML description (csr.yaml). Using that with a web template engine script (generators/yaml_jinja.py) and a template (templates/riscv-csr.h) code can be easily generated.
RISC-V ISA Update
October 11, 2020|Updates (riscv.org,spec)
The User ISA and Privileged ISA have been updated to tag draft-20201007-16f5002 in the upstream repo and re-generated as HTML.
The upstream changelog from git is:
RISC-V Draft Bitmanip Spec
September 27, 2020|Updates (riscv.org,spec)
Compiled to HTML from https://github.com/riscv/riscv-bitmanip.
RISC-V Instructions Quick Reference
May 16, 2020|Updates (isa,quickref)
Added details on how to call instructions from C, listed the CSR instructions, and linked the instruction groups.
RISC-V CSRs Quick Reference
May 16, 2020|Isa (csr,quickref)
Added details on how to access CSRs from C.
RISC-V CSRs Quick Reference
May 03, 2020|Updates (registers,quickref)
Updated CSR quick reference page
- Linked debug registers.
- Made table sortable.
- Added feature/extension classification.
RISC-V External Debug Spec
May 03, 2020|Updates (riscv.org,spec)
Compiled to HTML from https://github.com/riscv/riscv-debug-spec.git tex.
RISC-V Instructions/Extensions Quick Reference
January 21, 2020|Isa (registers,toolchain,quickref)
A few more quick reference pages:
- summarize ISA & extensions and
- an initial instruction list quick reference.
RISC-V Draft Vector Spec
January 04, 2020|Updates (riscv.org,spec)
Compiled to HTML from https://github.com/riscv/riscv-v-spec.
RISC-V ISA Update
January 02, 2020|Updates (riscv.org,spec)
The User ISA and Privileged ISA have been updated to tag draft-20191228-a6c204f in the upstream repo and re-generated as HTML.
The upstream changelog from git is:
RISC-V Tools Quick Reference
October 29, 2019|Toolchain (toolchain,quickref)
An initial toolchain quick reference.
RISC-V Registers Quick Reference
October 29, 2019|Isa (registers,quickref)
A few more quick reference pages:
- summarize registers
- CSRs and
- GPRs with the ABIs.
RISC-V Interrupts/Exceptions Quick Reference
August 26, 2019|Isa (interrupts,quickref)
I’ve made an attempt to understand and summarize RISC-V interrupts and exceptions. It mostly covers the machine mode interrupt model, entry and exit procedure. It also looks at supervisor mode, user mode and exceptions.
RISC-V ISA Update
August 22, 2019|Updates (riscv.org,spec)
The User ISA and Privileged ISA have been updated to tag draft-20190820-22bf021 in the upstream repo and re-generated as HTML.
This will include the ratified 1.11 spec and 1.12 draft.
The upstream changelog from git is:
RISC-V Compile Targets, GCC
June 26, 2019|Toolchain (gcc,base_isa,extensions,abi)
NOTE: Since this was written the riscv-toolchain-conventions document has been released.
Getting started with RISC-V. Compiling for the RISC-V target. This post covers the GCC machine architecture (-march), ABI (-mabi) options and how they relate to the RISC-V base ISA and extensions. It also looks at the multilib configuration for GCC.
Selecting
- the base ISA,
- the extensions, and
- the target ABIs.
About
May 15, 2019|Updates
I’ve setup this this blog to capture information I’ve found useful to develop RISC-V embedded firmware.
My experience is with bare metal RV32EC based systems. Previously I have worked with ARM Cortex-M0 and other processors, so initially I’ll capture the information needed to bootstrap such firmware and the gotchas that come from not yet thinking in RISC-V terms.
Some of the initial planned material is:
- ISA information presented as easy to reference HTML.
- Low level information for getting started with the RISC-V architecture.
- Some evaluation of how RISC-V compares to other architectures.