CVA6 and JIT domain tests - Setup for a fresh install

2023-10-19 786 words 4 minutes

Contents

Introduction

In the context of a research project with a PhD student, we have to work on the microarchitecture of a CVA6 and make it compatible with some JIT code (VMIL'23 talk). The official CVA6 repository has recently pushed new scripts to create the SDK. As we already started our work, we want to generate the SDK with the old method on a fork of the CVA6..

Repositories and environment variables

It is assumed you have downloaded two repositories in your home directory ($HOME):

https://github.com/QDucasse/jitdomain-tests, main branch.
https://github.com/QDucasse/cva6, jitdomain branch. This branch started at commit de986ed of the original CVA6 repository.

It is also assumed that you have the following environment variables:

        
export RISCV=/opt/riscv-newlib-jitdomain-test
export VERILATOR_ROOT=/opt/verilator # Location of Verilator could be changed
export PATH=$RISCV/bin:$VERILATOR_ROOT/bin:$PATH
export CVA6_REPO_DIR=$HOME/cva6

Comments:

Location of the RISC-V toolchain is hardcoded in the jitdomain-tests repository: https://github.com/QDucasse/jitdomain-tests/blob/main/patch/patch_toolchain.sh#L57.
You can change VERILATOR_ROOT where you want to install Verilator. This tutorial keeps /opt/verilator.

JITdomain-tests

Prerequisites

        
sudo apt-get install git help2man perl python3 make g++ libgz libfl2 libfl-dev zlibc zlib1g zlib1g-dev autoconf automake autotools-dev curl python3 python3-pip libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev ninja-build cmake libglib2.0-dev

On Ubuntu 20LTS, you should have g++ 9.4.0.

Cross-compiler (patched) and Spike simulator

        
        
        
    
# Toolchain
cd jitdomain-tests
./patch/patch_toolchain.sh
# Simulator
cd riscv-gnu-toolchain/spike
mkdir build
cd build
../configure --prefix=$RISCV
make
sudo make install

Comments:

patch/patch_toolchain.sh installs the toolchain only. In order to be able to simulate, we need to install the simulator as well as the RISC-V frontend server (through the fesvr library).

Related issue(s):

https://github.com/openhwgroup/cva6/issues/611.

Verilator

The recommended Verilator is v5.008: https://github.com/QDucasse/cva6/blob/jitdomain/verif/regress/install-verilator.sh#L23.
Later, we will see that we need C++ files from Verilator. As a consequence, we will install Verilator as a “run-in-place” instance: https://verilator.org/guide/latest/install.html#run-in-place-from-verilator-root (which is, by the way, the preferred method from Verilator team).

Warning

Be aware that you must apply a patch before compiling Verilator as done here: https://github.com/QDucasse/cva6/blob/jitdomain/verif/regress/install-verilator.sh#L24

        
        
        
    
diff --git a/docs/CONTRIBUTORS b/docs/CONTRIBUTORS
index 215fb6bd8..829752e6b 100644
--- a/docs/CONTRIBUTORS
+++ b/docs/CONTRIBUTORS
@@ -154,5 +154,6 @@ Yuri Victorovich
 Yutetsu TAKATSUKASA
 Yu-Sheng Lin
 Yves Mathieu
+Zbigniew Chamski
 Zhanglei Wang
 Zixi Li
diff --git a/include/verilated_types.h b/include/verilated_types.h
index cb7265e32..f1d482d8e 100644
--- a/include/verilated_types.h
+++ b/include/verilated_types.h
@@ -1012,8 +1012,8 @@ struct VlUnpacked final {
 
     // METHODS
     // Raw access
-    WData* data() { return &m_storage[0]; }
-    const WData* data() const { return &m_storage[0]; }
+    WData* data() { return (WData*)&m_storage[0]; }
+    const WData* data() const { return (const WData*)&m_storage[0]; }
 
     T_Value& operator[](size_t index) { return m_storage[index]; }
     const T_Value& operator[](size_t index) const { return m_storage[index]; }

        
        
        
    
$ cd /opt
$ git clone https://github.com/verilator/verilator
$ cd verilator
$ git checkout v5.008
$ autoconf
$ export VERILATOR_ROOT=`pwd`
$ ./configure
$ make

Comments:

        
        
        
    
$ $VERILATOR_ROOT/bin/./verilator --version
Verilator 5.008 2023-03-04 rev v5.008
# Two source files needed for VCD generation
$ ls $VERILATOR_ROOT/include |grep "verilated_fst_c.cpp\|verilated_vcd_c.cpp"
verilated_fst_c.cpp
verilated_vcd_c.cpp

CVA6 model generation (normal and debug versions)

        
$ cd $CVA6_REPO_DIR
$ git submodule update --init --recursive

Two things need to be modified:

Path to Verilator C++ files for VCD generation.
Flag to solve a timing issue in the model generation (see https://github.com/openhwgroup/cva6/issues/1162).

        
        
        
    
@@ -52,7 +52,7 @@ ifeq ($(support_verilator_4), 0)
 	verilator_threads := 1
 endif
 # Location of Verilator headers and optional source files
-VL_INC_DIR := $(VERILATOR_INSTALL_DIR)/share/verilator/include
+VL_INC_DIR := $(VERILATOR_ROOT)/include
 
 ifndef RISCV
 $(error RISCV not set - please point your RISCV variable to your RISCV installation)
@@ -535,6 +535,7 @@ verilate_command := $(verilator) verilator_config.vlt
                     +incdir+corev_apu/axi_node                                                                   \
                     $(if $(verilator_threads), --threads $(verilator_threads))                                   \
                     --unroll-count 256                                                                           \
+					--no-timing 																				 \
                     -Wall                                                                                        \
                     -Werror-PINMISSING                                                                           \
                     -Werror-IMPLICIT                                                                             \
@@ -550,7 +551,7 @@ verilate_command := $(verilator) verilator_config.vlt
                     $(if $(PROFILE),--stats --stats-vars --profile-cfuncs,)                                      \
                     $(if $(DEBUG), --trace-structs,)                                                             \
                     $(if $(TRACE_COMPACT), --trace-fst $(VL_INC_DIR)/verilated_fst_c.cpp)                        \
-                    $(if $(TRACE_FAST), --trace $(VL_INC_DIR)/include/verilated_vcd_c.cpp)                       \
+                    $(if $(TRACE_FAST), --trace $(VL_INC_DIR)/verilated_vcd_c.cpp)                       \
                     -LDFLAGS "-L$(RISCV)/lib -L$(SPIKE_INSTALL_DIR)/lib -Wl,-rpath,$(RISCV)/lib -Wl,-rpath,$(SPIKE_INSTALL_DIR)/lib -lfesvr$(if $(PROFILE), -g -pg,) -lpthread $(if $(TRACE_COMPACT), -lz,)" \
                     -CFLAGS "$(CFLAGS)$(if $(PROFILE), -g -pg,) -DVL_DEBUG"                                      \
                     --cc  --vpi                                                                                  \

Models generation and execution

        
        
        
    
# Normal model
make clean
make verilate
./work-ver/Variane_testharness <jitdomain_binary.elf>

# Model with VCD feature
make clean
make verilate DEBUG=1 TRACE_FAST=1
./work-ver/Variane_testharness -v <output_filename.vcd> <jitdomain_binary.elf>

TLDR (only for compilation/installation of all tools)

        
        
        
    
$ cd $HOME
$ git clone https://github.com/QDucasse/jitdomain-tests
$ git clone https://github.com/QDucasse/cva6
$ export RISCV=/opt/riscv-newlib-jitdomain-test
$ export VERILATOR_ROOT=/opt/verilator
$ export PATH=$RISCV/bin:$VERILATOR_ROOT/bin:$PATH
$ export CVA6_REPO_DIR=$HOME/cva6
$ sudo apt-get install git help2man perl python3 make g++ libgz libfl2 libfl-dev zlibc zlib1g zlib1g-dev autoconf automake autotools-dev curl python3 python3-pip libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev ninja-build cmake libglib2.0-dev
$ cd jitdomain-tests
$ ./patch/patch_toolchain.sh
$ cd riscv-gnu-toolchain/spike
$ mkdir build
$ cd build
$ ../configure --prefix=$RISCV
$ make
$ sudo make install
$ cd /opt
$ git clone https://github.com/verilator/verilator
$ cd verilator
$ git checkout v5.008
$ autoconf
$ export VERILATOR_ROOT=`pwd`
$ ./configure
$ make
$ cd $CVA6_REPO_DIR
$ git submodule update --init --recursive
# Fix the makefile as shown before