# How to add a processor in LiteX?


## Introduction
Two methods can be done:
- Including the processor code in LiteX framework (see CV32E40P [[4]]).
- Importing the processor code from an external repository (see FemtoRV [[5]]).

The choice mainly depends on how the HDL code is written. This tutorial focuses on the first method as targeted processors will be composed of several HDL files. We will see later how it is configured.

## LiteX workflow for custom processors
This tutorial is based on the work done by Antmicro for the CV32E40P [[4]].

{{< admonition type=warning title="Warning" open=false >}}
Be aware that the CV32E40P included in LiteX is based on a [fork not updated since May 2020](https://github.com/antmicro/cv32e40p). Moving to the upstream design is an open question since [February 2022](https://github.com/litex-hub/pythondata-auto/issues/29).
{{< /admonition >}}

### LiteX configuration for the CV32E40P
Adding a new processor can be done in 4 steps as for [[4]].
1. [Adding the CPU in LiteX CPU catalog](https://github.com/litex-hub/pythondata-auto/commit/9525e102f901a64b6e2f9ec6b95a115b54c7acfd).
2. [Adding the Python CPU wrapper address in `litex_setup.py`](https://github.com/enjoy-digital/litex/pull/535/commits/2903b1bf807f4ea887a62b947662df4c5179edb1).
3. [Adding support for the CPU in the LiteX BIOS](https://github.com/enjoy-digital/litex/pull/535/commits/ca8cb834245929769d947ab25b9daab2aa9a88ce).
4. [Adding CPU specific code in the software + SoC builder scripts](https://github.com/enjoy-digital/litex/pull/535/commits/2d6ee5aaf21cb16f8d007df4ce62b324118250ec).

Following section explain the integration flow in details for another processor.

## Case study - Integration of the CV32E41P [[1]]
The main goal of this section is to describe **precisely** steps needed to include a new processor in the LiteX framework. The OpenHwGroup CV32E41P has been chosen as it has a structure similar to the upstream CV32E40P.

### CV32E41P architecture
#### Top-level schematic
![cv32e41p](https://raw.githubusercontent.com/openhwgroup/cv32e41p/master/docs/images/CV32E40P_Block_Diagram.png)

CV32E41P and CV32E40P are represented by the same block diagram in OpenHwGroup documentations.

#### Interrupt controller

Document about interrupts are given at [[6]] and [[7]] (for instance, sections 3.1.6.1 and 3.1.9).

Things to be aware of:

> The `irq_i[31:0]` interrupts are controlled via the `mstatus`, `mie` and `mip` CSRs. CV32E41P uses the upper 16 bits of `mie` and `mip` for custom interrupts (`irq_i[31:16]`), which reflects an intended custom extension in the RISC-V Basic (a.k.a. CLINT) interrupt architecture.

Here is the structure for the `mip` register (cf. 3.1.9 of the RISC-V privileged specification):

![mip](../img/interrupt.png)

- MEIP/SEIP are for external interrupts.
- MTIP/STIP for timer interrupts.
- MSIP/SSIP for software interrupts.

There is a similar structure for the `mie` containing enable bits for those interrupts.

### LiteX configuration for the CV32E41P [[3]]
#### Adding the CPU in LiteX CPU catalog

The goal is to create the wrapper between the HDL code and Python. Fortunately, this step can be done in a [single commit](https://github.com/litex-hub/pythondata-auto/pull/27/commits/8bd8b7c2bb59173776afa4a92d5385765143def7).

```diff
+ [cv32e41p]
+ type = cpu
+ human_name = CV32E41P
+ src = https://github.com/openhwgroup/cv32e41p
+ contents = system_verilog
+ license = License :: OSI Approved :: Apache Software License
+ license_spdx = Apache-2.0
```

Afterwards, one of the maintainers will create the https://github.com/litex-hub/pythondata-cpu-cv32e41p repository:

- It includes a list of SystemVerilog needed for the Python SoC builder: https://github.com/litex-hub/pythondata-cpu-cv32e41p/blob/master/pythondata_cpu_cv32e41p/system_verilog/cv32e41p_manifest.flist.

- The repository includes a copy of the HDL source which is updated on a regular basis.

#### Adding the Python CPU wrapper address in `litex_setup.py`
In this step, we just had the URL of the previously generated wrapper into the main LiteX repository:

```diff
"pythondata-cpu-cv32e40p":     GitRepo(url="https://github.com/litex-hub/", clone="recursive"),
+ "pythondata-cpu-cv32e41p":     GitRepo(url="https://github.com/litex-hub/", clone="recursive"),
```

#### Adding support for the CPU in the LiteX BIOS
Here we have to add some code in order to handle interrupts. `mcause` is the "Machine Cause Register":

![](../img/mcause.png)

The goal is to catch an exception coming from the UART.

- `unsigned int cause = csrr(mcause) & IRQ_MASK` gives the exception code field from `mcause`.
- LiteX uses fast interrupts instead of existing external/timer/software interrupts. As a consequence, for an UART interrupt, the ISR needs `UART_INTERRUPT+FIRQ_OFFSET` to get the first fast interrupt.
- It will finally call the existing [UART ISR](https://github.com/enjoy-digital/litex/blob/c299a8296806f30def5496ada4888e2ed213c5cf/litex/soc/software/libbase/uart.c#L31).

#### Adding CPU specific code in the software + SoC builder scripts
The most important step as it includes the top-level of the CPU in Python and add other software elements. Changes are described file by file. The pull request for the CV32E41P is at: https://github.com/enjoy-digital/litex/pull/1207/commits/d4448827ac109454acb0427a4e204a1b27b98849

##### cores/cpu/cv32e41p/__init__.py

```python
from litex.soc.cores.cpu.cv32e41p.core import CV32E41P
```

##### cores/cpu/cv32e41p/boot-helper.S

```asm
.section    .text, "ax", @progbits
.global     boot_helper
boot_helper:
	jr x13
```

##### cores/cpu/cv32e41p/core.py

This file may be the most important one as it describes the top-level of the CPU in Python.

- `GCC_FLAGS` : the CV32E41P is a [RV32IMC](https://github.com/openhwgroup/cv32e41p) (the floating point unit isn't implemented by default).
- OPI/APB: LiteX uses the OBI/APB interfaces to connect to the Wishbone bus. Part related to buses and bridges were copy-pasted from existing CPUs.
- `add_manifest_sources`: it will list SystemVerilog files given at: https://github.com/litex-hub/pythondata-cpu-cv32e41p/blob/master/pythondata_cpu_cv32e41p/system_verilog/cv32e41p_manifest.flist
- `class CV32E41P(CPU)`: the Python class describing the processor. This class is quite easy to understand when we compare it with the [top-level SystemVerilog file](https://github.com/openhwgroup/cv32e41p/blob/master/rtl/cv32e41p_core.sv):
  - **Only input ports are given in this file**. Not for instruction/data buses. Example:
    - `i_pulp_clock_en_i` is related to `pulp_clock_en_i` in [SystemVerilog](https://github.com/openhwgroup/cv32e41p/blob/master/rtl/cv32e41p_core.sv#L48).
    - Syntax: a prefix `i_` is added on the SystemVerilog port name for inputs.
  - `i_irq_i = Cat(self.interrupt_padding,self.interrupt)` was a trick due to the structure of the interrupt controller in the CV32E41P.
    - https://github.com/openhwgroup/cv32e41p/blob/master/rtl/cv32e41p_int_controller.sv
    - https://docs.openhwgroup.org/projects/openhw-group-cv32e41p/exceptions_interrupts.html (for fast interrupts).
    - 16 fast interrupt lines + a padding of 16 to match the 32-bit width of the  port in the interrupt controller.
  - Finally, the developer has to take care of values given for inputs (as for HDL testbenches). A deep reading of CPU specifications may be needed here.

##### cores/cpu/cv32e41p/crt0.S

This file is a set of startup routines. In the vector table, interrupt numbers can be found in the CPU specifications. For instance, in the Machine Interrupt Enable Register: https://docs.openhwgroup.org/projects/openhw-group-cv32e41p/control_status_registers.html#machine-interrupt-enable-register-mie

| Bit # | Mode | Description                                                  |
| ----- | ---- | ------------------------------------------------------------ |
| 31:16 | RW   | Machine Fast Interrupt Enables: Set bit x to enable interrupt irq_i[x]. |
| 11    | RW   | **Machine External Interrupt Enable (MEIE)**: If set, irq_i[11] is enabled. |
| 7     | RW   | **Machine Timer Interrupt Enable (MTIE)**: If set, irq_i[7] is enabled. |
| 3     | RW   | **Machine Software Interrupt Enable (MSIE)**: if set, irq_i[3] is enabled. |

```asm
vector_table:
[...]
  j trap_entry # 3 software
[...]
  j trap_entry # 7 timer
[...]
  j trap_entry # 11 external
[...]
  j trap_entry # 16 firq0
[...]
```

##### cores/cpu/cv32e41p/csr-defs.h

Some constants related to CSR. For instance, `#define CSR_MSTATUS_MIE 0x8` is related to the position of the `mie` bit in the `mstatus` CSR:

![mstatus](../img/mstatus_mie.png)

##### cores/cpu/cv32e41p/irq.h

API to control interrupts. Copy-pasted from existing CPUs.

##### cores/cpu/cv32e41p/system.h

API to control interrupts. Copy-pasted from existing CPUs.

## References

1. CV32E41P documentation. [https://docs.openhwgroup.org/projects/openhw-group-cv32e41p/index.html][1]
2. CV32E41P HDL repository. [https://github.com/openhwgroup/cv32e41p][2]
3. CV32E41P pull request in LiteX. [https://github.com/enjoy-digital/litex/pull/1207][3]
4. CV32E40P pull request in LiteX. [https://github.com/enjoy-digital/litex/pull/535][4]
5. FemtoRV support in LiteX. [https://github.com/enjoy-digital/litex/commit/b1d8fe61f8613babeb486877a094bff759e31fd0][5] 
6. CORE-V CV32E41P User Manual - Exceptions and Interrupts. [https://docs.openhwgroup.org/projects/openhw-group-cv32e41p/exceptions_interrupts.html][6]
7. Andrew Waterman, Krste Asanovic, John Hauser.The RISC-V Instruction Set Manual - Volume II: Privileged Architecture. [https://github.com/riscv/riscv-isa-manual/releases/download/Priv-v1.12/riscv-privileged-20211203.pdf][7]

[1]: https://docs.openhwgroup.org/projects/openhw-group-cv32e41p/index.html
[2]: https://github.com/openhwgroup/cv32e41p
[3]: https://github.com/enjoy-digital/litex/pull/1300
[4]: https://github.com/enjoy-digital/litex/pull/535
[5]: https://github.com/enjoy-digital/litex/commit/b1d8fe61f8613babeb486877a094bff759e31fd0
[6]: https://docs.openhwgroup.org/projects/openhw-group-cv32e41p/exceptions_interrupts.html
[7]: https://github.com/riscv/riscv-isa-manual/releases/download/Priv-v1.12/riscv-privileged-20211203.pdf