Summary Exploring Single and Multi-Core Vector Processing arxiv.org
12,779 words - PDF document - View PDF document
One Line
The open-source vector processor Ara2, based on RISC-V V 1.0, demonstrates high utilization and energy efficiency while focusing on multi-core processors and addressing the need for architectural performance studies.
Slides
Slide Presentation (15 slides)
Key Points
- Ara2 is the first fully open-source vector processor that supports the RISC-V V 1.0 frozen ISA.
- Ara2 achieves a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and 1.35GHz of clock frequency.
- Multi-core vector processors can overcome the scalar core issue-rate bound and improve performance and energy efficiency.
- The RISC-V V extension (RVV) has gained significant interest due to its openness and extensibility.
- The paper provides insights into the performance and energy efficiency trade-offs of different design parameters and system configurations.
- Ara2 achieves high performance and energy efficiency on a diverse set of data-parallel kernels.
- The architecture incorporates a lightweight Slide Unit (SLDU) that reduces interconnect wiring and hardware.
- Multi-core systems outperform single-core architectures with the same overall computation capability.
Summaries
28 word summary
Ara2, an open-source vector processor based on RISC-V V 1.0, achieves high utilization and energy efficiency. It explores multi-core processors and addresses the lack of architectural performance studies.
66 word summary
Ara2, the first open-source vector processor supporting RISC-V V 1.0 frozen ISA, achieves high functional-unit utilization and energy efficiency. Implemented in 22nm technology, it operates at a clock frequency of 1.35GHz. The paper explores multi-core vector processors, highlighting their ability to improve performance and energy efficiency. Ara2 offers insights into design parameters and system configurations, addressing the lack of detailed architectural performance studies in this area.
125 word summary
Ara2 is introduced as the first fully open-source vector processor that supports the RISC-V V 1.0 frozen ISA. It achieves a high functional-unit utilization of 95% on data-parallel kernels and demonstrates a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and 1.35GHz of clock frequency. The paper explores the trade-offs of multi-core vector processors and highlights the ability of multiple vector cores to improve performance and energy efficiency. Ara2 is implemented in a 22nm technology and achieves a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and a clock frequency of 1.35GHz. The paper compares Ara2 with other RISC-V-based vector processors and addresses the lack of detailed architectural performance studies on design parameters and system configurations. Ara2 provides insights into the trade-offs of different design parameters and system configurations.
450 word summary
Ara2 is introduced as the first fully open-source vector processor that supports the RISC-V V 1.0 frozen ISA. It achieves a high functional-unit utilization of 95% on data-parallel kernels and demonstrates a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and 1.35GHz of clock frequency.
The paper explores the trade-offs of multi-core vector processors and highlights the ability of multiple vector cores to overcome the limitations of scalar cores and improve performance and energy efficiency. Vector processing has been effective in enhancing processor performance and efficiency for data-parallel workloads since the Cray-1 supercomputer in 1976.
The RISC-V V extension, RVV, has gained significant interest due to its openness and extensibility. Ara2 is the first open-source vector processor to support the RVV 1.0 specification. It analyzes the impact of the frozen RVV specification on the microarchitecture of a vector processor and provides an in-depth analysis of performance based on the application vector length.
Ara2 is implemented in a 22nm technology and achieves a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and a clock frequency of 1.35GHz. The paper analyzes the scalability of the vector architecture by studying the microarchitecture optimizations of the Slide Unit (SLDU), which is the most critical unit. It also investigates the impact of different hardware configurations on performance and energy efficiency.
The paper compares Ara2 with other RISC-V-based vector processors and highlights the lack of detailed architectural performance studies on design parameters and system configurations. Ara2 addresses this gap by providing insights into the trade-offs of different design parameters and system configurations.
The paper is structured to provide an introduction to the importance of energy efficiency, discuss the evolution of vector processors and the RVV extension, and present the Ara2 architecture. It describes the experiment setup, including performance analysis, physical implementation analysis, and multi-core analysis. The benchmark kernels used for evaluation are outlined, and their performance behavior is discussed.
In conclusion, Ara2 is the first fully open-source vector processor that supports the RVV 1.0 specification. It achieves high performance and energy efficiency on data-parallel kernels and provides valuable insights into the trade-offs of single- and multi-core vector processors.
Ara2 is an open-source RISC-V vector architecture implemented in 22nm FD-SOI technology. It incorporates a lightweight Slide Unit (SLDU) that reduces interconnect wiring and hardware by 70%. Compliance with the RVV specification requires the addition of a new Mask Unit (MASKU), which complicates the physical implementation.
The performance evaluation of Ara2 shows that it achieves over 50% of its throughput ideality starting from 128 Bytes per lane on average across all benchmark applications. The physical implementation of Ara2 in 22nm FD-SOI technology shows an increase in operational frequency compared to the previous design. The optimized Slide Unit (SLDU) reduces area by 83%.
614 word summary
Ara2 is introduced as the first fully open-source vector processor that supports the RISC-V V 1.0 frozen ISA. It achieves a high functional-unit utilization of 95% on data-parallel kernels and demonstrates a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and 1.35GHz of clock frequency. The paper explores the trade-offs of multi-core vector processors and highlights the ability of multiple vector cores to overcome the limitations of scalar cores and improve performance and energy efficiency.
Vector processing has been effective in enhancing processor performance and efficiency for data-parallel workloads since the Cray-1 supercomputer in 1976. However, current computing systems struggle to meet the demands of AI and ML workloads due to the growing volume of data. Energy efficiency has become crucial across all performance profiles.
The RISC-V V extension, RVV, has gained significant interest due to its openness and extensibility. Ara2 is the first open-source vector processor to support the RVV 1.0 specification. It analyzes the impact of the frozen RVV specification on the microarchitecture of a vector processor and provides an in-depth analysis of performance based on the application vector length.
Ara2 is implemented in a 22nm technology and achieves a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and a clock frequency of 1.35GHz. The paper analyzes the scalability of the vector architecture by studying the microarchitecture optimizations of the Slide Unit (SLDU), which is the most critical unit. It also investigates the impact of different hardware configurations, such as the number of vector cores and lanes per vector core, on performance and energy efficiency.
The paper compares Ara2 with other RISC-V-based vector processors and highlights the lack of detailed architectural performance studies on design parameters and system configurations. Ara2 addresses this gap by providing insights into the trade-offs of different design parameters and system configurations.
The paper is structured to provide an introduction to the importance of energy efficiency, discuss the evolution of vector processors and the RVV extension, and present the Ara2 architecture. It describes the experiment setup, including performance analysis, physical implementation analysis, and multi-core analysis. The benchmark kernels used for evaluation are outlined, and their performance behavior is discussed.
In conclusion, Ara2 is the first fully open-source vector processor that supports the RVV 1.0 specification. It achieves high performance and energy efficiency on data-parallel kernels and provides valuable insights into the trade-offs of single- and multi-core vector processors. The results contribute to the understanding of vector architecture design parameters and system configurations.
Ara2 is an open-source RISC-V vector architecture implemented in 22nm FD-SOI technology. It incorporates a lightweight Slide Unit (SLDU) that reduces interconnect wiring and hardware by 70%. Compliance with the RVV specification requires the addition of a new Mask Unit (MASKU), which complicates the physical implementation.
The performance evaluation of Ara2 shows that it achieves over 50% of its throughput ideality starting from 128 Bytes per lane on average across all benchmark applications. It also achieves over 75% of its maximum throughput on crucial kernels like matrix multiplications and convolutions from 128 Bytes per lane. The performance scalability of Ara2 is influenced by the ratio between the application vector length and the number of lanes.
Replacing the scalar core with an ideal dispatcher improves the performance ideality of Ara2 by removing issue-rate limitations and interference with memory transfers. The raw throughput ideality increases when the scalar core is replaced by an ideal dispatcher, especially for shorter vectors.
The physical implementation of Ara2 in 22nm FD-SOI technology shows an increase in operational frequency compared to the previous design. The area of Ara2 is higher due to the newly introduced functionality and a reduction in the size of the Vector Register File (VRF). The optimized Slide Unit (SLDU) reduces area by 83
1122 word summary
The paper presents Ara2, the first fully open-source vector processor that supports the RISC-V V 1.0 frozen ISA. The performance of Ara2 is evaluated on a diverse set of data-parallel kernels, achieving an average functional-unit utilization of 95% on the most computationally intensive kernels. Performance boosters and bottlenecks, including the scalar core, memories, and vector architecture, are identified and analyzed. Ara2 achieves a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and 1.35GHz of clock frequency. The performance and energy-efficiency trade-offs of multi-core vector processors are explored, demonstrating that multiple vector cores can overcome the scalar core issue-rate bound and improve performance and energy efficiency.
Vector processing has been effective in boosting processor performance and efficiency for data-parallel workloads since the Cray-1 supercomputer achieved top performance in 1976. However, today's computing systems struggle to meet the performance requirements of AI and ML workloads due to the increasing amount of data to process. Energy efficiency has become crucial for computing systems across all performance profiles.
The RISC-V V extension, RVV, has gained significant interest due to its openness and extensibility. Ara2 is the first open-source vector processor to support the RVV 1.0 specification. It analyzes the frozen RVV specification and its impact on the microarchitecture of a vector processor. The Ara2 microarchitecture and design are presented, along with an in-depth analysis of performance depending on the application vector length on benchmark kernels from different application domains.
Ara2 is implemented in a 22nm technology and achieves a state-of-the-art energy efficiency of 37.8 DP-GFLOPS/W and a clock frequency of 1.35GHz. The microarchitecture optimizations of the most critical unit, the Slide Unit (SLDU), are analyzed for scalability of the vector architecture. The impact of different hardware configurations, including the number of vector cores and lanes per vector core, on performance and energy efficiency is studied. It is demonstrated that a multi-core vector architecture can overcome the performance limitations of the scalar core and achieve improved energy efficiency.
The paper also compares Ara2 with other RISC-V-based vector processors and identifies the lack of detailed architectural performance studies on the effects of design parameters and system configurations. The Ara2 design addresses this gap by providing insights into the performance and energy efficiency trade-offs of different design parameters and system configurations.
The paper is structured as follows: it provides an introduction to the importance of energy efficiency in computing systems, discusses the evolution of vector processors and the RISC-V V extension, and presents the Ara2 architecture. The experiment setup, including performance analysis, physical implementation analysis, and multi-core analysis, is described in detail. The benchmark kernels used for evaluation are outlined, and their performance behavior is discussed.
In conclusion, Ara2 is the first fully open-source vector processor to support the RVV 1.0 specification. It achieves high performance and energy efficiency on a diverse set of data-parallel kernels. The paper provides valuable insights into the performance and energy efficiency trade-offs of single- and multi-core vector processors. The results contribute to the understanding of vector architecture design parameters and system configurations.
Ara2 is an open-source RISC-V vector architecture that aims to provide high-performance and energy-efficient computing solutions. The architecture is implemented with 2, 4, 8, and 16 lanes in 22nm FD-SOI technology, and it reaches a frequency of 1.35 GHz for designs with 8 lanes or fewer. The design incorporates a lightweight Slide Unit (SLDU) that reduces interconnect wiring and hardware in the all-to-all byte-connected unit by 70%. However, compliance with the RISC-V Vector (RVV) specification requires the addition of a new all-to-all connected Mask Unit (MASKU), which complicates the physical implementation.
The performance of Ara2 is evaluated through various benchmarks, and the results show that the architecture achieves over 50% of its throughput ideality starting from 128 Bytes per lane on average across all benchmark applications. It also achieves over 75% of its maximum throughput on crucial kernels like matrix multiplications and convolutions from 128 Bytes per lane. The performance scalability of Ara2 is influenced by the ratio between the application vector length and the number of lanes. When the number of elements per lane is constant, doubling the number of lanes also doubles the raw throughput. However, certain kernels, such as dotproduct, may experience a slight performance regression due to the latency of vector reduction instructions.
The baseline performance of Ara2 is compared across different system configurations and vector lengths. The results show that the performance ideality is affected by the ratio between the vector length and the number of lanes. Systems with smaller vector lengths and a higher number of lanes achieve higher performance ideality. The darker shades in heatmaps represent configurations with closer performance to the ideal. Some kernels, such as fft, dwt, softmax, and pathfinder, perform below average even for high ratios of Byte/lanes.
Replacing the scalar core (CVA6) with an ideal dispatcher improves the performance ideality of Ara2 by removing the issue-rate limitation and eliminating interference with memory transfers. The results show that the raw throughput ideality increases when CVA6 is replaced by an ideal dispatcher, especially for shorter vectors. However, some kernels, such as exp and roi-align, only experience slight benefits due to the presence of housekeeping scalar code.
Further performance considerations include the impact of the VRF layout and the main performance drivers in vector processors. The current implementation of Ara2 does not implement Barber's Pole VRF layout, which reduces initial stalls but can introduce new stalls for highly regular applications. The main performance drivers in Ara2 include the setup time of the internal pipeline, hazard resolution, and window size of simultaneous instructions.
The physical implementation of Ara2 in 22nm FD-SOI technology shows an increase in operational frequency compared to the previous Ara design. The area of Ara2 is higher than Ara due to the newly introduced functionality and a reduction in the size of the Vector Register File (VRF). The optimized Slide Unit (SLDU) reduces area by 83% compared to the unoptimized unit.
Multi-core analysis is performed to evaluate the performance and energy efficiency trade-off for different system configurations. Smaller vector core instances in multi-core systems outperform larger vector cores for applications that expose multiple dimensions of parallelization. The multi-core systems achieve performance improvements up to three times compared to a single-core architecture with the same overall computation capability.
Ara2 is compared to other state-of-the-art vector architectures, including Vitruvius+, Vicuna, and Spatz. Ara2 shows similar or improved performance compared to these architectures in various benchmarks. However, direct comparisons are challenging due to differences in benchmarking methodologies and available software implementations.
In conclusion, Ara2 is a promising open-source RISC-V vector architecture that offers high-performance and energy-efficient computing solutions. The architecture demonstrates good performance scalability and achieves high performance ideality for various applications. The multi-core systems provide flexibility in exploiting different dimensions of parallelization and can significantly
Raw indexed text (79,720 chars / 12,779 words / 1,829 lines)
UNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
1
Ara2: Exploring Single- and Multi-Core Vector
Processing with an Efficient RVV1.0 Compliant
Open-Source Processor
Matteo Perotti, Student Member, IEEE, Matheus Cavalcante, Student Member, IEEE,
Renzo Andri, Member, IEEE, Lukas Cavigelli, Member, IEEE, and Luca Benini, Fellow, IEEE
Abstract—Vector processing is highly effective in boosting processor performance and efficiency for data-parallel workloads. In this
paper, we present Ara2, the first fully open-source vector processor to support the RISC-V V 1.0 frozen ISA. We evaluate Ara2’s
performance on a diverse set of data-parallel kernels for various problem sizes and vector-unit configurations, achieving an average
functional-unit utilization of 95% on the most computationally intensive kernels. We pinpoint performance boosters and bottlenecks,
including the scalar core, memories, and vector architecture, providing insights into the main vector architecture’s performance drivers.
Leveraging the openness of the design, we implement Ara2 in a 22nm technology, characterize its PPA metrics on various configurations
(2-16 lanes), and analyze its microarchitecture and implementation bottlenecks. Ara2 achieves a state-of-the-art energy efficiency of 37.8
DP-GFLOPS/W (0.8V) and 1.35GHz of clock frequency (critical path: ∼ 40 FO4 gates). Finally, we explore the performance and
energy-efficiency trade-offs of multi-core vector processors: we find that multiple vector cores help overcome the scalar core issue-rate
bound that limits short-vector performance. For example, a cluster of eight 2-lane Ara2 (16 FPUs) achieves more than 3x better
performance than a 16-lane single-core Ara2 (16 FPUs) when executing a 32x32x32 matrix multiplication, with 1.5x improved energy
efficiency.
Index Terms—RISC-V, Vector, ISA, RVV, Processor, Efficiency, Multi-Core.
✦
1
I NTRODUCTION
I
N 1976, the Cray-1 supercomputer [1] achieved the world’s
top performance of 250 MFLOPS by exploiting Data Level
Parallelism (DLP) through a vector Instruction Set Archi-
tecture (ISA). Almost half a century later, in 2020, another
vector architecture took the highest spot in the TOP500 list
by scoring around 537 PFLOPS of peak performance: the
Fugaku supercomputer, based on Fujitsu’s A64FX processor
and Arm’s Scalable Vector Extension (SVE).
Despite an increase in raw computer performance by nine
orders of magnitude, today’s computing systems struggle to
meet the performance requirements of ubiquitous Artificial
Intelligence (AI) and Machine Learning (ML) workloads. The
amount of data to process grows faster than the technology-
driven processing-speed gain [2], and the power required to
sustain the ever-increasing performance needs is the main
limiting factor from the cloud (operating cost, cooling) to
the edge (battery lifetime, weight). It comes as no surprise
that the quest for energy efficiency is crucial for today’s
computing systems [3] across all performance profiles, from
supercomputers (Green500 ranking) to ultra-low power AI
processors (MLPerf Tiny).
•
•
•
•
Matteo Perotti is with the Integrated Systems Laboratory (IIS), ETH
Zurich, 8092 Zurich, Switzerland. E-mail: [email protected]
Matheus Cavalcante is with the Electrical Engineering Department,
Stanford University, California 94305, USA. E-mail: [email protected]
Renzo Andri and Lukas Cavigelli are with the Huawei Zurich Research
Center. E-mail: { renzo.andri, lukas.cavigelli } @huawei.com
Luca Benini is with the Integrated Systems Laboratory (IIS), ETH Zurich,
Switzerland, and also with the Department of Electrical, Electronic and
Information Engineering (DEI), University of Bologna, 40126 Bologna,
Italy. E-mail: [email protected].
While the Cray-1 vector processor was conceived for
pure performance in the heydays of Emitter-Coupled Logic
(ECL) bipolar transistor technology, the recent feat of Fugaku
proves that vector ISAs are well-positioned to answer the
conflicting needs of rapidly increasing computational de-
mands under a tightly-constrained power budget. First, they
effectively address the Von Neumann Bottleneck (VNB) by
reducing the number of instructions fetched from memory
through the execution of the same instruction on multiple
elements, in the so-called single instruction, multiple data
(SIMD) execution model. Furthermore, vector processors
leverage a Vector Register File (VRF) to enhance data reuse
in computationally intensive applications while remaining
vector-length agnostic, as the vector length can be pro-
grammed at runtime.
After the release of Arm’s SVE and SVE2 vector ISAs,
RISC-V also developed its RISC-V “V” (RVV) vector ex-
tension, getting much interest due to its openness and
extensibility. RVV has reached frozen state with its 1.0 release
in 2021 after six years of design and standardization ef-
forts. Companies and academia developed numerous vector
designs supporting various pre-release versions of RVV
and targeting different application domains such as High-
Performance Computing (HPC) [4], [5], [6], Deep Neural
Networks (DNN) [7], AI/Augmented Reality (AR) [8], real-
time applications [9], and Internet-of-Things (IoT) endpoints
[10]. These designs are characterized by their heterogeneity
in terms of deployed system architectures: [4], [5], [9], [11]
are single-core systems, while [8], [10], [12], [13] are multi-
core or multi-core-ready. Among them, [4], [9], [12] work
with a relatively simple in-order core, while [5], [8], [10], [11],UNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
[13], [14] deploy more aggressive superscalar or out-of-order
scalar pipelines.
These designs show a wide spectrum of vector architec-
tures, but there is a fundamental lack of detailed architectural
performance studies on the effects that the design parameters
of a vector processor (VRF organization and size, number of
processing elements (PEs), target application vector length)
and system configurations (scalar core, scalar caches, single-
or multi-core configuration) have on the system’s perfor-
mance and energy efficiency. As already noticed in [4], the
scalar core can highly limit the system’s performance when
processing short vectors. Moreover, despite the efforts to
propose standard RVV benchmarks to ease architectural
development and comparison [15], the aforementioned
publications provide performance data only for, in the best
case, a small set of kernels.
In the constellation of RISC-V-based Vector processor
architectures, Ara [4] was among the earliest ones. Its internal
number of Floating Point Units (FPUs) could be scaled from
2 to 16, and the design, implemented in a 22nm technology,
reached 1 GHz in typical conditions in all the configurations,
with a leading State-of-the-Art (SoA) of 41 DP-GFLOPS/W
energy efficiency at its peak and 97% of FPUs peak utilization.
The architecture, though, was compliant with only a limited
set of RVV 0.5, the very first and preliminary version of RVV,
missing key instructions like vector reductions, ubiquitous in
the ML domain. The ISA architectural specifications and the
software ecosystem deeply changed with the RVV updates,
and RVV0.5-compliant designs are now obsolete.
In this paper, we present the RVV-1.0 Ara2 vector
processor and leverage its efficient parametric architecture
to explore a wide range of configurations to optimize
computational efficiency. To our knowledge, Ara2 is the first
open-source vector processor to support the specification
RISC-V V 1.0. We analyze the frozen RVV specification and
its major effects on the microarchitecture of a vector processor,
highlighting the novelties/changes and the issues that arise
from them.
Furthermore, Ara2 aims to close the gap between the
various RVV architectures proposed throughout the years
and the lack of thorough performance/efficiency analyses for
microarchitecture and implementation. The key contributions
of this work are:
•
•
•
The Ara2 microarchitecture and design 1 , with support
for the RISC-V “V” 1.0 ISA. We analyze the changes
from the previous preliminary RVV specifications and
discuss their impact on the system microarchitecture.
To the best of our knowledge, Ara2 is the first open-
source processor to support RVV 1.0.
An in-depth analysis of performance depending on
the application vector length on benchmark kernels
from different application domains (Linear algebra,
Digital Signal Processing (DSP), ML). Furthermore,
we analyze the effect that the scalar processor and its
memory system have on performance.
An implementation of the Ara2 system in a 22nm
Fully Depleted - Silicon on Insulator (FD-SOI) technol-
ogy and its Power, Performance, Area (PPA) metric
analysis from 2 to 16 lanes, with insights into the
1. https://github.com/pulp-platform/ara
2
•
impact that microarchitectural optimizations of the
most critical all-to-all unit (Slide Unit (SLDU)) have
on the scalability of the vector architecture.
A performance/efficiency trade-off study on different
hardware configurations, varying the number of
vector cores and lanes per vector core on different
problem sizes. We demonstrate how a multi-core
vector architecture can mitigate the performance
impact of the scalar core’s issue-rate limitation and
identify the most efficient architecture for specific
application vector lengths.
The paper is structured as follows: Sections 1 and 2
present an introduction and a background to contextualize
the overall work. Section 3 discusses the main changes
between RVV 0.5 and RVV 1.0, and Section 4 the implications
on a lane-based vector processor, while, in Section 5, we
present the updated architecture. In Section 6, we describe
our experiment setup and describe the benchmark kernels
used for our evaluations. Then, we present performance
analysis, physical implementation results, and a perfor-
mance/efficiency trade-off study on single- and multi-core
architectures in Sections 7, 8, and 9, respectively. Finally, we
provide a comparison with the state-of-the-art in Section 10.
2
B ACKGROUND AND R ELATED W ORK
The vector processor architecture was first implemented by
the Cray-1 supercomputer. A vector processor is character-
ized by its internal VRF, composed of a fixed number of
registers whose size is upper-bounded by the ISA (usually
with a very loose upper bound). Arm SVE and RVV feature
32 vector registers, each of which with a size from 128 to
2048 (SVE) or 64k (RVV) bits. The VRF can be conceptualized
as L0 storage, meant to buffer data elements re-used multiple
times close to the PEs 2 to avoid more costly memory accesses.
Buffering more elements in the VRF lowers the pressure on
the data memory [16]. The VRF, acting as an L0 buffer, also
allows vector processors to be latency-tolerant with respect
to memory stalls since every instruction occupies the PEs
for multiple cycles. As noticed in [17], the tolerance against
memory latency is generally higher when the architecture
works on longer vectors.
Moreover, the vector architecture amortizes the instruc-
tions’ fetch, decode, and issue cost of each vector instruction
by executing it on as many data elements as they fit in a
vector register. Intuitively, the longer the vector, the less
pressure on the instruction memory.
Two examples of successful commercial vector architec-
tures are the NEC SX-Aurora TSUBASA and the Fujitsu
A64FX. The latest generation of SX-Aurora TSUBASA be-
longs to the NEC SX architecture family and features a 16-
nm technology 1.6 GHz 16-core architecture equipped with
a 128-KiB VRF per core, composed of 64 vector registers
connected to three 32-way Fused Multiply-Add (FMA)
pipelines, totaling a peak performance of 192 DP-FLOP/cycle
per core and 4.91 TFLOPs for the whole system. Instead,
the 7-nm FinFET-technology A64FX is an Arm-SVE-based
multi-core cluster-based architecture, where clusters of 13
2. With PE, we identify every execution unit within a vector architec-
ture, e.g., FPU, Multipliers, Arithmetic-Logic Units (ALUs), etc.UNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
3
TABLE 1: Overview of RISC-V Vector Processors.
Core Name
[14]
[8]
[13]
[11]
[6]
[12]
[9]
[7]
[18]
[5]
[10]
[19]
[4]
[20]
RVV Target XLEN float VLEN Split VRF Open-
version
(bit) supp. (bit)
(lanes) Source
This work
1.0
SiFive P870
1.0
SiFive X280
1.0
SiFive P270
1.0rc
Andes NX27V
1.0
S.D. VU
1.0
Spatz
1.0 e
Vicuna
0.10
Arrow
0.9
Johns et al.
0.8
Vitruvius+
0.7.1
XuanTie 910
0.7.1
RISC- V 2
?
Ara
0.5
Hwacha
Non-Std.
ASIC
ASIC
ASIC
ASIC
ASIC
ASIC
ASIC
FPGA
FPGA
FPGA
ASIC
ASIC
ASIC
ASIC
ASIC
64
64
64
64
64
64
32
32
32
32
64
64
?
64
64
✓
✓
✓
✓
✓
✓
✗
✗
✗
✗
✓
✓
✗
✓
✓
1024 a
128
512
256
512
128-4096
128-512
128-2048
?
32
16384
128 a
256
4096 a
512 a
✓
?
?
?
?
✓
✗
✗
✓ b
✗
✓
✓
✗
✓
✓
✓
✗
✗
✗
✗
✗
✗
✓
✗
✗
✗
✗ c
✓ d
✓
✓
a
VLEN per lane.
VRF is split horizontally.
c
The vector unit is not open-source.
d
The scalar core is not open-source.
e
Limited subset of RVV.
b
cores (core-memory groups) are individually coupled with
8-GiB HBM2 modules and are connected together through
a TofuD network on chip (NoC) interconnect, with a peak
theoretical performance of almost 3.38 TFLOPs. All non-
RISC-V vector architectures are commercial products and
the microarchitectural details remain trade secrets. On the
other hand, Table 1 summarizes the SoA of the RISC-V-based
processors, which have been developed by companies as
well as researchers. Ara2 is the first processor that supports
RVV 1.0 (Vicuna [9] does not implement the floating-point
and is only compliant with the Zve32x subset).
Among these vector architectures, [6], [8], [11], [12], [13],
[14], are multi-core or multi-core-ready, testifying the interest
of industry towards coupling the two paradigms. Section 10
provides a quantitative comparison of our design and the
architectures in Table 1.
3
T HE E VOLUTION OF RISC-V V
The RVV extension adds a VRF composed of 32 vector
registers, which can individually buffer vector elements
of the same type (e.g., FP32). The ISA includes element-
wise operations between different vectors, reductions of
vectors to a single scalar element, and predicated execution
to implement simple control flows, i.e., the computation can
be enabled/disabled with a single-element granularity.
RVV was first proposed in 2015 [21] and went through
several revisions before becoming stable, which were initially
not clearly formalized [22], [23], [24]. We refer to the last
informal specification (2018) as v0.5, as done in [4].
The specification has now reached v1.0 frozen status.
While the fundamental concept of RVV has remained con-
sistent over time, three major differences deserve to be
discussed: 1) the organization of the vector register file, 2)
the encoding of the instructions, and 3) the organization of
the mask registers.
3.1
Random-Access Memories (SRAMs), and its byte layout
highly impacts the architecture design.
v0.5: The register file had both a global and local state.
Users could dynamically enable registers, and the hardware
calculated the maximum vector length by distributing the
byte space among the enabled registers. Each register could
be individually programmed for various data types.
v1.0: The register file state is global. The vector register
file consists of 32 VLEN-bit vector registers, where VLEN
is a parameter specific to the implementation, representing
the bit width of a single vector register (in RVV, V LEN ≤
2 16 ). The granularity of the register file can be adjusted
by configuring the parameter LMUL; for example, setting
LMUL to 2 results in sixteen 2 × VLEN-bit vector registers.
Moreover, the register file is agnostic on the data type of the
stored elements.
Striping distance: The initial proposal did not provide
strict constraints on the byte layout of the vector register
file. This was specified later, and with the introduction of
the striping distance (SLEN) parameter, it became especially
lane-friendly in version 0.9 of the specifications.
v0.9: SLEN ≤ VLEN: each vector register can be divided
into a total of VLEN / SLEN sections with SLEN bits. Consec-
utive vector elements are mapped into consecutive sections,
wrapping back around to the first section until the vector
register is full [25].
v1.0: SLEN = VLEN: the VRF is considered a contiguous
entity, and consecutive element bytes are stored in consecu-
tive VRF bytes.
3.2
Instruction Encoding
v0.5: The instruction encoding was polymorphic, as the
data type of the vector elements was set for each vector
register. For example, the instruction vadd was used for both
integer and floating-point vector additions.
v1.0: The encoding is monomorphic, and there are differ-
ent instructions for different data types, i.e., integer, fixed-
point, and floating-point. Consequently, the ISA has more
instructions, becoming one of the largest extensions in the
RISC-V ecosystem.
3.3
Mask Vectors
Vector processors run conditional code through predication,
which, in RVV, is implemented by means of mask vectors,
each containing one bit per element. When the operation
is masked, the core executes on element i only if the i -th
mask-vector bit is asserted.
v0.5: Each element of a mask vector could host one mask
bit in its Least Significant Bit (LSB), and there were no
instructions to manipulate mask vectors.
v1.0: Every register of the VRF can host a mask vector,
and the mask bits are sequentially packed one bit after each
other, starting from the LSB of the vector register. There are
logical instructions to manipulate mask vectors.
Vector Register File
VRF state: The VRF is an additional level in the memory
hierarchy to buffer the vector operands. When the supported
vector length is wide, it is usually implemented with Static
4
RISC-V V AND L ANES
In this section, we discuss the impact of the RVV extension
on the microarchitecture. We will consider Ara as an exampleUNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
4
of a design tuned to RVV 0.5, even if the discussion is not
limited to it, while Ara2 is our new architecture.
Ara2 is a modular architecture with parametric VLEN
targeting high performance and efficiency on a broad range
of vector lengths. For example, with VLEN = 4096 , the unit
can process vectors up to 4 KiB, when LMUL = 8 , with
a 16 KiB VRF. Pushing for high vector lengths has many
advantages: operating on vectors that do not fit the VRF
requires strip-mining with its related code overhead, which
translates into higher bandwidth on the instruction memory
and more dynamic energy spent in decoding and starting the
processing of the additional vector instructions. In addition,
longer vectors help tolerate the memory latency [17] and the
setup time of each vector instruction.
Fig. 1: 2-lane Ara2 VRF byte layout for different data widths.
4.3
4.1
VRF and Lanes
In Ara2, as in Ara, each lane contains a chunk of the
distributed VRF, implemented with 8 1RW SRAM banks per
lane. In the following, we motivate the distributed approach
by comparing it with the alternative implementation, i.e., a
monolithic VRF, which would make the VRF interconnect
complexity grow with the square of the number of lanes ℓ 2 .
In general, the area of a crossbar ( A xbar ) is proportional
to both the number of masters and slaves, as it requires
Masters de-multiplexers and Slaves arbiters. With a split
VRF, all the functional units of a lane (masters, M lane ) connect
to all the lane banks ( B lane = 8 ), so the total interconnect
area is the interconnect area of one lane multiplied by the
number of lanes ( ℓ ),
split
A xbar ∝ M lane × B lane × ℓ = M lane × 8 × ℓ,
(1)
while a monolithic VRF would connect all its banks ( B tot =
8 × ℓ ) to every master of each lane,
2
A mono
xbar ∝ (M lane × ℓ) × B tot = M lane × 8 × ℓ .
(2)
The quadratic dependency on the number of lanes limits
the scalability of a vector processor based on a monolithic
VRF. Moreover, the split VRF allows for more freedom on the
macro placement during floorplanning and better routability
since the interconnect is local to each lane.
4.2
Byte Layout
With RVV 1.0, the VRF byte layout is the same as the memory
byte layout, i.e., byte i of a vector in memory is kept in
byte i of the VRF. This can be sub-optimal in the case of
a laned vector processor. Indeed, not to complicate mixed-
width operations and to maximally exploit DLP, Ara2 maps
consecutive elements to consecutive lanes, i.e., element i is
kept in lane i modulo ℓ . In this case, depending on the
Element Width (EW), the same byte can be mapped to
different lanes, as summarized in Figure 1.
As a consequence, the processor must track the last EW
with which each vector register was written, to be able to
restore its content, and the units that access a vector register
must be able to re-map its elements.
Shuffle/Deshuffle
The remapping is realized with shuffling (bytes to VRF) and
deshuffling (bytes from VRF) logic, which translates into a
level of byte multiplexers, one per output byte. With L lanes
with a 64-bit datapath each, a unit that accesses the VRF can
shuffle/deshuffle L × 8 bytes every cycle by using L × 8
multiplexers. Since an RVV unit should support EWs of 8,
16, 32, and 64 bits, each multiplexer has four input bytes.
4.4
RVV 1.0 Implementation Challenges
Supporting RVV 1.0 with a lane-based architecture poses
non-trivial challenges:
Source Registers Every time a source register is read with
old
an EW new
vs different than its previously encoded EW vs , Ara2
should reshuffle its bytes to reinterpret its content according
to the new element width. Generally, if the operation is
executed outside of the lanes, the units can automatically
reshuffle the data elements on the fly. In the case of in-lane
operations, the SLDU should reshuffle the source register
content since, with a different EW, bytes belonging to the
same element can be in different lanes. We do this by injecting
a reshuffle operation before the “offending” instruction.
Destination Registers RVV also mandates to support
a tail-undisturbed policy. When in place, the destination
elements past the last active one should not be modified.
When an EW new
vd -instruction writes a vector register vd
new
previously encoded with EW old
vd ̸ = EW vd and the old content
of vd is not fully overwritten, the unmodified bytes get
corrupted since the byte mapping of vd is no longer unique.
To avoid corrupting the tail elements, when this situation
occurs, Ara2 should deshuffle the destination register using
new
EW old
vd and reshuffle it back using EW vd by injecting a
reshuffle operation (i.e., a vector slide with null stride) into
the SLDU before the offending instruction. Since it is not
possible to know how many bytes need to be reshuffled
unless both the vector length and the element widths are
dynamically tracked for each vector register, the reshuffle
acts on the whole register. No reshuffle is injected when
the offending instruction writes a whole vector register, as
reshuffling hurts the Instructions Per Cycle (IPC) if its latency
cannot be hidden and if this operation causes structural
hazards on following slide or reduction instructions.
Reshuffling is costly, as the offending instruction has a
read-after-write (RAW) dependency on the reshuffle. The
compiler can alleviate the problem by assigning EW-encoded
vector registers to instructions with the same EW.UNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
5
Baseline SLDU
L0
L1
L(N-1)
64
64
64
Memory
ordering
ISSUE STALL
slide
enc_in
enc_out
64
L0
64
L1
64
L(N-1)
Input Bytes mapped to Output Byte 15 Input Bytes mapped to Output Byte 15
Output Byte 15 Input Bytes not mapped to Output Byte 15
Output Byte 15
VLSU
ADDRgen
VLDU
VSTU
VRF chunk
8 SRAM banks
Fig. 3: Comparison between the baseline and the optimized slide units,
with a focus on how an arbitrary byte (15th) is mapped to the input
bytes. The baseline slide unit supports arbitrary slide amounts and can
slide and re-encode a vector in the same cycle. Each output byte is
mapped to every input byte, so that the total number of connections is
O( L 2 ). In the optimized design, we support only power-of-two slides.
Moreover, slides and re-encoding cannot happen at the same time. The
total number of connections grows following an O( L×log 2 (L) ) behavior.
Fig. 2: Top-Level block diagram of the Ara2 system with the vector
co-processor (green), a more detailed diagram of the lane (magenta),
and the host scalar core CVA6 (blue).
-70%
5
A RCHITECTURE
In Figure 2, we show the block diagram of the scalar core
CVA6 and the vector unit Ara2. The architecture is based on
Ara [4], with the following major enhancements:
Decoding: With the new RISC-V V specifications, the
encoding of the vector instructions now fully specifies the
data type of the vector elements on which the instruction
operates. This allows moving most of the decoding logic
and vector-specific Control and State Registers (CSRs) from
CVA6 to the vector unit, making CVA6 more agnostic on
the V extension. In the updated architecture, CVA6 keeps
only the pre-decoding logic strictly needed to know 1) if the
instruction is a vector instruction (to dispatch it to the vector
unit when it reaches the head of the scoreboard), 2) if the
vector instruction is a memory operation (to enforce cache
coherence), and 3) if the instruction needs a scalar value from
the integer or floating-point scalar register files.
CVA6-Vector Unit Interface: The interface between the
host processor CVA6 and the vector unit is generalized: the
unit is implemented as a modular accelerator with its own
CSR file. While decoding, CVA6 identifies vector instructions,
pushes them to a dispatcher queue, and dispatches them to
the accelerator once they are no longer speculative.
Memory Coherence: CVA6 and the vector unit have
separate memory ports, and CVA6 has a private L1 data
cache. At the same time, the RISC-V ISA mandates a strictly
coherent memory view between the scalar and vector pro-
cessors. Ara [4] violates this requirement and needs explicit
memory fences that write back and invalidate the CVA6
data cache between accesses on shared memory regions,
adding a significant performance overhead and reducing
code portability. In Ara2, we develop a hardware mechanism
to ensure coherence. We adapt the CVA6 L1 data cache to
a write-through policy so that the main memory, which is
accessed by the vector unit as well, is always up-to-date.
When the vector unit performs a vector store, it invalidates
the corresponding cache lines in the CVA6 data cache.
Moreover, we issue 1) scalar loads only if no vector stores
are in-flight, 2) scalar stores only if no vector loads or stores
-56%
Fig. 4: Number of 2-to-1 multiplexers needed to implement a slide
unit as a function of the number of lanes with different configurations.
An all-to-all slide unit connects every input byte to every output byte
and supports every slide and reshuffle operation in one cycle. Other
configurations support only slides by powers of two (SlideP2), slides by
one (Slide1), and can slide and reshuffle or either slide or reshuffle.
are in-flight, and 3) vector loads or stores only if no scalar
stores are pending.
Mask Unit: After the update to RVV 1.0, the mask bits
used by one lane can be stored in a different one. Thus, we
designed a Mask Unit, which can access all the lanes at once
to fetch, deshuffle at bit granularity, and dispatch the correct
mask bits to each lane. Moreover, the Mask Unit (MASKU)
combines the results of many vector mask instructions, such
as vcpop and vfirst. The introduction of an additional
unit that accesses all the lanes leads to a greater routing
complexity, especially when scaling up the number of lanes,
as already noticed in [4].
Optimized Slide Unit: The mask unit makes the physical
implementation effort harder due to its all-to-all connections
to/from the lanes. Reducing the complexity of each all-
to-all connected unit is paramount to ease the physical
implementation (placement and routing) of the system. A
preliminary study of how the area of the system scales with
the number of lanes shows that the most problematic block
is the slide unit, being the largest all-to-all connected unit
and scaling quadratically with the number of lanes. This is
expected, as the block supports slides by arbitrary amounts
for element widths down to 8 bits by mapping the 8 × L
input bytes to the 8 × L output bytes. This interconnect can
be simplified by limiting the type of operations supported
by the unit. For example, we can constrain the datapath
to support only slide-by-1 or slide-by-power-of-two slideUNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
instructions, with the other strides handled with multiple
micro-operations.
In Figure 4, we attach a plot showing the number of
2-to-1 multiplexers needed to implement the interconnect
with different slide unit configurations. This number is also
a lower bound to the number of wires that connect the input
bits before the slide to the output slided bits and is a valid
metric to estimate the area of the interconnect and the wiring
complexity during the routing stage. Since non-power-of-two
slides and reshuffling are uncommon, we limit the strides
fully supported in hardware to powers of two and handle
the remaining strides with micro-operations, saving up to
70% of the estimated area and wires, as shown in Figure 3.
Reductions Since our design has lanes, we implement
reductions using a 3-steps approach: intra-lane, inter-lane,
and SIMD reduction steps. The intra-lane reduction fully
exploits the data locality within each lane, maximizing the
ALU (or FPU) utilization and efficiency, reducing all the
elements already present in the lane. The inter-lane reduction
moves and reduces data among the lanes in log 2 (ℓ) + 1 steps,
ℓ being the number of lanes, using the slide unit; since there
is a dependency feedback between the slide and ALU (or
FPU) units, the latency overhead of the communication is
paid at every step. Finally, the SIMD reduction reduces the
SIMD word, if needed; therefore, its latency logarithmically
depends on the element width.
Floating-point reductions need special care since the
FPU is pipelined. To keep the throughput at its maximum,
the internal pipeline registers of the FPU are used as
accumulators. During the intra-lane phase and after adding
the first two elements together, each FPU is constantly fed
with one operand per cycle. The other FPU input is either a
neutral value (zero for the addition) or one of the partial
sums coming from the FPU output, when available. In
the best case, after N/L cycles, each FPU pipeline register
contains a partial sum, and the reduction will be completed
in R × (1 + log 2 ⌈R⌉) − (⌈R⌉ − R) − 1 cycles. Here, N is the
number of 64-bit packets of operands in the vector to reduce,
L is the number of lanes, and R is the number of FPU pipeline
stages. When R is a power of two, the formula simplifies
to R × (1 + log 2 (R)) − 1 . Then, during both the inter-lane
and SIMD reductions, the FPU latency directly affects each
micro-operation. Since the number of FPU pipeline registers
increases with the EW, reductions on lower bit widths can
be faster than the ones on higher bit widths.
6
E XPERIMENT S ETUP
In the following section, we describe our experiments in
detail, including the system configuration, tools, and metrics
used for performance characterization, physical implementa-
tion, and single-/multi-core trade-off evaluations. Then, we
outline the details of our benchmark pools and the rationale
behind selecting our kernels. In the following, we will use
the term system to refer to the system-under-test composed
of CVA6, its caches 3 , and Ara2. Ara2 and the L1 caches
connect to a 1-cycle-latency SRAM memory 4 via the AXI
memory interconnect shown in Figure 2. While analyzing a
3. I$: 4 KiB, 4 sets, 128-bit line. D$: 8 KiB, 4 sets, 256-bit line
4. Memory: 2M words, 4 × L Byte/word, L being the number of lanes
6
complex memory hierarchy is out of the scope of this work,
this simple configuration includes L1 caches and a standard
system bus initiator.
Performance analysis: We characterize the system’s per-
formance in terms of raw throughput ideality, which identifies
the ratio between the measured raw throughput and the ideal
maximum raw throughput achievable by the vector processor
for a particular kernel. With raw throughput, we identify
the ratio between the number of “useful” operations of an
algorithm and the number of clock cycles spent to execute
it, counted from the first vector instruction dispatched from
CVA6 until the last vector instruction fully executed by Ara2
(raw since there is no frequency information). This metric
helps study the ideality of the design on different vector
lengths (different problem sizes). For our measurements,
we run all benchmarks on our vector system using a cycle-
accurate simulation, with all the benchmark instructions and
data preloaded in the SRAM main memory.
We measure the performance of each benchmark for
different vector lengths (from 32 to 1024 Byte) and hardware
configurations (Ara2 with 2, 4, 8, and 16 lanes) to constitute
a baseline for further performance analyses.
What-if analyses: We repeat the performance measure-
ments by replacing CVA6 and its L1-caches with a FIFO
that contains the whole dynamic vector instruction trace of
the program in analysis and the correct scalar register file
entries to be forwarded to Ara2 if needed. This allows us
to determine the impact of the scalar part of the system
on the overall computation and set an upper bound on
performance gains achievable by streamlining the scalar sub-
system. In these conditions, performance is only limited by
the vector co-processor. Additionally, we track the number
of L1-cache misses to study the effect of scalar memory
system non-idealities on kernel performance. Finally, we
further streamline the vector processor with modifications
that would impact the vector processor hardware (e.g., by
aggressively upsizing the internal queues and scoreboard)
and study their impact on the system’s performance. The
overall experiment highlights the bottlenecks that limit the
computation performance and the cost to overcome them.
Physical implementation analysis: After synthesizing,
placing, and routing the system in 22nm FD-SOI technology
using industrial-grade tools, we evaluate its power, perfor-
mance, and area (PPA) metrics. The system includes CVA6, its
L1 caches, the invalidation filter, Ara2, and the interconnects
up to the system crossbar. To analyze the switching activity of
the netlist internal signals, we use a cycle-accurate simulator
to generate a VCD file with delay back-annotations from the
physical place-and-routed design. We repeat the analyses for
systems with 2, 4, 8, and 16 lanes. The input matrices are
samples from a uniform distribution over [0,1).
Multi-Core analysis: The system is modified to accommo-
date multiple clusters of the previously implemented system,
and the SRAM main memory is multi-banked to support
multiple Ara2 instances, with one bank per Ara2 and every
bank with a parallelism of 4 × L Byte each, with L being the
number of lanes of each Ara2. Moreover, we implement a
lightweight synchronization engine via system-level CSRs
to let the clusters synchronize at the end of the execution.
The main objective is to evaluate the impact of the multi-core
vector architecture on performance and energy efficiency.UNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
TABLE 2: Benchmarks used in the performance analysis.
Program
Domain
DType CB M S SMO IMO R Max Perf. [FLOP/cycle]
matmul
Linear Algebra, ML
FP64 Y N N
N
N N
1 × 2.0 × L
conv2d
Signal Processing, ML
FP64 Y N Y
N
N N
1 × 2.0 × L
dotproduct
Linear Algebra FP64, int64 N N N
N
N Y
1 × 0.5 × L
jacobi2d
Stencil
FP64 Y N Y
N
N N
1 × 1.0 × L
dropout
ML
FP32 N Y N
N
N N
2 × 0.25 × L
fft
Signal Processing
FP32 Y Y Y
N
Y N
2 × 5/4 × L
dwt
Signal Processing
FP32 N N N
Y
N N
2 × 0.5 × L
pathfinder
Routing Algorithm
int32 N Y N
N
N N
2 × 1.0 × L
exp
Scientific, ML
FP64 Y N N
N
N N
1 × 28/21 × L
softmax
ML
FP32 Y N N
N
N Y
2 × 32/25 × L
roi-align
ML
FP32 N N N
N
N N
2 × 3/5 × L
CB: Compute Bound, M: Masks, S: Slides, SMO: Strided Mem. Ops., IMO: Indexed Mem. Ops.,
R: Reductions
To do this, we vary the number of cores per cluster, the
number of lanes of each vector core, and the application
vector length of the floating-point matrix multiplication
algorithm. The modified kernel used in this phase is slightly
different from the one used in Section 7, and scalar caches are
not warmed up to ease the automatic VCD dumping script.
However, the comparison is still fair as all the configurations
run the same program.
We insert a synchronization point before and after the
kernel execution, measure performance and energy efficiency
within this time span, and simulate fmatmul between square
matrices with different sizes ranging from 4x4 to 256x256
elements on systems with { 1, 2, 4, 8 } cores and { 2, 4, 8, 16 }
lanes, with no more than 16 FPUs in total.
Benchmark selection: Table 2 summarizes the bench-
marks used to evaluate the Ara2 System’s performance. The
benchmarks include a mix of kernels from various domains,
including signal processing, linear algebra (BLAS), and
machine learning (ML). The kernels were chosen to stimulate
all the main units of Ara2 and include both compute-bound
and memory-bound kernels. Here is a brief overview of each
benchmark:
matmul and conv2d are two compute-bound kernels
that are used in multiple domains (Signal processing, linear
algebra (BLAS), ML). Their performance behavior is crucial
due to the kernels’ ubiquity. We re-used matmul from [4],
and optimized the 3x7x7-kernel conv2d by keeping, after the
initial setup steps, seven output vectors in the VRF for every
loaded input vector, to maximize data reuse. fft and dwt are
well-known algorithms used in the signal processing domain.
We implement a vectorized version of fft by exploiting
the large Ara2 VRF to buffer all the input samples. This
avoids costly transfers to and from memory, in a similar way
as shown in Bertaccini et al. [26], and sets a limit to 128 ×
Lanes input samples (e.g., 2048 input samples with 16 Lanes).
We take pathfinder, jacobi2d, and exp from the RiVec
benchmark suite repository [15], [27], and optimize them
with assembly instructions. We chose these three kernels
from RiVec since they were the easiest ones to bring to
bare-metal C code, as the suite was primarily developed
to run on gem5 with OS support. We vectorized dropout,
roi-align, and softmax, three kernels used in the ML
domain, and we included dotproduct since it does not
expose multiple dimensions for parallelization and forces the
implementation to use a reduction.
7
P ERFORMANCE CHARACTERIZATION
In this section, we analyze the system’s performance and
how it changes with the system configuration, the scalar core,
7
the configuration of the scalar memory hierarchy, the number
of lanes, and the application vector length.
7.1
Lanes and Vector Length
When we scale up the Ara2 co-processor by doubling its num-
ber of lanes, we also double 1) its AXI bus data width and 2)
the parallelism of the all-to-all internal units (MASKU, SLDU,
Vector Load/Store Unit (VLSU)). For a deeper understanding
of the performance scalability, Figure 5 shows the correlation
between the performance ideality of the Ara2 co-processor,
its number of lanes, and the application vector size, for two
different kernels: dotproduct and fmatmul. Specifically,
on the x -axis of the plots, we list different vector sizes (in
Byte), i.e., the number of vector elements times the element
size. For example, the entry (8L, 512 Byte) corresponds
to the performance ideality achieved by an 8-lane system
when dealing with vectors composed of 64 64-bit elements
each. When the ratio #Byte/#Lane is constant, the achieved
performance ideality is similar for systems with different
configurations. This means that the raw throughput almost
doubles when the number of lanes doubles if the number of
elements per lane is constant. This experiment shows that the
computation ideality of Ara2 is related to the ratio between
the application vector length and the number of lanes and
that a vector is short or long only depending on the vector
architecture on which will be processed. On the right, the
two heat plots visually remark the same concept, as entries
on the same diagonal (same byte-per-lane ratio) have similar
colors, i.e., similar performance.
Both analyzed kernels show a slight performance re-
gression for a constant byte-per-lane ratio. In the case of
the dotproduct, the performance decreases when increas-
ing the number of lanes because of the vector reduction
instruction, whose latency depends on the number of lanes
during the inter-lane phase. On the other hand, fmatmul
is computed on square matrices, and when the matrix size
increases, the arithmetic intensity increases as well. Moreover,
the performance drop of the 2L configuration at 16 and 32
Byte/Lane testifies that when the number of elements is too
low, the setup time of the kernel is less amortized.
7.2
Baseline System’s Performance
Figure 6 presents the performance ideality of the baseline
system across all benchmarks in our pool, varying the
application vector length. The darker the green shades in the
plot, the closer the system performance is to the ideal. As
noticed in Section 7.1, the ratio between the vector length
and the number of lanes significantly affects performance,
as shown by the 2-lane design reaching over 70% of average
performance ideality with a 1KiB vector (512 Byte/lane),
while the 16-Lane system, with the same vector, would reach
only 33% (64 Byte/lane). This is apparent in the heatmaps,
where the darker colors shift right by one position every time
the lanes double.
fft, dwt, softmax, and pathfinder perform below
average, even for high ratios of Byte/lanes, with softmax
suffering from the floating-point division data-dependent
latency and the large setup time for preloading the ap-
proximation function coefficients of the software-emulated
exponentiation. dwt is slowed down by misaligned stridedUNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
Raw Throughput Ideality
Raw Throughput Ideality
dotproduct
fmatmul
Vector Size [B]
Vector Size [B]
Fig. 5: Correlation between Raw throughput Ideality and Byte/lane ratio
for dotproduct (top) and fmatmul (bottom). The raw throughput ideality
tends to be similar when the number of elements per lane is the same
(on the diagonals).
2-Lane Ara2 - Raw Throughput Ideality 4-Lane Ara2 - Raw Throughput Ideality
8-Lane Ara2 - Raw Throughput Ideality 16-Lane Ara2 - Raw Throughput Ideality
Fig. 6: Performance evaluation of the system with different configura-
tions and number of lanes on different kernels and vector lengths.
memory accesses, and fft by the indexed stores at the end
of the program. However, even considering these slower
kernels, the system achieves, on average, 50% of its raw
throughput ideality on all the kernels and configurations
starting from 128 Byte/Lane.
The Ara2 system achieves higher performance at lower
byte-per-lane ratios with the most crucial kernels like
fconv2d and fmatmul, reaching over 95% of its maximum
theoretical throughput starting from 128 Byte/Lane, or 75%
from 64 Byte/Lane.
7.3
Ideal Dispatcher System’s Performance
Replacing the scalar core (CVA6) with an ideal dispatcher
idealizes the scalar core, its memory system, and the non-
vectorial code. In this way, we can trace an upper bound
to the available performance achievable through the Ara2
system alone, abstracted from the scalar subsystem. With
an ideal dispatcher, the performance is no longer limited by
the issue-rate limitation described in [4], and CVA6 cannot
interfere with Ara’s memory transfers.
Figure 7 shows, for the 2- and 16-Lane system, the raw
throughput ideality gain we get by replacing CVA6 with
an ideal dispatcher. We report the plots for 64, 256, and
1024 elements. The yellow lines track the number of scalar
8
L1I-cache and L1D-Cache misses during the vector program
computation, from the first vector instruction to the last one.
When the number of Bytes per lane is lower than 8 (e.g.,
16-Lane Ara2 - 64B), the effective number of lanes is reduced
for 64-bit data kernels since there are not enough elements
to fill all the lanes, and the real maximum performance
achievable is the maximum performance multiplied by
the number of elements per lane. In this condition, the
performance is not limited by CVA6.
When the number of Bytes per lane is lower than 64
(2-Lane Ara2 - 64B and 16-Lane Ara2 - 256B), the effective
number of banks used in each lane is reduced from eight to
the number of elements per lane for the 64-bit data kernels,
especially if we do not implement a Barber Pole’s VRF layout.
For example, if the vector length is 128B and the system has
16 lanes, only one bank per lane is effectively used; all the
read and write operations will target the same bank in each
lane, increasing the probability of conflicts. Since multiple
PEs experience a stall when they try to access the same bank
simultaneously, and the conflict probability grows when we
reduce the number of banks, these configurations are slowed
down by more bank stalls. The lower the byte-per-lane ratio,
the more impact a bank stall will have on the final runtime.
In this region, we see that some kernels moderately
benefit from the ideal dispatcher. This also happens when the
number of elements grows until the number of byte-per-lane
becomes sufficiently long to peak performance, and the non-
idealities of the scalar core and memory system are almost
completely amortized by Ara.
fmatmul, fconv2d, and jacobi2d greatly benefit from
the ideal dispatcher, showing a correlation between the D-
Cache misses and their performance increase when CVA6 is
replaced with an ideal dispatcher. To prove the correlation,
we simulate the three kernels on a 16-Lane System with 128
elements (1024 Byte), idealizing the cache system so that it
always hits. Figure 8 shows the results. With an ideal cache
system, the three kernels virtually perform as when executed
by the ideal-dispatcher system, which shows the importance
of the scalar memory system for kernels that heavily rely
on operand forwarding between the scalar and the vector
core. Instead, exp and roi-align slightly benefit from the
ideal dispatcher because of the non-negligible amount of
housekeeping scalar code in the program.
7.4
Further Performance Considerations
In the following paragraphs, we discuss the effect of further
system/level and microarchitectural modifications used in
vector processors that can impact performance and identify
the main performance drivers that can boost the system’s
performance with low Byte/Lane ratios.
7.4.1 VRF Layout
The current instance of Ara2 does not implement Barber’s
Pole VRF layout, which was originally introduced to avoid
the initial stalls that every vector instruction with at least
two source registers would experience [4]. These stalls are
especially critical for short-vector applications where each
vector instruction lasts for fewer cycles.
Barber’s Pole does not always reduce the number of
runtime stalls, though. Highly regular applications such asUNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
9
Fig. 7: Scalar cache misses and performance gain when CVA6 is replaced by an ideal dispatcher, varying the number of lanes in the Ara2 system
and the number of elements in the vector, measured in bytes.
Fig. 8: Performance comparison of a 16-Lane System with 64B/Lane
among three different configurations: a) Baseline System, b) Baseline
System with ideal CVA6 data-cache, c) System with ideal dispatcher.
fmatmul suffer more stalls when implementing this byte
layout and dealing with vectors that are sufficiently long
to amortize the initial stalls. Without Barber’s Pole, when
an application is highly regular, and the vector processor
receives enough computational vector instructions to keep
its operand requesters busy, the VRF experiences a regular
access pattern during execution, especially when the vector
length is multiple of the number of banks in a lane. After the
first stalls, the source operand requesters make their requests
in an interleaved fashion, once per cycle, without conflicting
anymore. When an instruction is over, and the next one
starts, the pattern is preserved since the previous instruction
issued its last request to the last VRF bank, while the next
instruction will ask an operand to bank 0 one cycle later.
Instead, with Barber’s Pole layout, the next instruction will
likely start its requests from a different bank that depends
on the new source register. This uncertainty perturbates the
bank access pattern and creates new stalls.
Figure 9 shows this effect. We report the baseline system
performance with and without Barber’s Pole for fmatmul,
varying the number of elements. We also report the same
comparison but with CVA6 replaced by the ideal dispatcher.
Since the trends are similar, we can focus on the baseline
system. Barber’s Pole has a positive effect on performance
up to 16 double-precision elements (32 Byte per lane), as
it increases the number of effective banks used in each
lane. From 32 elements (64 Byte/lane, i.e., the effective
number of banks is the same as the physical number of
banks), the performance is decreased due to the newly
introduced conflicts. The longer the vector, the less frequent
the introduced stalls happen, as the fetch from the same
source registers is not perturbed by Barber’s Pole. Barber’s
Pole effect on performance is mainly driven by the number of
Bytes per lane; therefore, shorter data widths will experience
the negative effect of this byte layout at higher vector lengths.
Since Barber’s Pole layout increases the complexity of the
hardware that calculates the VRF addresses and can hurt
performance when handling long vectors, we decided not to
implement it in Ara2.
7.4.2
Main Performance Drivers
To fully characterize Ara2’s performance, we studied its
behavior when further optimized with modifications that
can highly impact its PPA metrics. To amortize the setup time
of the internal pipeline when the byte-per-lane ratio is low,
Ara2 needs to have larger buffers at each step of its pipeline.
We artificially increased the size of the instruction buffers
for each internal unit and the number AXI cut-registers
from/to Ara’s load-store unit. Moreover, we changed the
way in which the hazards on the load unit and slide unit
are handled, complicating the internal control but allowing
for faster hazard resolution. To further decrease the number
of stalls, another possible change would be to duplicate the
window of simultaneous instructions handled within Ara2
from 8 to 16. All these changes impact the area and timing
of a design that requires careful optimizations to balance
the timing paths; therefore, they are only used now as a
comparison point for more viable solutions presented in
Section 9.UNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
10
IR Lim.
Fig. 9: Barber’s Pole layout performance effects during a matrix
multiplication (8 Byte/element). Small benefits exist for shorter vectors,
while medium-long vectors suffer from it.
Figure 10 shows the performance of the optimized system
compared to the baseline. We also report the system, in both
configurations, attached to an ideal dispatcher. The black
lines represent the issue-rate limitation, i.e., the intrinsic
performance limitation due to the non-ideal rate at which the
scalar core issues the vector instructions to Ara. We can see
that the optimized system with the ideal dispatcher is not
limited by the issue-rate limitation and gets a non-negligible
performance boost for a vector 32, 64, and 128 bytes long
(up to 32 Byte/lane) if compared with the baseline with the
ideal dispatcher and with the current system baseline. On the
other hand, we can see that the current system, coupled with
CVA6, suffers from multiple problems with short vectors.
Namely, the fact that the housekeeping code overhead and
the setup time weigh more when the runtime is shorter
(smaller problem size).
Figure 11 summarizes the results of the performance
analysis, analyzing the various sources of inefficiencies for
the current system. Ara’s performance, when dealing with
shorter vectors, is limited by multiple factors that depend on
the vector length. We merged more contributions when their
combined effect was way higher than their effects added
(64B, 128B cases). CVA6 highly penalizes the 64B case only
if Ara2 is optimized; otherwise, its inefficiencies are hidden
by Ara’s ones. We also want to highlight that the CVA6- and
Cache-related columns are upper bounds to the system’s
performance. Further optimizations would require a heavy
re-design of the system, as the most impactful optimizations
are the merged ones (violet and orange columns). The effect
of the Ara2 inefficiencies on performance drops below 5%
from 256B on, demonstrating the computational efficiency of
the architecture for medium-long vectors.
8
P HYSICAL IMPLEMENTATION
We summarize the system implementation results in Table 3,
while Table 4 presents the performance, power, and energy
efficiency analysis for a matmul on different data types.
Table 5 shows the area breakdown of the system and the
cost of supporting arbitrary strides during slide operations
compared with the optimized system, which supports only
power-of-two strides and time-multiplexes the slide and
reshuffle operations, as explained in Section 5. When scaling
up, the all-to-all unoptimized (old) slide unit becomes the
IR Lim.
Fig. 10: Performance analysis with further Vector Processor streamlining
when executing a matrix multiplication (8 Byte/element). The issue-rate
limitation is a hard limit only for the system with CVA6 (opaque lines).
IR Lim.
Fig. 11: Main sources of inefficiency. Ara2’s architecture especially limits
short vectors from 32 to 128 Byte (up to 32 B/lane), while caches play
an important role for 128 and 256 Byte vectors. Kernel: floating-point
matrix multiplication (8 Byte/element).
largest non-lane block from four lanes on, growing exponen-
tially with the lanes and dominating the 8-lane design. The
mask unit only needs to support reshuffling and generate an
enable signal at bit granularity; however, no all-to-all input-
to-output byte mapping is required. The load and store units
quickly increase their size when scaling up the system but
at a lower pace thanks to their limited memory bandwidth
( 4 × L Byte/cycle), which is half of the byte throughput of
the main computational units ( 8 × L Byte/cycle) and needs
L 2
2 connections. Between these two units, the load unit is
the most problematic since its interconnect is duplicated
to connect the input AXI memory bytes to the two-entry
result queue. The store unit, instead, connects the input
register to the AXI bytes directly. The optimized slide unit
shows a linear increase, with an area reduction of 83% with
respect to the unoptimized one. The greater reduction in area
w.r.t. the predicted one can be explained by the diminished
routing density that was effectively dominating the unit area.
Looking at the 16-lane design with the new slide unit, we see
the mask unit and load unit exploding in size and hindering
the system’s scalability.
9
M ULTI -C ORE A NALYSIS
In this section, we analyze the performance and energy
efficiency trade-off for different system configurations (single-
core and multi-core) as a function of the application’s vector
length. The analysis from Section 8 suggests that 16 lanesUNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
11
TABLE 3: Physical implementation metrics in 22nm FD-SOI. In brackets,
the increment w.r.t. the same architecture with half the lanes.
Metric (factor)
2 Lanes 4 Lanes 8 Lanes 16 Lanes 16 Lanes*
SS Freq. [GHz]
TT Freq. [GHz] 950
1.35 960 (1.0 × )
1.35 (1.0 × ) 940 (1.0 × )
1.35 (1.0 × ) 750 (0.8 × )
1.08 (0.8 × ) 860 (0.9 × )
1.26 (0.9 × )
Die Area [mm 2 ]
Cell+Macro Area [kGE]
Macro Area [kGE]
Cell Area [kGE] 0.59
2291
557
1733 0.95 (1.6 × )
3688 (1.6 × )
769 (1.4 × )
2919 (1.7 × ) 1.88 (2.0 × )
6768 (1.8 × )
1180 (1.5 × )
5587 (1.9 × ) 4.47 (2.4 × )
14773 (2.2 × )
2010 (1.7 × )
12763 (2.3 × ) 4.47 (2.4 × )
12864 (1.9 × )
2010 (1.7 × )
10854 (1.9 × )
En. Eff.** [FLOPS DP /W]
34.1
37.8 (1.1 × )
35.7 (0.9 × )
* No fixed-point support, minimal mask unit.
** fmatmul between 256x256 matrices, at typical corner — 30.3 (0.8 × )
TABLE 4: Performance, power, and energy efficiency for different kernels
for a 4-lane design, at 1.35 GHz, in typical conditions, and 2KiB vectors.
Program
fmatmul64
fmatmul32
fmatmul16
imatmul64
imatmul32
imatmul16
imatmul8
Elements Power [mW] Performance [GOPS] Efficiency [GOPS/W]
256
512
1024
256
512
1024
2048 283
238
218
272
245
231
222 10.7
21.4
42.8
10.4
20.9
41.8
83.5 37.8
90.0
195.9
38.3
85.2
181.0
376.0
are a hard limit due to congestion and routing challenges
in the all-to-all units of the vector processor. Moreover, a
single-core vector processor needs to buffer longer vectors
exposed by the application on a single dimension to scale
up performance and efficiency when its number of lanes
increases, by keeping the same byte-per-lane ratio. However,
not all applications expose long vectors on one dimension.
A multi-core system offers flexibility in exploiting different
parallelization dimensions at the cost of synchronization and
memory-transfer overheads, as reported in Figure 13. The
experiment includes four systems with the same number of
FPUs (16), ranging from a single-core 16-lane Ara2 system to
a multi-core system consisting of eight 2-lane Ara2 systems
coupled with eight CVA6 instances. The results, depicted
in Figures 14, 15, and 16, showcase the raw throughput
(FLOP/Cycle), real throughput (FLOPS), and energy effi-
ciency (FLOPS/W) of the various system configurations.
9.1
Performance
The raw throughput allows for evaluating the system’s per-
formance abstracted from the physical implementation. For
medium/short vectors (8, 16, 32, 64 64-bit elements), multi-
core systems composed of smaller instances outperform
those with fewer but larger vector cores. This is due to the
exploitation of another dimension of parallelization, allowing
Fig. 12: Floorplan of the 8-lane Ara2 implementation.
TABLE 5: Ara2’s area breakdown and scale-up behavior. In brackets,
the area scaling factor w.r.t. the same block in Ara2 with half the lanes.
Thanks to the optimization of the SLDU, the MASKU and Vector Load
Unit (VLDU) are the main units whose area skyrockets during the
system upscaling. When we remove most of the shuffling functions from
the MASKU, its area is reduced by 60% in a 16-lane design.
Area [kGE] (factor) 2 Lanes 4 Lanes 8 Lanes 16 Lanes 16 Lanes*
CVA6
Lane
Dispatcher
Sequencer
MASKU
ADDRGEN
VLDU
VSTU
New SLDU 894
612
16
14
38
35
15
8
24 896 ( 1.0× )
617 ( 1.0× )
17 ( 1.1× )
15 ( 1.1× )
97 ( 2.5× )
36 ( 1.0× )
45 ( 3.0× )
21 ( 2.8× )
48 ( 2.0× ) 906 ( 1.0× )
626 ( 1.0× )
19 ( 1.1× )
17 ( 1.2× )
300 ( 3.1× )
44 ( 1.2× )
212 ( 4.7× )
64 ( 3.1× )
94 ( 2.0× ) 904 ( 1.0× )
628 ( 1.0× )
23 ( 1.2× )
29 ( 1.6× )
1105 ( 3.7× )
59 ( 1.3× )
1286 ( 6.1× )
332 ( 5.2× )
196 ( 2.1× ) 904 ( 1.0× )
573 ( 0.9× )
20 ( 1.1× )
29 ( 1.6× )
442 ( 1.5× )
60 ( 1.4× )
1135 ( 5.4× )
342 ( 5.3× )
190 ( 2.1× )
Old SLDU
39 131 ( 3.4× ) 577 ( 4.4× )
* No fixed-point support, minimal mask unit. 2900 ( 5.0× ) 2860 ( 5.0× )
Fig. 13: Comparison of a single-core 4-lane Ara2 system architecture
and a two-core architecture made of 2-lane Ara2 processors. Both
configurations have 4 FPUs in total. The vector processor can vectorize
in one dimension only, while the multi-core system can parallelize
multiple dimensions. This allows the multi-core system to reach a higher
Byte/Lane ratio when the vectorizable dimension is shorter than the size
of a vector register and the application exposes an additional dimension
to exploit DLP.
for a byte-per-lane ratio increase. In contrast, as the problem
size increases, the FPUs are fully utilized even with the
maximum number of lanes. The dual-core 8-lane and single-
core 16-lane systems surpass the others at vector lengths of
128 and 256 elements, respectively, as the synchronization
overhead and pressure on the memory system decrease
with fewer cores. Extremely small problem sizes (8x8x8)
suffer from the program setup time because of their lower
arithmetic intensity and shorter runtime.
Issue rate limitation: Thanks to the RVV specification
update, the number of assembly instructions in the main
loop of a matmul drops from four to three, bringing the
issue rate of vfmacc instructions from five to four cycles per
vfmacc (the scalar coefficient can now be forwarded with
the vfmacc). As shown in Figure 14, this new implicit scalar
forwarding contributes to pushing the issue-rate-limitation
line [4] to the left. However, the non-ideal issue rate of CVA6
still limits performance for medium-short vectors.
The multi-core vector architecture helps overcome the
issue-rate limitation. While the single-core 16-lane Ara2
system cannot theoretically go beyond 16 DP-GFLOP/cycle
when operating on 32x32x32 matrices, all the multi-core
instances with 16 FPUs exceed this value, with the 8-core
2-lane system reaching 23.6 DP-GFLOP/cycle. This happens
since the maximum performance of a 2-lane Ara2 system is
lower than the issue-rate limitation. Intuitively, having more
scalar cores makes the total issue rate increase, too. Also, aUNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
multi-core design with smaller Ara2 instances behaves better
than a single-core larger Ara2 with the same number of FPUs
coupled with an ideal scalar core and memory subsystem,
even if the latter system is not affected by the issue-rate
limitation. This result is shown in Figure 14.
The previous analysis is valid if the application exposes
two dimensions of parallelization. The applications that ex-
pose only one parallelizable dimension will always perform
better on a single-core larger implementation, as the multi-
core architecture will not help increase the byte-per-lane ratio
and will pay the synchronization price.
As shown in Figure 15, the overall picture of the sys-
tem’s performance does not change when we take into
consideration the real throughput of the various systems
by considering the maximum frequencies of the physical
implementations for the designs up to 8 lanes. The 16-lane
system experiences a non-negligible frequency drop that
penalizes the whole computation, which becomes slower
than all the other designs for all the different vector lengths.
9.2
Energy efficiency
With the multi-core system, we waste the energy of the
replicated scalar cores and their memory transfers. This
fact is counter-balanced by the different energy efficiencies
reached by the physical implementations of the systems with
a different lane count. As shown in Section 8, the 4-lane
design is the most efficient system, followed by the 8-lane,
2-lane, and 16-lane ones. The multi-core overhead and the
different efficiencies of the implementations allow the 4-core
4-lane design to be the most efficient one, reaching more
than 39 DP-GFLOPS/W on a 256x256x256 problem. The
2-core 8-lane system immediately follows with almost 38
DP-GFLOPS/W on the same problem, thanks to a lower
multi-core related power overhead. As expected, the 8-core
2-lane system shows an efficiency from 5% to 18% lower
than the 4-core, suffering both from the lower efficiency of
the single 2-lane Ara2 systems and the maximum number
of cores in the system, with the related overhead. The 16-
lane system suffers from the specification update and the
additional all-to-all connected unit that degrades the overall
energy efficiency. Nevertheless, it overtakes the 8-core 2-lane
system on the largest problem size.
In Figures 18 and 19, we present a log-log summary of the
previous diagrams. Colors indicate the number of FPUs, and
shapes indicate the number of lanes of each Ara2 instance
that makes up the multi-core system. The more complex the
shape, the higher the number of lanes per core.
The larger systems with fewer cores take over the
configurations with smaller cores (simple shapes on the plot)
when the number of FPUs is constant (same color on the plot)
when the vector length is increased, meaning that having
more lanes in the single instances is more beneficial with
longer vectors, as we noticed before.
The plots show that for smaller vectors, many smaller
vector processors are better than a large one that cannot
effectively exploit all the dimensions of parallelism present
in the application. If we take this to the limit, we see that
with extremely short vectors (i.e., 1 or 2 elements), having
a vector processor does not make sense, and a multi-core
CVA6 scalar system would be more performant and efficient.
12
10 C OMPARISON WITH S O A
10.1 Performance
As already noticed in Section 1, it is not easy to compare our
performance metrics with the SoA since the benchmarking is
often done on a very limited number of kernels and because
the software implementation is not available. Moreover, the
metrics are not always clearly defined. In [5], we could not
find the definition of the throughput metric and the lengths
of the application vectors. For this reason, we assume them
to be the longest possible, i.e., at least 256 DP-Elements per
vector (256 Byte/Lane). The 8-lane Ara2 system is almost
as performant as Vitruvius+ even with a 80KiB/32KiB =
2.5× smaller VRF and no register-renaming features. Indeed,
it gets the same performance for matmul and pathfinder,
while Ara2’s FFT is 1.3× faster even for a lower byte-per-
lane ratio (128 against 256). jacobi2d’s performance is hard
to compare as it is unclear how the authors calculated the
throughput metric, getting more than what we consider the
maximum throughput for that kernel (8 DP-FLOP/cycle).
Ara2’s performance for fmatmul, fconv2d gains a 1.1×
boost compared to [4] thanks to the frequency gain. Other
than that, the comparison with Hwacha done in [4] is still
valid, as no updated results have been published.
The comparison with Vicuna [9] is especially hard since
it is a 32-bit integer-only vector core. Its medium and fast
configurations have the same computational capabilities, in
terms of total throughput, of a 2-lane and 16-lane Ara2 sys-
tems. The reported utilization results are relative to two 8-bit
fmatmul whose sizes are comparable with a 256 × 32 × 256
and a 1024 × 128 × 1024 64-bit fmatmul problems. Our
results refer to 32x32x32 and 128x128x128 matrices, and this
makes the comparison unfair, as increasing the other two
dimensions boosts the utilization. Nevertheless, the 2-lane
Ara2 system shows virtually the same utilization. Our 16-lane
system, instead, lags behind ( ∼ 30% vs. ∼ 60% ), especially
when executing the smaller problem. This is mainly due
to the size of Ara’s D-Cache, 16× smaller than Vicuna’s
in the same configuration. Spatz [12], an integer vector
processor coupled with Snitch [28], is able to reach almost
70% utilization with eight cores and 2 Image Processing
Units (IPUs) per core, while Ara2 reaches 30% for the same
problem size. However, Spatz is a 32-bit integer-only vector
architecture, and all the benchmark data are kept in the low
latency L1 memory, which eliminates miss-related stalls.
10.2
Power, Performance, Area
The novel Ara2 architecture boosts the design’s operational
frequency up to 15% (i.e., 8 lanes) compared to Ara [4], with a
critical path of ∼ 40 FO4 inverter delays for the 4-lane design.
Its area is 2%, 7%, 15%, and 37% higher than Ara with 2,
4, 8, and 16 lanes, respectively. This is due to the newly
introduced functionality, coupled with a 4 × size reduction
of the VRF. The MASKU, introduced to keep the architecture
compliant with RVV, decreases the implementation efficiency
of the whole design starting from 8 lanes and a visible area,
and energy-efficiency degradation hinders a further scale-up.
Further study on the MASKU and VLDU is needed to reduce
their footprint without hitting the system’s performance.
E.g., the data of the VLDU can be aligned before entering
the system, easing the routing in the unit. On the otherUNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
Fig. 14: A multi-core implementation helps
overcome the issue-rate limitation for small
vectors (fmatmul).
13
Fig. 15: Multi- and single-core through-
put for different 16-FPU configurations
of the Ara2 system in typical conditions
(fmatmul).
Fig. 16: Multi- and single-core energy effi-
ciency for different 16-FPU configurations
of the Ara2 system in typical conditions
(fmatmul).
Fig. 17: Performance comparison of multi-
and single-core 16-FPU Systems (fmatmul). Fig. 18: Throughput comparison of single- Fig. 19: Energy Efficiency comparison of
In the last column of each set, CVA6 is and multi-core vector architectures with 2, 4, single- and multi-core vector architectures
replaced by the Ideal Dispatcher.
8, and 16 FPUs (fmatmul).
with 2, 4, 8, and 16 FPUs (fmatmul).
hand, a complete 8-lane Ara2 system (CVA6, Ara2, scalar
caches) reaches virtually the same frequency and area as
a Vitruvius+ instance alone without the scalar core and
memories (1.4 GHz, 1.3 mm 2 ) [5].
Comparing the energy efficiency across different imple-
mentations is hard due to the sensitivity that the power
consumption has with respect to the input data distribution.
E.g., executing the same 256x256x256 fmatmul kernel on our
4-lane Ara2 System can lead to energy efficiencies from 38.8
to 65 GFLOPS DP /W depending on the input distribution.
Using a normal distribution with mean 0 and variance 1,
instead of a uniform one between [0,1), also leads to a 7.5%
power difference. In this scenario, comparing our energy
efficiency with Vitruvius+ is hard, as the paper does not
report the used input number distribution, and it is not clear
if the power figures consider the scalar core and memories
too, as we do. Moreover, we could not reproduce the number
reported in their comparison tables. A comparison with Spatz
is also nontrivial, as the architecture is a 32-bit integer only,
leading to substantially lower power figures.
11
C ONCLUSION
In this work, we presented Ara2, the first open-source RVV
1.0 vector architecture. We implement the design with 2,
4, 8, and 16 lanes in 22nm FD-SOI technology and reach
1.35 GHz (for 8 and fewer lanes) thanks to a new lightweight
SLDU able to reduce the number of interconnect wiring
and hardware of the all-to-all byte-connected unit by 70%.
For compliance with RVV, Ara2 requires a new all-to-all
connected MASKU to support the modified VRF layout of
the mask vectors, complicating the physical implementation.
This, together with the newly supported features like floating-
point reductions, make the 4-lane configuration become the
most efficient design point, with an SoA energy efficiency of
almost 39 GFLOPS DP /W during a matrix multiplication.
The flexible Ara2 architecture reaches, on average, more
than 50% of its throughput ideality starting from 128 Bytes
per lane on the whole benchmark pool with every system
configuration. Furthermore, with only a 64 byte-per-lane
ratio, it achieves more than 75% of its maximum throughput
on the most crucial kernels like matrix multiplications and
convolutions and more than 90% from 128 Byte/Lane. We
also investigate the detrimental impact of the Barber’s Pole
layout on long-vector performance and thoroughly analyze
the primary performance drivers of the vector architecture,
including the scalar core and its memory system.
To overcome the issue-rate performance limitation, we
evaluate multi-core vector architectures, which show that
smaller vector core instances can boost the performance of
applications that expose more than one dimension of par-
allelization (e.g., matmul), especially when the vectorizable
dimension cannot provide high byte-per-lane ratios, with
performance improvements up to 3x when compared to a
single-core architecture with the same overall computation
capability. This is especially beneficial when the system
cannot exploit the advantages of long vectors, e.g., when
the application does not expose them on a single dimension.
A CKNOWLEDGMENTS
This work was supported by the ETH Future Computing
Laboratory (EFCL), financed by a gift from Huawei Technolo-UNDER REVIEW AT IEEE TRANSACTIONS ON COMPUTERS
gies, and by the TRISTAN (101095947) project that received
funding from the HORIZON KDT-JU programme.
R EFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
R. M. Russell, “The CRAY-1 computer system,” Commun. ACM,
vol. 21, no. 1, p. 63–72, 1978.
Fujitsu,
“How
world-first
technology
is
unleashing
higher
processing
speeds
for
AI’s
ever-rising
computational
needs,”
2020.
[Online].
Available:
https://corporate-blog.global.fujitsu.com/fgb/2020-02-03/how-
world-first-technology-is-unleashing-higher-processing-speeds-
for-ais-ever-rising-computational-needs/
J. Ma et al., “An analytical framework for estimating scale-out
and scale-up power efficiency of heterogeneous manycores,” IEEE
Transactions on Computers, vol. 65, no. 2, pp. 367–381, 2016.
M. Cavalcante et al., “Ara: A 1-GHz+ scalable and energy-efficient
RISC-V vector processor with multiprecision floating-point support
in 22-nm FD-SOI,” IEEE Transactions on Very Large Scale Integration,
vol. 28, no. 2, pp. 530–543, 2020.
F. Minervini et al., “Vitruvius+: An area-efficient RISC-V decoupled
vector coprocessor for high performance computing applications,”
ACM Trans. Archit. Code Optim., vol. 20, no. 2, pp. 1–25, 2023.
R. Espasa, “Introducing SemiDynamics high bandwidth RISC-
V IP cores,” RISC-V Global Forum 2020, 2020. [Online].
Available: https://www.european-processor-initiative.eu/wp-
content/uploads/2021/03/202012.RISCV-SUMMIT.pdf
I. A. Assir et al., “Arrow: A RISC-V vector accelerator for machine
learning inference,” arXiv preprint arXiv:2107.07169, 2021.
SiFive
Intelligence
X280,
SiFive
Corp.,
San
Mateo,
CA,
USA,
2022,
revision
21G3.
[Online].
Avail-
able: https://sifive.cdn.prismic.io/sifive/62e0df53-be02-4b50-
b211-aa55b7042fc8 x280-datasheet-21G3.pdf
M. Platzer and P. Puschner, “Vicuna: A timing-predictable RISC-
V vector coprocessor for scalable parallel computation,” in 33rd
ECRTS, ser. LIPIcs, vol. 196. Schloss Dagstuhl, 2021, pp. 1:1–1:18.
C. Chen, X. Xiang et al., “Xuantie-910: A commercial multi-core
12-stage pipeline out-of-order 64-bit high performance RISC-V
processor with vector extension: Industrial product,” in ISCA’20,
2020, pp. 52–64.
AndesCore NX27V, Andes Technology, Hsinchu City, Taiwan, 2020.
[Online]. Available: http://www.andestech.com/en/products-
solutions/andescore-processors/riscv-nx27v
M. Cavalcante et al., “Spatz: A compact vector processing unit
for high-performance and energy-efficient shared-L1 clusters,” in
ICCAD’22. ACM, Oct. 2022, pp. 1–9.
SiFive Performance P270, SiFive Corp., San Mateo,
CA,
USA,
2022,
revision
21G3.
[Online].
Avail-
able: https://sifive.cdn.prismic.io/sifive/859c28c0-8bd5-4fc4-9113-
a25a2a89bf9c P270+Data+Sheet.pdf
SiFive, “P870 high-performance RISC-V processor,” in Hot Chips: A
Symposium on High-Perf. Chips. IEEE, Aug. 2023.
C. Ramı́rez et al., “A RISC-V simulator and benchmark suite for
designing and evaluating vector architectures,” ACM Trans. Archit.
Code Optim., vol. 17, no. 4, pp. 1–30, 2020.
H. T. Kung, “Memory requirements for balanced computer archi-
tectures,” SIGARCH Comp. Arch. News, vol. 14, no. 2, May 1986.
P. Vizcaino et al., “Short reasons for long vectors in HPC CPUs: a
study based on RISC-V,” arXiv preprint arXiv:2309.06865, 2023.
M. Johns and T. J. Kazmierski, “A minimal risc-v vector processor
for embedded systems,” in FDL’20, 2020, pp. 1–4.
K. Patsidis et al., “RISC-V2: A scalable RISC-V vector processor,” in
ISCAS’20, 2020, pp. 1–5.
C. Schmidt et al., “An eight-core 1.44-GHz RISC-V vector processor
in 16-nm FinFET,” IEEE Journal of Solid-State Circuits, vol. 57, no. 1,
pp. 140–152, 2022.
——, RISC-V Vector Extension Proposal, 2015. [Online].
Available: https://riscv.org/wp-content/uploads/2015/06/riscv-
vector-workshop-june2015.pdf
Krste Asanovic, Vector Extension Proposal v0.2, 2016. [Online]. Avail-
able: https://riscv.org/wp-content/uploads/2016/12/Wed0930-
RISC-V-Vectors-Asanovic-UC-Berkeley-SiFive.pdf
K. Asanovic et al., The RISC-V Vector ISA, 2017. [Online].
Available: https://riscv.org/wp-content/uploads/2017/12/Wed-
1330-RISCVRogerEspasaVEXT-v4.pdf
14
[24] ——, RISC-V Vector Extension Proposal, 2018. [Online].
Available: https://riscv.org/wp-content/uploads/2018/05/15.20-
15.55-18.05.06.VEXT-bcn-v1.pdf
[25] ——, RISC-V Vector Extension Proposal, 2020. [Online]. Available:
https://github.com/riscv/riscv-v-spec/releases/tag/0.9
[26] L. Bertaccini, L. Benini, and F. Conti, “To buffer, or not to buffer?
a case study on FFT accelerators for ultra-low-power multicore
clusters,” in ASAP’21, 2021, pp. 1–8.
[27] Cristóbal Ramı́rez Lazo, “RiVec benchmark suite.” [Online]. Avail-
able: https://github.com/RALC88/riscv-vectorized-benchmark-
suite
[28] F. Zaruba et al., “Snitch: A tiny pseudo dual-issue processor for
area and energy efficient execution of floating-point intensive
workloads,” IEEE Transactions on Computers, vol. 70, no. 11, pp.
1845–1860, 2020.
Matteo Perotti received his M.Sc. degree in Elec-
tronic Engineering from the Polytechnic University
of Turin, Italy, in 2019. He is currently pursu-
ing a Ph.D. degree at the Integrated Systems
Laboratory of ETH Zurich, Switzerland, under
the supervision of Prof. Luca Benini. Matteo’s
research interests include highly efficient comput-
ing architectures, vector processing, computation
with high dynamic-range data types, and the
RISC-V ecosystem in general.
Matheus Cavalcante received his M.Sc. degree
in Integrated Electronic Systems from the Greno-
ble Institute of Technology (Phelma) in 2018, and
his Ph.D. from ETH Zurich in 2023. During his
Ph.D. studies, Matheus worked with the Digital
Circuits and Systems Group under the super-
vision of Prof. Luca Benini. Matheus’ research
interests include vector processing, large-scale
high-performance computer architectures, and
emerging VLSI technologies.
Renzo Andri received the B.Sc., M.Sc. and
Ph.D. degree in Electrical Engineering and In-
formation Technology at ETH Zurich in 2013,
2015, and 2020, respectively. Since 2020 he is
working at the Huawei Research Center Zurich.
His research focuses lies on energy-efficient ma-
chine learning acceleration and processor design
down to full-custom IC design including hardware-
algorithm codesign. In 2019, he won the IEEE
TCAD Donald O. Pederson Award.
Lukas Cavigelli received the d Ph.D. degree in
electrical engineering and information technology
from ETH Zurich in 2019. After a year as a Post-
doc at ETH Zurich, he has joined Huawei’s Zurich
Research Center in Spring 2020. His research
interests include deep learning, computer vision,
embedded systems, and low-power integrated
circuit design. He has received several best pa-
per awards including the IEEE TCAD Donald O.
Pederson award in 2019.
Luca Benini holds the chair of Digital Circuits
and Systems at ETH Zurich and is Full Professor
at the Università di Bologna. He received a Ph.D.
from Stanford University. Dr. Benini’s research
interests are in energy-efficient parallel comput-
ing systems, smart sensing micro-systems and
ML hardware. He is a Fellow of the ACM and a
member of the Academia Europaea. He received
the IEEE Mac Van Valkenburg Award in 2016,
the ACM/IEEE A. Richard Newton Award in 2020,
and the IEEE E. McCluskey Award in 2023.
