User Manual of C3V-W Dual (LPDDR4) EVB

This manual serves as a guide for utilizing the C3V-W Dual (LPDDR4) EVB. The C3V-W Dual (LPDDR4) EVB is engineered with three primary objectives:

With mounting two NPUs, NN computing power is doubled.
With mounting another 3.75GB DRAM on slave C3V-W, comprise 11.75GB DRAM totally, for LLM testing.
Test the CPIO functionality.

The original design purpose of CPIO is to allow C3V-W to connect to a P-chip (peripheral chip) through CPIO bridge. The P-chip is a custom-designed peripheral. C3V-W, the C-chip (computing chip), can connect to either a P-chip, or another C-chip.

The C3V-W Dual (LPDDR4) EVB comprises two subsystems: one is Master C3V-W subsystem, which operates Linux software, and the other is Slave C3V-W subsystem, offering additional 3.75 GiB DRAM, a NPU, and various peripherals (including USB3, SD card, GPIO, and more). Please refer to the functional block diagram of the C3V-W Dual (LPDDR4) EVB. The Master C3V-W subsystem features 8 GiB eMMC, 8 GiB LPDDR4 SDRAM, an SD card slot, USB3 Type C port, and a 10M/100M/1000M Ethernet port. On the other hand, the Slave C3V-W subsystem includes 8 GiB eMMC, 8 GiB LPDDR4 SDRAM, an SD card slot, and USB3 Type C port. The CPIO bus connects both subsystems together.

Functional Block Diagram of C3V-W Dual (LPDDR4) EVB

Table of Contents

1 1. Main Devices or Interfaces Description
2 2. Boot Devices and Configuration
3 3. Map of Addressing Space of Master C3V-W.
4 4. Setup the CPIO Interface
- 4.1 4.1 Menu config setup of x-boot
- 4.2 4.2 Synchronization of the CPIO interfaces
  - 4.2.1 GPIO Signals
  - 4.2.2 Procedure
5 5. Boot Flow of Software
6 6. Log of Master and Slave C3V-W Subsystem
- 6.1 6.1 Log of Master C3V-W:
- 6.2 6.2 Log of Slave C3V-W:

1. Main Devices or Interfaces Description

The picture below is a photo of the C3V-W Dual (LPDDR4) EVB.

The table below outlines the main components and interfaces:

Items	Subsystem	Explanations
1	Global	12V DC power input. The diameter of the DC Jack plug is 5.5mm. The power supply current of the adapter must be greater than 1A.
2	Global	Main-power switch. Turn down to ON, and turn up to OFF.
3	Slave	Pin-headers J10: For connecting I2C0 and I2C1 signals of Slave C3V-W. Pin-headers J12: For connecting SPI_CB4 signals of Slave C3V-W. Pin-headers J14: For connecting PWM (0, 1) and I2C2 signals of Slave C3V-W.
4	Slave	CM4 console (UA6) of Slave C3V-W. Note that the GND pin is at the bottom-most pin of the 3x1 pin-header. This is default serial port of Cortex M4. The default baud rate is 115,200. No parity and 1 stop-bit.
5	Slave	Main console (UA0) of Slave C3V-W. Note that the GND pin is at the bottom-most pin of the 3x1 pin-header. This is default serial port of i-boot, x-boot, Trusted Firmware-A (TF-A), OP-TEE, U-Boot and Linux kernel. The default baud rate is 115,200. No parity and 1 stop-bit.
6	Slave	8 GiB LPDDR4 SDRAM of Slave C3V-W
7	Global	Boot configuration switch
8	Slave	Jumper header J31 is designed for supplying power to burn the OTP of the Slave C3V-W chip. Simply insert a jumper into this header to activate the power supply for the burning process.
9	Master	Socket of micro SD card of Master C3V-W
10	Slave	The Type C socket of USB 3.1 Gen1 of Slave C3V-W, supporting Low, Full, High, and Super speeds. Additionally, it offers support for Host, Device, and Dual-Role Data (DRD) functionalities. The current limit for VBUS is set at 1A.
11	Slave	Slave C3V-W chip (15mm x 15mm, 526-pin, TF-BGA)
12	Slave	8 GiB eMMC (FGBA-153) of Slave C3V-W
13	Master	RJ-45 socket of Ethernet of Master C3V-W, supporting 10M, 100M, and 1000M speeds.
14	Slave	Socket of micro SD card of Slave C3V-W
15	Master	8 GiB eMMC (FGBA-153) of Master C3V-W
16	Master	Jumper header J26 is designed for supplying power to burn the OTP of the Master C3V-W chip. Simply insert a jumper into this header to activate the power supply for the burning process.
17	Master	Master C3V-W chip (15mm x 15mm, 526-pin, TF-BGA)
18	Master	The Type C socket of USB 3.1 Gen1 of Master C3V-W, supporting Low, Full, High, and Super speeds. Additionally, it offers support for Host, Device, and Dual-Role Data (DRD) functionalities. The current limit for VBUS is set at 1A.
19	Global	Reset key. Reset CM4 and main power-domains, but does not reset RTC.
20	Master	CM4 console (UA6) of Master C3V-W. Note that the GND pin is at the left-most pin of the 3x1 pin-header. This is default serial port of Cortex M4. The default baud rate is 115,200. No parity and 1 stop-bit.
21	Master	8 GiB LPDDR4 SDRAM of Master C3V-W
22	Master	Main console (UA0) of Master C3V-W. Note that the GND pin is at the right-most pin of the 3x1 pin-header. This is default serial port of i-boot, x-boot, Trusted Firmware-A (TF-A), OP-TEE, U-Boot and Linux kernel. The default baud rate is 115,200. No parity and 1 stop-bit.
23	Master	Pin-headers J11: For connecting I2C0 and I2C1 signals of Master C3V-W. Pin-headers J13: For connecting SPI_CB5 signals of Master C3V-W. Pin-headers J15: For connecting PWM (0, 1) and I2C2 signals of Master C3V-W.

2. Boot Devices and Configuration

The C3V-W Dual (LPDDR4) EVB supports booting from SD card and eMMC for both Master and Slave C3V-W subsystems. Refer to the table below for the selection of boot configuration switches for Master and Slave C3V-W subsystems.

Master	Slave	Boot Configuration Switch

Master	Slave	Boot Configuration Switch
SD Card	SD Card
SD Card	eMMC
eMMC	SD Card
eMMC	EMMC

3. Map of Addressing Space of Master C3V-W.

Refer to the map below for the address space from the perspective of Master C3V-W when using 8 GiB LPDDR4 SDRAM:

Lower 3.75 GiB Space: This is mapped to the bottom 3.75 GiB of the 8 GiB DRAM in the Master C3V-W subsystem.
Device Registers (0.25 GiB): A contiguous 0.25 GiB space is allocated for device registers specific to the Master C3V-W subsystem.
Upper 4 GiB Space: This is mapped to the upper 4 GiB of the 8 GiB DRAM in the Master C3V-W subsystem.
Slave C3V-W Subsystem (4 GiB): Starting from address 0x2 0000 0000, a segment of 4 GiB is dedicated to the Slave C3V-W subsystem.
Remaining Lower DRAM Space (0.25 GiB): The final 0.25 GiB space, commencing from address 0x3 0000 0000, is allocated for the remaining portion of the lower 4 GiB of the 8 GiB DRAM in the Master C3V-W subsystem.

In this configuration, the address space beginning from 0x2 0000 0000 is specifically assigned to the lowest 4 GiB space of the Slave C3V-W subsystem through the CPIO interface. Consequently, if the CPU of the Master C3V-W board intends to access the DRAM of devices on the Slave C3V-W subsystem, it must reference the base address of the Slave C3V-W subsystem, starting from 0x2 0000 0000.

The memory node in device-tree source should looks like this:

	memory@0 {
		device_type = "memory";
		reg = <0x0 0x0        0x0 0xf0000000>,  /* Lower 3.75 GiB */
		      <0x1 0x0        0x1 0x00000000>,  /* Upper 4 GiB */
		      <0x2 0x0        0x0 0xf0000000>,  /* Slave C3V-W Subsystem 4 GiB */
		      <0x3 0x0        0x0 0x10000000>;  /* Remaining Lower DRAM 0.25 GiB */
	};

This configuration ensures that the Master C3V-W can efficiently access both its own memory and the memory allocated to the Slave C3V-W subsystem.

4. Setup the CPIO Interface

The CPIO interfaces are setup in x-boot for both Master and Slave C3V-W chip.

4.1 Menu config setup of x-boot

For Master C3V-W, please navigate to the project's top directory and execute the command:

make xconfig

In the ensuing menu, depicted in the image below, navigate the cursor to "CPIO Mode" and select "Master."

For Slave C3V-W, similarly, please navigate to the project's top directory and execute the command:

make xconfig

In the menu displayed, as illustrated in the accompanying image below, maneuver the cursor to "CPIO Mode" and choose "Slave."

4.2 Synchronization of the CPIO interfaces

To ensure seamless operation, the CPIO interfaces of both the Master and Slave C3V-W must be synchronized during initialization and training. This synchronization is facilitated through the exchange of GPIO signals.

GPIO Signals

Direction	Master C3V-W	Slave C3V-W

Direction	Master C3V-W	Slave C3V-W
Slave → Master	GPIO72 (RX)	GPIO96 (TX)
Master → Slave	GPIO74 (TX)	GPIO94 (RX)

Procedure

Master C3V-W: Initially, the Master C3V-W awaits GPIO72 to register a HIGH signal. Once received, GPIO74 is set to HIGH, signaling the commencement of CPIO initialization and training.
Slave C3V-W: Upon detecting a HIGH signal on GPIO96, the Slave C3V-W sets GPIO94 to HIGH and waits for GPIO94 to register a HIGH signal. Upon receiving the HIGH signal on GPIO94, the Slave C3V-W initiates CPIO initialization and training.

5. Boot Flow of Software

Refer to flow chart of C3V-W Dual (LPDDR4) EVB below:

The boot process follows a predefined sequence:

Master C3V-W: The boot sequence initiates with i-boot, followed by x-boot, TF-A (Trusted Firmware-A), OP-TEE (Open Portable Trusted Execution Environment), U-Boot, and ultimately Linux. Throughout this sequence, the Master C3V-W subsystem undergoes a series of initialization and configuration steps to prepare for system operation.
Slave C3V-W: Conversely, upon completion of DRAM initialization, the boot process for the Slave C3V-W subsystem ceases. Unlike the Master C3V-W subsystem, it does not proceed to execute subsequent software components. However, it remains operational within the system, providing access to its devices, including DRAM, for the Master C3V-W subsystem.

6. Log of Master and Slave C3V-W Subsystem

6.1 Log of Master C3V-W:

[    2.569441] Run /sbin/init as init process
/etc/init.d/rcS starts...
Mounting other filesystems ...
rc.extra [bg]
sdcard boot set...
[    2.795789] remoteproc remoteproc0: powering up f800817c.remoteproc
[    2.798426] remoteproc remoteproc0: Booting fw image firmware, size 244280
[    2.799348] virtio_rpmsg_bus virtio0: rpmsg host is online
[    2.804314] remoteproc0#vdev0buffer: registered virtio0 (type 7)
[    2.810305] remoteproc remoteproc0: remote processor f800817c.remoteproc is now up
[    2.815307] virtio_rpmsg_bus virtio0: creating channel rpmsg-tty-raw addr 0x0
[    2.826105] virtio_rpmsg_bus virtio0: creating channel rpmsg-tty-raw addr 0x1
[    2.838487] virtio_rpmsg_bus virtio0: creating channel rpmsg-tty-raw addr 0x2
Boot CM4 firmware by remoteproc
extra done
End of /etc/init.d/rcS

login[143]: root login on 'console'
~ # [    2.983820] fbcon: Taking over console
~ # devmem 0x2f8800000
0x00000A30
~ #

After successful booting of Linux, you can execute the command

devmem 0x2f8800000

to retrieve the chip ID of the Slave C3V-W. A correctly functioning CPIO interface should yield a chip ID of 0x00000A30 for the Slave C3V-W. This confirmation signifies the proper operation of the CPIO interface.

6.2 Log of Slave C3V-W:

Run draiminit@0xFA20859D
bootdevice=0x00000019
Built at Mar 13 2024 19:47:40
dram_init
dwc_umctl2_lpddr4_1600_SP7350_for_realchip
MT53E2G32D4_C, 2rank, FBGA=D8CJN
SDRAM_SPEED_1600
dwc_ddrphy_phyinit_main 20231212
dwc_ddrphy_phyinit_out_lpddr4_train1d2d_3200_SP7350
bootdevice:0x00000019
XBOOT_len=0x0000B278
1D IMEM checksum ok
1D DMEM checksum ok
Start to wait for the training firmware to complete v.00 !!!
End of CA training.
End of initialization.
End of read enable training.
End of fine write leveling.
End of read dq deskew training.
End of MPR read delay center optimization.
End of Wrtie leveling coarse delay.
End of write delay center optimization.
End of read delay center optimization.
End of max read latency training.
Training has run successfully.(firmware complete)
bootdevice:0x00000019
2D IMEM checksum ok
2D DMEM checksum ok
Start to wait for the training firmware to complete v.00 !!!
End of initialization.
End of 2D write delay/voltage center optimization.
End of 2D write delay/voltage center optimization.
End of 2D read delay/voltage center optimization.
End of 2D read delay/voltage center optimization.
Training has run successfully.(firmware complete)
Register programming done!!!
Register programming done!!!
dram_init_end
Done draiminit
dram test 0x00800000 - 0x00800400


---- CPIO-R slave mode Begin ----

VCO: 4.0G, PLL: 1.0G
PHY status change: 0x08000001
PHY status check Passed
CPIO Initial Finished
PHY Mode: 0x0000008D
Timer start: 0x00000000
Timer End: 0x000000B2

After completing DRAM initialization and training, the Slave C3V-W subsystem proceeds to set up and establish the CPIO interface. Once the interface is configured and connected, the Slave C3V-W subsystem ceases further execution and enters a stopped state.