bootloader for quartz64-a

[jyoung]

Hi,

I'm setting up gentoo on a quartz64 model-a (64-bit arm SBC), and I'm stumped on the bootloader. From what I've read, the only option is the u-boot bootloader. However, building u-boot for this processor (rk3566) requires a TPL binary. Here's what I've done:

First, some info on the board

www.pine64.org/quartz64a/

The pine64 folks do provide a small amount of information about setting up a bootloader
https://wiki.pine64.org/wiki/Quartz64_UEFI_with_U-Boot

Crucially, this does eventually lead to a github page for a u-boot build appropriate for this processor
https://github.com/Kwiboo/u-boot-rockchip.git

Okay, when I clone this page, setup the config for this processor, and then make

Code: Select all

make quart64-a-rk3566_defconfig
make -j 4 all

I get

Code: Select all

Image 'simple-bin' is missing external blobs and is non-functional: rockchip-tpl atf-bl31

/binman/simple-bin/mkimage/rockchip-tpl (rockchip-tpl):
   An external TPL is required to initialize DRAM. Get the external TPL
   binary and build with ROCKCHIP_TPL=/path/to/ddr.bin. One possible source
   for the external TPL binary is https://github.com/rockchip-linux/rkbin.

/binman/simple-bin/fit/images/@atf-SEQ/atf-bl31 (atf-bl31):
   See the documentation for your board. You may need to build ARM Trusted
   Firmware and build with BL31=/path/to/bl31.bin

Image 'simple-bin' is missing optional external blobs but is still functional: tee-os

/binman/simple-bin/fit/images/@tee-SEQ/tee-os (tee-os):
   See the documentation for your board. You may need to build Open Portable
   Trusted Execution Environment (OP-TEE) and build with TEE=/path/to/tee.bin

Image 'simple-bin-spi' is missing external blobs and is non-functional: rockchip-tpl

The TPL error seemed the most urgent since it came with the "non-functional" tag. In menuconfig, it's possible to disable TPL functionality altogether

Code: Select all

 [*] Use external TPL binary

But that's probably a bad idea since it looks like this would render u-boot unable to read the system RAM
https://lore.kernel.org/all/20230214103 ... kwiboo.se/

The error message points toward this github for the required binary
https://github.com/rockchip-linux/rkbin

It's a little sparse on documentation, but it seems to contain several binaries that sound appropriate:

Code: Select all

rk35/rk3566_ddr_1056MHz_eyescan_v1.16.bin  rk35/rk3566_ddr_1056MHz_v1.18.bin       rk35/rk3566_ddr_920MHz_ultra_v1.10.bin
rk35/rk3566_ddr_1056MHz_ultra_v1.10.bin    rk35/rk3566_ddr_528MHz_ultra_v1.10.bin  rk35/rk3566_ddr_920MHz_v1.17.bin
rk35/rk3566_ddr_1056MHz_v1.17.bin          rk35/rk3566_ddr_780MHz_ultra_v1.10.bin  rk35/rk3566_ddr_920MHz_v1.18.bin

Here's where I'm stumped: which of these should I use, and how do I tell the u-boot build about the path? The u-boot menuconfig doesn't seem to have an option called ROCKCHIP_TPL. I've also been able to find out what kind of memory I have. I know that seems like a really basic thing, but I've not been able to find documentation, and the system doesn't seem to know

Code: Select all

dmidecode -t 17
# dmidecode 3.3
# No SMBIOS nor DMI entry point found, sorry.

Trying other approaches, I get the same requirement for a TPL binary when I try other u-boot githubs, for example

https://github.com/u-boot/u-boot

Finally, here is an excellent wiki for the quartz64pro, including a pre-built image

https://wiki.gentoo.org/wiki/Embedded_H ... uartzPro64

However, the quartz64pro uses an rk3588 processor, which is a different option in the u-boot build, so the pre-built image doesn't work, and the u-boot github linked therein doesn't support the rk3566. It does support the rk3568, but the thread I linked above indicates that that has the same TPL problem.

Lastly, I've been working from an armbian image, which loads without a hitch, so I know that the hardware is okay.

I'd be grateful for any help!

[pingtoo]

Are you intent to replace the u-boot with your own? Setup Gentoo does not require a new u-boot.

Setup Gentoo primary is create a rootfs with Gentoo content. Optionally replace kernel with you own compilation.

So I suggest you use existing bootstrap process get a working rootfs with Gentoo content before trying to get u-boot work.

If you really want to replace u-boot, we can try try to review the process in more detail to understand what procedure will lead to success.

[jyoung]

Okay, some progress: NeddySeagoon pointed me toward this tutorial

https://gitlab.collabora.com/hardware-e ... m_uboot.md

Technically, this is for the rk3588, but it does explain how to pass the TPL binary path to make

Code: Select all

export ROCKCHIP_TPL=../rkbin/bin/rk35/rk3588_ddr_lp4_2112MHz_lp5_2736MHz_v1.08.bin

I'm still not sure what binary is the right choice for me. Does anyone know how to determine the stats on system memory? But there are only nine options for the rk3566, so for now let's pick one a try it:

Code: Select all

export ROCKCHIP_TPL=../rkbin/bin/rk35/rk3566_ddr_1056MHz_ultra_v1.10.bin

Now, make gets past this step. Excellent! But it still fails on

Code: Select all

Image 'simple-bin' is missing external blobs and is non-functional: atf-bl31

/binman/simple-bin/fit/images/@atf-SEQ/atf-bl31 (atf-bl31):
   See the documentation for your board. You may need to build ARM Trusted
   Firmware and build with BL31=/path/to/bl31.bin

The tutorial address this too, with a recommend rkbin github

https://gitlab.collabora.com/hardware-e ... 3588/rkbin

and then

Code: Select all

export BL31=../rkbin/bin/rk35/rk3588_bl31_v1.28.elf

But this is weird, since neither this github nor the rkbin github that I linked earlier contain this file. Perhaps more critically for me, neither contains a bl31 binary for the rk3566.

[jyoung]

pingtoo, that sounds like a good idea. Can you post a link to that material?

[pingtoo]

pingtoo, that sounds like a good idea. Can you post a link to that material?

I don't know any specific online instruction. But you should be able to do just like any Gentoo installation with alternative OS.

Since you have armbian image working. Just boot off from armbian, then follow Getnoo chroot install procedure, for example follow Gentoo wiki https://wiki.gentoo.org/wiki/Installati ... stribution

[jyoung]

Interesting, this aligns fairly well with one of my first attempts. I tried to boot off armbian and install gentoo on a second disk using the normal install procedures. But, I'd still need to setup a bootloader. I first tried grub, but I don't think that grub is supports this type of board (correct me if I'm wrong here, using grub would be much easier). After that, I began experimenting with u-boot.

Another idea I tried was to write the armbian disk to an image, but overwrite the linux partition with gentoo, carefully preserving the /boot path structure so that the bootloader on the armbian image would lock onto the gentoo kernel image. That didn't work either, although I might try that again.

Or, are you thinking of something different pingtoo? One of the items in the tutorial you linked suggested installing gentoo side by side on the same disk as an existing install. That's an interesting idea, although we'd need to get the bootloader to detect gentoo as an option.

[NeddySeagoon]

jyoung,

You also need /lib/modules from the donor OS, as they will match to donor kernel.
You may need /lib/firmware too, in case there are any special firmware files that Gentoo will not provide.

In general, if you get all the bits in / that the donor needs, it should get you started.

[pingtoo]

I don't have rk3588 board on hand, so I cannot give you specific instruction.

Usually u-boot have autoboot feature turn on but the setting will give you a few seconds before device probing. so you can press any key to interrupt boot sequence. this will drop you at u-boot prompt.

u-boot is equivalent to grub in PC world. with some effort it is possible to program u-boot to give a menu to allow you select boot media and/or program to run (kernel is just a elf program)

I will use my rk3399 board (nanopi r4s) as example. my board boot off from a microSD card, so it is show as MMC in Linux kernel.

Code: Select all

lsblk -o NAME,FSTYPE,FSVER,LABEL,PARTLABEL,UUID,FSAVAIL,FSUSE%,MOUNTPOINT
NAME            FSTYPE      FSVER LABEL    PARTLABEL UUID                                   FSAVAIL FSUSE% MOUNTPOINT
sda             LVM2_member                          VVuvLx-YxDN-7Eoh-3ZC3-WHA6-7qfG-qfmsVE                
├─vg0-swapvol   swap              SWAP                                                                     [SWAP]
├─vg0-dockervol xfs               DOCKER             28472ce5-a147-41ee-a63f-ded30d830c4a     63.9G     2% /opt/docker
├─vg0-tftpvol   ext4                                 05247fe1-b875-4b18-b140-ed8aab9d6a82      3.5G    37% /tftp
├─vg0-nfsvol    ext4                                 2b255df1-0f84-4694-bf0a-b622288b43c7     46.4G     0% /nfs
└─vg0-zramvol                                                                                              
mmcblk1                                                                                                    
├─mmcblk1p1                                uboot                                                           
├─mmcblk1p2                                trust                                                           
├─mmcblk1p3                                misc                                                            
├─mmcblk1p4                                dtbo                                                            
├─mmcblk1p5                                resource                                                        
├─mmcblk1p6                                kernel                                                          
├─mmcblk1p7                                boot                                                            
├─mmcblk1p8     ext4              rootfs   rootfs    ff313567-e9f1-5a5d-9895-3ba130b4a864                  
└─mmcblk1p9     ext4              userdata userdata  269e9152-5cb6-035b-9c34-e1ed87cca310

Please have a look at the "PARTLABEL" column. In my case you can see that microSD card was partitioned for different purpose. this node is based on OpenWrt so in your case it may be different. But the general ideal is use the rootfs partition Or if you use additional attached storage device like a USB disk to load the gentoo stage3.

if your boot sequence were displayed on your monitor than you can try to tell u-boot to your own command line options to select kernel/rootfs. otherwise you need to attach a serial console to yoir SBC to get u-boot prompt that you can specify correct rootfs info for booting.

[jyoung]

NeddySeagoon, if I got that route, would I then be able to install a gentoo kernel?

pingtoo, the u-boot that comes with armbian isn't able to display, at least not on the hdmi monitor that I have. I've been trying to get a display through the serial pins through a usb adapter using minicom, but without much luck. This wiki gives the pin connections in the "Connect Debug UART" section

https://wiki.pine64.org/wiki/Quartz64

And this is the device that I've been using

http://www.adafruit.com/product/954

[NeddySeagoon]

jyoung,

If the gentoo kernel supports your hardware, yes.

The userland is the easy bit. The kernel can be difficult if you need to track down the device tree sources and vendor specific patches, or close to impossible if you really need evil binary only kernel modules. They will be tied to specific kernel and gcc versions.

e.g. My old arm64 chromebook needs the 3.14 kernel to have any video acceleration at all as the video driver is an evil binary only kernel module.
That won't run properly under a modern gentoo install as the userland makes system calls the kernel doesn't have.

I can use fbdev but then the GPU is just a chunk of RAM. The CPU plots every pixel.

More difficult than the kernel is the boot loader. If it works, just use it. It can be made to boot whatever you tell it.
Flashing the boot loader can be scary. You may need JTAG to recover.

If you don't know what JTAG is, don't risk it, unless its all on removable media that can be reprogrammed elsewhere.

[pingtoo]

jyoung,

Similar to Neddy suggestion, I would separate this objective in to smaller portion and tackle it one by one.

3 portions as in

1. rootfs (gentoo based)
2. bootloader(u-boot or what not)
3. kernel

I will not go much of the rootfs since it is already discussed in previous posts.

for the bootloader portion, you will need to access to console in order to make adjustment. so question, How do you access armbian? over ssh? can you check the kernel command line to see if kernel was programmed to output to serial console? finial I saw the wiki page you point to about using UART, it stated that possible incorrect label about pins, so have you try to change pins to see if that is case?

Only having above than tackle kernel will make things a lot easier.

[NeddySeagoon]

jyoung,

Like pingtoo says, if you are going to eat a whole elephant, do it one plateful at a time. :)

[jyoung]

So far I've been accessing armbian over ssh, although it does output to hdmi also.

I think I found the issue with the uart: serial/usb conversion wasn't enabled in the kernel on my desktop. Let me update that, and get back to you folks.

[jyoung]

Okay, I have the usb/serial driver installed, and I'm booting up the board with the armbian image. The wiki recommends to invoke picocom as so

Code: Select all

picocom -b 1500000 /dev/ttyUSB0

But, I don't any output. As you pointed out pingtoo, the wiki suggests switching the RK and TK pins. Still nothing with picocom. With the pins switched I did get an output from minicom with this command:

Code: Select all

minicom -b 115200 -D /dev/ttyUSB0

However, it just random code. Any recommendations?

[NeddySeagoon]

jyoung,

The baud rate, the 1500000 bit, needs to be the same both ends or you will get nothing or jibbereish, depending on you luck.
The parity and number oy stop bits needs to match too.

No parity and one stop bit is common.

Baby steps.

Connect your usb/serial doda to your PC but not to anything else. Keep at the wires separate.
Set up minicom for 1500000n1 and no handshake.
Turn off local echo, or you will be confused when thus works.

Type into minicom. you should see nothing.

Connect the Tx and Rx wires together. This is called a loopback test.

Type into minicom again.
This time you should see what you type echoed back to you. Your typing is going out of the Tx pin and back into the Rx pin, so you see it in minicom.

Until this works, there is no point in trying to see the serial console on your target board.

Next steps, the kernel serial part settings will be on the kernel command line on the target.
Have a look in dmesg.

Code: Select all

console=ttyS0,115200n8

Whatever it says, set up minicom to match,
Connect Ground, Tx and Rx to the target. If you Have a 4th wire keep it well out oy the way. It will be 5v.
Its quite safe to sort out Rx and Tx by trial and error.

[jyoung]

NeddySeagoon, from what I'm seeing I can turn off local echo within minicom with ctrl A+E. How do I set it up without a handshake?

[jyoung]

Okay, I tried these two tests, running minicom without and then with the RX and TX pins connected. As suggested, I invoked it with

Code: Select all

minicom -b 1500000n1 -D /dev/ttyUSB0

The first test was successful --- when I typed without the pins connected, there was no response. But there was still no response with the second test, when I connected the pins. Any ideas?

[NeddySeagoon]

jyoung,

Code: Select all

     +-------------------------------------------------------------------+
     |                      Minicom Command Summary                      |
OPTIO|                                                                   |
Compi|              Commands can be called by CTRL-A <key>               |
Port |                                                                   |
     |               Main Functions                  Other Functions     |
Press|                                                                   |
     | Dialing directory..D  run script (Go)....G | Clear Screen.......C |
     | Send files.........S  Receive files......R | cOnfigure Minicom..O |
     | comm Parameters....P  Add linefeed.......A | Suspend minicom....J |
     | Capture on/off.....L  Hangup.............H | eXit and reset.....X |
     | send break.........F  initialize Modem...M | Quit with no reset.Q |
     | Terminal settings..T  run Kermit.........K | Cursor key mode....I |
     | lineWrap on/off....W  local Echo on/off..E | Help screen........Z |
     | Paste file.........Y  Timestamp toggle...N | scroll Back........B |
     | Add Carriage Ret...U                                              |
     |                                                                   |
     |             Select function or press Enter for none.              |
     +-------------------------------------------------------------------+

Choose 'O' For Serial Port setup. Check

Code: Select all

    | E -    Bps/Par/Bits       : 115200 8N1                                |
    | F - Hardware Flow Control : No                                        |
    | G - Software Flow Control : No                                        |

For 'E' use whatever your target kernel sets the serial port to.

Minicom can save the settings

If you have any flow control at all, it may not work.
With Hardware Flow Control on, it certainly won't work as USB/Serial converters are usually missing the pins.
With software flow control on but not in use, it may or may not work.
Using software flow control will prevent binary transfers as Ctl-S and Ctl-R are reserved for frow control.

[jyoung]

Okay, when I open up the setup menu on minicom, I see this:

Code: Select all

Welcome to minicom 2.8

OPTI+-----------------------------------------------------------------------+
Comp| A -    Serial Device      : /dev/ttyUSB0                              |
Port| B - Lockfile Location     : /var/lock                                 |
    | C -   Callin Program      :                                           |
Pres| D -  Callout Program      :                                           |
    | E -    Bps/Par/Bits       : 115200 8N1                                |
    | F - Hardware Flow Control : Yes                                       |
    | G - Software Flow Control : No                                        |
    | H -     RS485 Enable      : No                                        |
    | I -   RS485 Rts On Send   : No                                        |
    | J -  RS485 Rts After Send : No                                        |
    | K -  RS485 Rx During Tx   : No                                        |
    | L -  RS485 Terminate Bus  : No                                        |
    | M - RS485 Delay Rts Before: 0                                         |
    | N - RS485 Delay Rts After : 0                                         |
    |                                                                       |
    |    Change which setting?                                              |
    +-----------------------------------------------------------------------+

That seems reasonable. The device /dev/ttyUSB0 is the right name for the serial/USB adapter, and Bps is the value from the last time I invoked minicom with -b. This documentation suggests that the target has a rate of 1500000

https://wiki.pine64.org/wiki/QuartzPro64_Development

I've installed minicom on the armbian image, which could let me check the rate, but I'm not sure what the name for the outgoing serial port is in /dev. Alas, unlike the USB adapter, I can't remove it and see which device disappears.

[jyoung]

The loopback test isn't working, but optimistically I just tried to connect to the quartz64 with

Code: Select all

minicom -b 1500000 -D /dev/ttyUBS0

It still came back as gibberish.

[NeddySeagoon]

jyoung,

Reading the UART section of that link,

UART
Plug in the USB-C port labelled "DEBUG PORT" on the QP64 board to another computer with a USB-A-to-C cable.
It will show up as a FT232 USB Serial adapter in lsusb

It appears that the "DEBUG PORT" is the USB side of a USB to serial port.
You only need

a USB-A-to-C cable

when you connect that, the USB adapter on board the quartz64-a will show up as /dev/ttyUSBx on your PC.
No separate USB/Serial adaptor is required.

Its all very well to say use 1500000 baud but check the kernel command line on the quartz64-a.

Code: Select all

less /proc/cmdline

That has to set up the serial port as a console.
Whatever that says, set minicom to match.

If that does not set console=tty... something, there won't be a serial port console.

Code: Select all

console=ttyS0,115200n8 console=ttyAMA0,115200n8

My serial port changes name during boot.

Lets take it on trust that the kernel uses the right serial port, so minicom needs to match 115200n8, or 1500000n8 ... or whatever.
With flow control off.

As you have a separate USB/serial convertor, make the loopback test work before you do anything else.

Your PC kernel will need

Code: Select all

$ zgrep FTDI_SIO /proc/config.gz
CONFIG_USB_SERIAL_FTDI_SIO=m
$ grep FTDI_SIO "/boot/config-$(uname -r)"
CONFIG_USB_SERIAL_FTDI_SIO=m

for the QP64 board USB to serial converter.

[jyoung]

Sorry, that was the wrong link. That is for the quartzpro64 board, which is slightly different. Here is the correct link

https://wiki.pine64.org/wiki/Quartz64

On the board, /proc/cmdline has

Code: Select all

root=UUID=45ec3db6-db76-48dc-8177-d8160f50fbd8 console=ttyS02,1500000 console=tty0 rw no_console_suspend consoleblank=0 fsck.fix=yes fsck.repair=yes net.ifnames=0 splash plymouth.ignore-serial-consoles

So, it looks like the device should be named ttyS02, but I'm not seeing that, only /dev/ttySx, where x=0-7.
I'm a little worried about plymouth.ignore-serial-consoles.


On the PC, I have

Code: Select all

CONFIG_USB_SERIAL_FTDI_SIO=y

in both locations, does it matter that it's not a module?

[NeddySeagoon]

jyoung,

Code: Select all

console=ttyS02,1500000

is how the kernel will set up /dev/ttyS02. That's a real serial port.
The n1 is the default.

Only it isn't. ttyS02 is not standard.

According to GPIO Pins (Quartz64 Model A) the debug UART on pins 6, 8 and 10.

Get your loopback test working, using minicom set to 1500000n1 to match the board but I suspect that when you move to the board the board will be silent as it won't have a /dev/ttyS02
That will need the kernel command line fixed.

The Quartz64 model-A schematic v2.0 shows the debug port as UART2
UART1 is for Bluetooth the others are free.

The USB to serial port driver on the PC has to match your USB to serial port driver as you will be connecting to the header on the board.

[pingtoo]

On the PC, I have

Code: Select all

CONFIG_USB_SERIAL_FTDI_SIO=y

in both locations, does it matter that it's not a module?

Are you sure your USB to TTL device use FTDI? according to the URL you provided,

Revision History

• Prior to July 1 2014 this cable would come with a PL2303HX chipset and a black USB body. After July 1, 2014 we changed to a violet body and the PL2303TA chipset. The new 'TA cables are Win8 compatible, and are more reliable especially when dealing with random USB-disconnects.
• As of Dec. 21, 2016 we will be shipping cables with SiLabs CP2012 chipset instead of Prolific. SiLabs chips seem to have better driver stability, and can handle very high speeds and 'unusual' non-standard baud rates. Other than the driver installation, the cables perform identically.

So you likely need kernel configure CONFIG_USB_SERIAL_CP210X=[y/m].

[NeddySeagoon]

jyoung,

We need to see lsusb showing the USB/Serial converter.