Had at first one Raspberry Pi, now at four

 I started with one Raspberry Pi, now at four and counting. After I had the first one in 2019, a Pi 4 with 4GB, I ran only the vendor encouraged Raspian OS. After getting the second Pi 4 in 2020 (hey--8GB!) and reading docs on both NetBSD and FreeBSD I picked the latter as having made more progress in the install phase, if not more user friendliness.

[Previously:

https://jspath55.blogspot.com/2019/08/48-hours-with-raspberry-pi-4-4gb.html

https://jspath55.blogspot.com/2019/08/raspberry-pi4-quick-cooling-test.html

]

Pi4 running FreeBSD

I had used the first Pi for LibreOffice, VLC audio, web browsing, a bit of python, and at least 2 database platforms. The postgreSQL was my primary interest after not being as familiar as the gamut from Oracle to SQLServer to mysql with a smattering of DB2, Progress, and the OpenOffice-bundled HyperSQL. But I've been running NetBSD for decades, starting before that with 386BSD, and occasional FreeBSD, SUSE, and RedHat open source deployments.

As I moved from one OS to the next, backup and restore along with deploy from scratch techniques often changed incrementally. Honestly, the first 386BSD and NetBSD installations I brought up via floppy disk transfers. I've toasted more than one SSD drive, and struggled with boot sectors on some hand-me-down machines as I like to see how to bring a cast-off back into service. The last 2 major machines I used for NetBSD were an appliance-size 386 class, which has gone down “soft” due to fan noises that predict a future “hard” down. The newer platform is an AMD 64 bit NetBSD, last at 9.1; meanwhile I set up a NAS to keep backups and share a few file sets.

Back to the Pi story. FreeBSD 13, which was in "current" status when I first got it installed, started up on the Pi 4. It can access hardware primitives such as CPU clock frequency, and core temperature. Over the past year or so since I had the 2 pi 4s, FreeBSD has run with a growing set of apps, and is now at the 13-Release level. Each time I reinstalled the OS and deployed various packages, keeping note of the process (as Linux, NetBSD and FreeBSD use different deploy steps), and resolving conflicts whenever possible. 

One such conflict involves mysql and LibreOffice (which I use instead of OpenOffice for docs, sheets, and database access panels). BSD would not handle both in the package matrix, but could deal with the alternate mariadb.

# pkg_delete mysql-client
Package `mysql-client-5.7.34' is still required by other packages:
        libreoffice-7.1.2.2nb6

[4/11] Installing mysql57-client-5.7.32...

pkg: mariadb105-client-10.5.8 conflicts with mysql57-client-5.7.32 (installs files into the same place).  Problematic file: /usr/local/bin/mysql

Installed packages to be REMOVED:

        libreoffice6: 6.4.7_2

        mysql57-client: 5.7.32

Reading up on NetBSD, I decided perhaps the Pi 3 series would work best (in early 2021) so I got 2 of them, hoping to have one wired and one wireless, perhaps relegating FreeBSD from Pi 4 to the Pi 3 level if I could get NetBSD to run.

Pi3 running NetBSD wireless

I'll skip the dead end installs other than to acknowledge how complex the stack is, and how an alternate OS could even boot on the machine much less get to the point of running Firefox and even xscreensaver. Meanwhile as I was burning microSD cards and watching boot light shows, NetBSD 9.2 came out. My successful installs of NetBSD were from the -current code base, and as it turns out, a few tweaks there aren't yet in the 9.x path. Will have to wait for 10 it seems.

What tasks did I put these machines to?

A primary use is database servers, clients, and SQL data crunchers. I put household projects on one, and nutrition views in another. Even recipes, though that doesn't use a mainstream database. Secondary use is music track storage and display, after converting decades of CD collections to either MP3 or Itunes file. Spreadsheets with LibreOffice is a change of pace from MS Office or the borg-ish Google calc.

FreeBSD

One limit that's documented is the lack of "native" audio output. To me, that means I would have an audio server (minidlna) run on that pi but not use it for playback. I've learned DLNA players may be found in Roku sticks, BluRay/DVD players, and even on my Android phone and tablet. The phone has foobar2000 while the tablet can handle the perennial favorite VLC.

LibreOffice installs easily (other than the mysql conflict I found) and behaves well enough but as speedily as I'd like. Response time is good but startup, save, and shutdown not so good. Admittedly this is partly due to using an SD card only other than, so far, only NAS with slow but large spinning disks.

Hardware wise, I only played around a bit with connecting external disks through USB until I fried one SSD drive somehow. NFS makes that less of a local plug-in problem. I added a fan after looking at the temperature charts on the pi 4. When I tested FreeBSD on the pi 3, I saw a little heat relief with the case lid off, but as both of the pi 3 machines are now running NetBSD I left off getting more fans for the older and hopefully cooler boards,

I don't think there are drivers for the built-in Pi wifi yet, either 3 or 4. As I've gotten wireless to work on the pi3 with NetBSD, I didn't go further yet testing USB wifi on FreeBSD. I did get a couple USB sound "cards", one of which, happily, has a chip set that FreeBSD recognizes and I think it was the more expensive of the 2 ($25 instead of $10).

X11

FreeBSD X Windows works just like earlier versions of X11, other than I have not been able to alter any fonts on xterm. In the NetBSD X11 connections, they work fine, even adding a new size in the menus (“Enormous”). One has Unicode, the other VT, probably because of the windows manager (/usr/X11R7/bin/ctwm versus twm)

Audio

I have 2 primary music source: an MP3/M4A library, and streaming source URLs (or XMLs). These depend on the audio hardware working, which on Linux is assumed but on BSD not so much. As I have audio output working on both BSD flavors, one control of interest is the route, whether HDMI or the 1/8 inch (3.5mm) jack. For NetBSD, I did this:

mixerctl -w outputs.select=headphones

On FreeBSD with an external USB audio dongle:

$ cat /dev/sndstat

Installed devices:

pcm0: <USB audio> (play/rec) default

No devices installed from userspace.

"Jazz hands":

> cvlc kcsm.xspf 

VLC media player 3.0.11.1 Vetinari (revision 3.0.11.1-0-g52483f3ca2)

[0000f03ec6b84380] dummy interface: using the dummy interface module...

[0000f03ec5d430c0] mpeg4audio demux packetizer: AAC channels: 1 samplerate: 22050

Packages

Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:13:aarch64/quarterly, please wait…

# pkg install mariadb105-server

# pkg install scribus

# pkg install xscreensaver

https://www.davidschlachter.com/misc/freebsd-usb-audio

https://wiki.freebsd.org/Sound

NetBSD

My main stumbling block has been getting the right install image for the machines I have. In one case (9.2), I burned the compressed OS image onto the microSD card, using the Raspberry Pi install software on Windows. It happily recognizes .gz install files as viable, even if the code won't boot up that way. I only speculate this based on a few failures of other NetBSD images, including ebijun’s and at least one -current. I even managed to get one as far as noting that X was not there, making a valiant effort to build it from pkgsrc, and giving up after too many swap failures.

Pi3 wired running NetBD

Swap is apparently pretty important for doing any complex compute work on the Pi 3. (1GB main memory rather than the 4 or 8GB on the Pi 4). The good news is with an external drive, even though USB-connected, greater things are possible For example, running minidlna for the first time pointing at an audio library takes some time (reading the files, associating album artwork if found, and writing a usable sqlite3 database). Without swap defined, in 9.2, the library build only made it to the "B"s.

[ 244273.226534] UVM: pid 16739 (minidlnad), uid 0 killed: out of swap
Jun  2 18:22:56 nib /netbsd: [ 558967.3390542] UVM: pid 12874 (dymaxionmap), uid 1000 killed: out of swap

After I started this post I decided I would try to add a swap device or file, then consulted https://www.netbsd.org/docs/misc/index.html#swap and decided to reuse a 2GB microSD card and a small USB adapter. I formatted it on the PC as FAT32, then added a disklabel, changing the type from MSDOS to swap, and then adding an entry in /etc/fstab. If that chip fails or gets pulled out I'm not sure the system will come up, but will deal with it then.

> swapctl -l

no swap devices configured

<fiddling occurs> 

> swapctl -l
Device      1K-blocks     Used    Avail Capacity  Priority
/dev/sd0e     1931196    35064  1896132     2%    0

For the -current image, I was pleasantly surprised to find that audio output works (but it doesn't directly in 9.2). I put the cheaper USB sound chip in the Pi with 9.2, and a USB wifi chip in the -current board.

# pkg_add mysql-server

# pkg_add mariadb-server

# pkg_add rrdtool

# pkg_add xli

> pkg_add raspberrypi-userland

pkg_add: no pkg found for 'raspberrypi-userland', sorry.

pkg_add: 1 package addition failed

uaudio0: <Antlion Audio Antlion USB adapter, class 0/0, rev 1.10/1.00, addr 2> on usbus0

After adding an SSD disk with a SATA/USB adapter to the first NetBSD system I got running, I put the pkg and pkgsrc directories there, then started adding other directories that would benefit from the faster I/O (and larger headroom). A home directory was easy enough, so I also created db, log, run, and tmp folders. To get builds to use the tmp directory there, rather than the one on SD card, I set TMPDIR to /usr/pkgsrc/tmp/. And put into a dot profile for persistence.

I took a chance to re-host the SSD drive with an "unpowered" USB adapter. For spinning disks, there is unlikely to be sufficient power without a separate feed besides the Pi, but that adds hardware and wires. The SSD has been fine so far, though I had a few mysterious halts prior to setting up swap on the external drive.

wifi - ifconfig

urtwn0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500

        ssid ""

        powersave off

        address: 98:48:xx:xx:xx:xx

        media: IEEE802.11 autoselect (DS1)

        status: no network

See: wla_supplicant in the manuals.

[     6.972019] ugen0: Realtek (0x2357) 802.11ac NIC (0x0138), rev 2.10/2.10, addr 5

[     6.992442] urtwn0: Realtek (0x0bda) 802.11n NIC (0x8179), rev 2.00/0.00, addr 5

In order to have the Pi recognized correctly on wireless after connecting by wire, I ended up modifying the DHCP configuration. In the past, I switched from dhcpcd to dhclient, partly because I was more familiar, but to some degree, it's easier to type correctly.

In the dhcpcd.conf man page, the stanza per interface lets you pass DHCP a host name:

< # Wed May 26 00:59:54 UTC 2021

< interface meu0

< hostname arm64

< #

< interface urtwn0

< hostname arm64a

< #

< interface bwfm0

< hostname arm64b

< #

The "meu0" interface is wired, the "urtwn0" is wireless on a USB plug-in, and the "bwfm0" is the built-in Broadcom device. In testing, that last connection seemed to work partway yet not completely sane. 

top

Each OS has a top command, and while pretty similar, switching to view all 4 CPUs has different methods, or just didn't work for me.

By the way, I've only been able to build/compile/run "catclock" on NetBSD. It's a wily one.

Stock Linux Pi4

dmesg

While each of the running systems has a dmesg command, the contents, order, and usefulness are distinct. Like baby ducks, you imprint the first one you come across, and variations or rewrites seem odd. I've plucked out basic data such as CPU type, memory, and network circuitry, whether wired or wireless. As noted above, the standard Pi OS is wireless, as is one of the NetBSD boards. I've left off many details around USB, video, and disk specifics, though I wish "boot from USB/SSD disk" was easier to pull off.

All emphases mine.

NetBSD: (pi 3; wireless)

[     1.000000] NetBSD 9.99.82 (GENERIC64) #0: Tue Apr 27 05:40:29 UTC 2021

[     1.000000] total memory = 930 MB

[     1.000000] avail memory = 897 MB

[     1.000000] simplebus0 at armfdt0: Raspberry Pi 3 Model B Plus Rev 1.3

[     1.000000] cpu0 at cpus0: Arm Cortex-A53 r0p4 (v8-A), id 0x0

[     6.978679] urtwn0 at uhub1 port 2

[     6.988681] urtwn0: Realtek (0x0bda) 802.11n NIC (0x8179), rev 2.00/0.00, addr 5

[     8.028819] mue0 at uhub2 port 1

[     8.038821] mue0: vendor 0424 (0x0424) product 7800 (0x7800), rev 2.10/3.00, addr 6

[     8.318858] mue0: LAN7800 id 0x7800 rev 0x2

[     8.328860] ukphy0 at mue0 phy 1: OUI 0x00800f, model 0x0013, rev. 2

[     8.338861] ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto

NetBSD: (pi 3; wired)

[     1.000000] NetBSD 9.2 (GENERIC64) #0: Wed May 12 13:15:55 UTC 2021
[     1.000000] total memory = 933 MB
[     1.000000] avail memory = 900 MB
[     1.000000] simplebus0 at armfdt0: Raspberry Pi 3 Model B Plus Rev 1.3
[     1.000000] cpu0 at cpus0: Cortex-A53 r0p4 (Cortex V8-A core)
[    11.479420] mue0: vendor 0424 (0x424) product 7800 (0x7800), rev 2.10/3.00, addr 8
[    11.769459] mue0: LAN7800 id 0x7800 rev 0x2
[    11.779461] ukphy0 at mue0 phy 1: OUI 0x00800f, model 0x0013, rev. 2
[    11.779461] ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto

FreeBSD (pi4; wired)

FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr  9 06:06:55 UTC 2021

real memory  = 4147916800 (3955 MB)

avail memory = 4023050240 (3836 MB)

CPU  0: ARM Cortex-A72 r0p3 affinity:  0

genet0: <RPi4 Gigabit Ethernet> mem 0x7d580000-0x7d58ffff irq 82,83 on simplebus2

genet0: GENET version 5.0 phy 0x0000

brgphy0: <BCM54213PE 1000BASE-T media interface> PHY 1 on miibus0

brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto

Linux (Raspios?)

[    0.000000] CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), cr=30c5383d

[    0.000000] OF: fdt: Machine model: Raspberry Pi 4 Model B Rev 1.4

[    0.000000] Memory: 7800096K/8245248K available (10240K kernel code, 1354K rwdata, 3152K rodata, 2048K init, 890K bss, 183008K reserved, 262144K cma-reserved, 7458816K highmem)

[    1.256495] bcmgenet fd580000.ethernet: GENET 5.0 EPHY: 0x0000

?

[    5.756276] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.txt failed with error -2

[    9.098683] bcmgenet fd580000.ethernet: configuring instance for external RGMII (RX delay)

[    9.099110] bcmgenet fd580000.ethernet eth0: Link is Down

[   15.358288] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready

XScreensaver

This doesn't need words; needs to be seen to be believed.

Temperature

After getting the first Pi, I found an online tutorial on measuring (or recording more properly) the Pi CPU temperature. 

https://github.com/gzachos/rpi-cputemp

The Pi 4 can run hot, and it became quickly clear that more load ran up the heat, which then triggered a CPU slowdown from on-board health controls. I experimented with a couple fans and ending up cooling both Pi 4s. The Pi 3 seemed to be less of a toaster, so I didn't seek the heat values initially. Well, I could not find the correct command as it turns out, so once I did I proceeded to set up cron jobs and local web pages to make history. Yeah, the Pi 3 is also a hot machine, and I'll likely search again for cooling methods. In one experiment, I pushed the clock speed on the NetBSD machines and then discovered overheating as shown on the on-screen "thermometer". Which doesn't help if you don't look at that console, but run remote commands via ssh.

Temperature readings:

Linux

TEMP_READING=$(/opt/vc/bin/vcgencmd measure_temp)

FreeBSD

TEMP_READING=$(/sbin/sysctl dev.cpu.0.temperature)

NetBSD

TEMP_READING=$(/usr/sbin/envstat|/usr/bin/tail -1|/usr/bin/awk '{print $2}') 

The NetBSD command is more complex in 9.2 than in -current, as the latter has an option missing in the former:

$ envstat

                 Current  CritMax  WarnMax  WarnMin  CritMin  Unit

[vcmbox0]

  temperature:    69.832   85.000                             degC

$ envstat -n

72.522

Yes, 72.5 Celsius...

# sysctl -w machdep.cpu.frequency.target=1200

machdep.cpu.frequency.target: 600 -> 1200

Extra hardware

Older and cheaper devices are sometimes better since they more likely may have available open source drivers. It's a gamble. Before tax: $30. Not shown, the $15 USB to SATA adapter.

SKU	Description	Quantity	Price Per	Total Price
874834	TPLINK 150MBPS WRLSN NNO USB ADP	1	9.99	9.99
854281	ANTLION ANTLION USB SOUND ADAPTER	1	9.99	9.99
550509	SABRENT USB SOUND ADAPTER CMEDIA	1	9.99	9.99

Example links (sometimes you can start further up the tree, sometimes not)

pkg_add http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/aarch64/9.0_2020Q4/All/vlc-3.0.11.1.tgz
pkg_add ftp://ftp.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/9.0/All/firefox-82.0.2.tgz
pkg_add -v http://cdn.netbsd.org/pub/NetBSD/misc/jun/aarch64/aarch64/2019-06-22/python37

References:

• https://www.embeddedarm.com/blog/netbsd-toaster-powered-by-the-ts-7200-arm9-sbc/
• https://www.instructables.com/IronForge-the-NetBSD-Toaster/

FreeBSD on Pi

• https://docs.freebsd.org/en/books/handbook/x11/
• https://paulgorman.org/technical/freebsd-raspberry-pi-2.txt.html
• https://download.freebsd.org/ftp/releases/arm64/aarch64/ISO-IMAGES/13.0/FreeBSD-13.0-RC4-arm64-aarch64-RPI.img.xz

NetBSD on Pi 

(apologies for http instead of https on some links)

• https://nycdn.netbsd.org/pub/NetBSD-daily/HEAD/202104270600Z/evbarm-aarch64/binary/gzimg/arm64.img.gz
• https://mail-index.netbsd.org/port-arm/2018/04/28/msg004773.html
• http://mail-index.netbsd.org/port-arm/2019/07/12/msg005918.html
• http://mail-index.netbsd.org/port-arm/2020/06/09/msg006740.html
• https://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc/misc/raspberrypi-userland/README.html
• http://cdn.netbsd.org/pub/NetBSD/misc/jun/aarch64/aarch64/2021-1/
• http://nycdn.netbsd.org/pub/NetBSD-daily/HEAD/202105182000Z/evbarm-aarch64/binary/gzimg/arm64.img.gz
• http://cdn.netbsd.org/pub/NetBSD/misc/jun/raspberry-pi/2021-05-14-aarch64/2021-05-14-netbsd-raspi-aarch64.img.gz
• http://cdn.netbsd.org/pub/NetBSD/misc/jun/aarch64/aarch64/2019-06-22/
• https://github.com/ebijun/NetBSD/tree/master/RPI/RPIimage/Image/aarch64

TO DO:

Besides watching the core temperature with an eye towards better heat dispersal, I'd planning on:

• Review log file growth, and control with "logrotate"
• Figure out how to delay starting mindlna until NFS comes online, particularly after a power outage.
• Wait for NetBSD 10
• Correlate xscreenxaver hacks with the resultant heat rejected
• Run benchmarks at different core frequencies
• Test USB-3 hubs, as even 4 ports fill up