NetBSD 11 RC3 heat, RC4, and more

The last post was about NetBSD 11, Release Candidate 3. Meanwhile, RC4 was just announced, so this write-up is mainly about testing RC3, with a little RC4 overlap as I upgraded several machines this week.

CPU metrics during User Testing

CPU temperature pattern, where I scheduled the automated test framework to run twice per day on a Raspberry Pi 3 (aarch64).

Now, running processes; the test cycles kick off a couple at a time. The spikes line up with heat generation, unsurprisingly.

Another Zabbix chart for a 48 hour period showing 1 minute average per core. Spikes above 0.5 are rare, and not always aligned in a test "burst", presumably due to the Zabbix data collection cycle where short duration events might be missed.

Interrupts per second. Not much above normal background load other than one obvious spike per test run. Might be interesting to drill down to the specific timeframe and see what tests were running.

Context switches per second, with occasional spikes. Some correspond to other metrics, some not. Closest match is interrupts.

 Test run failures

As with prior ATF runs, the errors differ from architecture to architecture. The one-core Raspberry Pi0W had the most faults, while the 4-core Pi3 had the least.

grep "failed test cases" tests-*txt | grep -v expected |  awk '{print $2}' | sort -n | uniq -c
   1 4
   4 5
   8 6
   5 7
  13 8
  11 9
   5 10
   3 11
   1 12
   1 13

Above is for an AMD64, which being the fastest of the systems tested, has more runs in a day/week. I did scatter plots to show the variations.

AMD64:

i386:

Pi0W:

Pi3:

These charts are not the standard distribution bell curve, but close, where the number of errors must be zero or a positive number. The variation in errors from one run to the next means the next analytical step is measuring frequencies of specific test failures

Doesn't look like any one test fails every time for this machine:

net/npf/t_npf:npf_guid, usr.sbin/tcpdump/t_tcpdump:promiscuous, fs/tmpfs/t_vnode_leak:main, modules/t_x86_pte:svs_g_bit_set, crypto/libcrypto/t_libcrypto:threads

net/carp/t_basic:carp_handover_ipv6_halt_nocarpdevip, net/npf/t_npf:npf_guid, fs/tmpfs/t_vnode_leak:main, fs/vfs/t_renamerace:puffs_renamerace_cycle, modules/t_x86_pte:svs_g_bit_set, crypto/libcrypto/t_libcrypto:threads

net/net/t_unix:sockaddr_un_fstat, net/carp/t_basic:carp_handover_ipv6_halt_nocarpdevip, net/ndp/t_ndp:ndp_cache_expiration, net/npf/t_npf:npf_guid, fs/tmpfs/t_vnode_leak:main, crypto/libcrypto/t_libcrypto:threads

net/carp/t_basic:carp_handover_ipv6_halt_nocarpdevip, net/carp/t_basic:carp_handover_ipv6_ifdown_nocarpdevip, net/npf/t_npf:npf_guid, fs/tmpfs/t_vnode_leak:main, fs/vfs/t_renamerace:msdosfs_renamerace, fs/vfs/t_renamerace:puffs_renamerace_cycle, modules/t_x86_pte:svs_g_bit_set, crypto/libcrypto/t_libcrypto:threads

net/carp/t_basic:carp_handover_ipv6_ifdown_nocarpdevip, net/ndp/t_ndp:ndp_cache_expiration, net/npf/t_npf:npf_guid, fs/tmpfs/t_vnode_leak:main, fs/vfs/t_renamerace:puffs_renamerace_cycle, crypto/libcrypto/t_libcrypto:threads

$ sort  /tmp/ers | uniq -c
   5 crypto/libcrypto/t_libcrypto:threads
   5 fs/tmpfs/t_vnode_leak:main
   1 fs/vfs/t_renamerace:msdosfs_renamerace
   3 fs/vfs/t_renamerace:puffs_renamerace_cycle
   3 modules/t_x86_pte:svs_g_bit_set
   3 net/carp/t_basic:carp_handover_ipv6_halt_nocarpdevip
   2 net/carp/t_basic:carp_handover_ipv6_ifdown_nocarpdevip
   2 net/ndp/t_ndp:ndp_cache_expiration
   1 net/net/t_unix:sockaddr_un_fstat
   5 net/npf/t_npf:npf_guid
   1 usr.sbin/tcpdump/t_tcpdump:promiscuous