After I put a couple Raspberry Pi environment monitoring devices into my shopping cart and pile of unconnected devices, I looked around for a central data aggregator that would be a step up from individual cron jobs and RRD repositories. The open source platform Zabbix looked interesting, as it was noted in online posts about pulling data from remote devices. Around the end of January, as the weather pushed me to inside projects, I started figuring out the moving parts.
My first thought was I'd like to run the database and system on NetBSD, but after looking at the state of Zabbix server and agent version availability, I found FreeBSD has an advantage in having better coverage. I thought I could put the server processes on one node and the database itself on a different node, since I already had working PostgreSQL systems. Trying to activate NetBSD packages on the arm Pi systems ended up with a battle between mysql and mariadb, meaning the package wouldn't use postgreSQL as I wanted.
The package version of Zabbix for FreeBSD, is also pre-configured for MySQL, despite the configuration file indicating that a few tweaks could alter the database connection. I tried several permutations of ports and other values before coming to the realization that if I wanted a different database I'd need to build the application from sources. But running both a binary package repository and one from source on one node can be problematic, leading me to start from scratch with a fresh FreeBSD system.
After a few feints, I was able to get FreeBSD 13 set up on an SSD drive connected through USB to a Pi 4. Turns out, that was the easy part. I tried to find the least common denominator of underlying libraries to minimize the time spent watching auto-configure and make run through huge software stacks. I was expecting there might be several days worth of churn ahead but was pleasantly surprised as I watched the component parts get laid down.
Somewhere around Groundhog Day, I had the Zabbix server working, along with at least one other agent, and proceeded to deploy the front end. Fortunately, I had a working httpd server on another FreeBSD on Pi, so adding the php code was a minor hassle.
In a prior life, I worked with enterprise scale software/hardware/application monitoring software, from PMC Patrol/Enterprise Manager, to HP OpenView, to Computer Associates tools, and finally, to the now-tainted SolarWinds. Much of the design of Zabbix, as well as the agent technologies, looked pretty familiar. After the web front end was running, it looked very clean and modern, with menus and paths that looked straightforward.
43331:20220204:013743.375 cannot send list of active checks to "x.x.x.x": host [Name] not found
(04 February 2022)
Alas, not everything was as simple as it appeared. The learning curve was not too steep, though it involved translating a few terms into understandable chunks. Like "not supported" as a state for a monitoring element. On the surface, that would mean to me that the combination was just not going to work, as opposed to a condition where an element went offline or was unreachable for some reason. There were more subtleties under the surface as I'd learn by trial and error.
The first hurdle was the nomenclature of server and agent, where you could put a label on something that matched a DNS entry, or didn't match. I knew that once I started making configuration choices (like short name or long name), I'd probably be stuck with that decision once the beast took on a life of its own.
85706:20220205:173436.157 resuming Zabbix agent checks on host "sample": connection restored
(05 February 2022)
(07 February 2022)
$ cat /sys/class/thermal/thermal_zone0/temp
$ vcgencmd measure_temp
(08 February 2022)