Zabbix server build trial and error

After running Zabbix in house for a few months, I thought it prudent to set up a second server as a backup in case the primary system failed, being on a Raspberry Pi 4 with an external SSD drive 

FreeBSD

At first I looked at pre-built packages; but found postgres unsupported. As I preferred PostgreSQL I didn't use the package manager

I built Zabbix 5.4 from source, over a few days. At that time, February 2022, FreeBSD had a port of 5.4 while NetBSD had 5.0. The build was easy enough, though would probably have taken longer if I only used an SD card.

This system has been working since February; I've added a few targets as well as bringing in outside data such as temperate at the local airport using zabbix_get with traps.

Original installation/configuration post: jspath55.blogspot.com/2022/02/building-out-intra-home-data-aggregator.html

Since I've experimented with multiple different OS installs, I didn't have another FreeBSD image running, so looked at alternatives.

Rock64

I got this arm64 board via mail order from China, and while I intend to put NetBSD on there, I have not located a usable image. Meanwhile, to check out the performance, I went with a Linux distro; first was old kernel (4.x), second was "almost" good enough (5.x). The HDMI is a little choppy, but at least the headphone jack worked.

 For Zabbix, I again wanted a Postgres database; got that installed.

rock64:/usr/share/zabbix-server-pgsql$ cat schema.sql  | sudo -u zabbix psql zabbix

CREATE TABLE

CREATE INDEX

CREATE TABLE

CREATE INDEX

CREATE INDEX

...

 Next, I needed Apache; got that (but config is non-classic).

The only PHP package available was 8.x

rock64:/usr/share$ sudo apt install libapache2-mod-php8.1

Reading package lists... Done

Building dependency tree... Done

Reading state information... Done

libapache2-mod-php8.1 is already the newest version (8.1.2-1ubuntu2).

And the frontend web backend  parts:

Congratulations! You have successfully installed Zabbix frontend.

Configuration file "/etc/zabbix/zabbix.conf.php" created.

 I got the install working up to initial screens; multiple fail messages ensued.

Jun 14 18:13:35 rock64 apachectl[289859]: [Tue Jun 14 18:13:35.313693 2022] [php:crit] [pid 289859:tid 281472868511776] Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe.  You need to recompile PHP.

The workaround for threadsafe is known, except that didn't matter in the end. I bagged this install as compiling php7 was problematic compared to using an available built package.

rock64:/etc/apache2/mods-enabled$ ls -l  /usr/lib/apache2/modules/libphp*

-rw-r--r-- 1 root root 5246072 Apr  7 13:46 /usr/lib/apache2/modules/libphp8.1.so

Pi 4 Open SUSE Linux

 I tried an install on an OpenSUSE system running on a Raspberry Pi 4. When I looked at the available packages, postgres database looked good. But the only Zabbix server package was 4.x (running 5.4 now, 6.x is available)

 I discarded this option as 4.x seemed a poor starting level, and I trusted pkgsrc given decades of familiarity, over yap or whatever the SUSE package manager is called.

  This was as far as I went:

$ sudo apt install  zabbix-server-postgresql

Retrieving: zabbix-server-4.0.39-1.5.aarch64.rpm ...[done]

Pi 4 running NetBSD

I had already started using this install with NetBSD-current from early 2022, with built packages. I  decided to build from /usr/pkgsrc if necessary, so did a CVS pull of pkgsrc from this year:

# cvs checkout -r pkgsrc-2022Q1 -P pkgsrc

The available Zabbix server package is still 5.0 but not 5.4, went ahead anyway. Hoping 5.4 (or later) is a work in progress.

The postgres database got built and configured. And I had to reinstall earlier packages that were postgres 11 but not the same patch level. :(

The program "postgres" was found by "/usr/pkg/bin/initdb" but was not the same version as initdb.

# pkg_delete postgresql11-server postgresql11-11.12nb1

The following files should be created for postgresql11-server-11.15:

Tried to get ahead of myself and run the server with an empty database; that didn't work but the error message was plain enough.

 29883:20220615:154343.702 [Z3005] query failed: [0] PGRES_FATAL_ERROR:ERROR:  relation "users" does not exist

The timezone error wasn't seen when I did the first install:

Time zone for PHP is not set (configuration parameter "date.timezone").

If you notice the times are different on the graphs included at the end of this post, you make notice both a skew in time (possible timezone) but also in the 12 hour versus 24 hour clock. Now that I have both I can decide which I prefer, or if having a discrepancy keeps the base differences more noticeable.

This parameter is important, particularly if you use the zabbix_send operative to load external data. If not matched to the right timezone, you end up with skewing.

See blogs.sap.com/2022/04/08/the-abap-detective-gets-their-clock-cleaned/ for more on the timezone for Zabbis database, sever, and data feeds.

It started working after I configured Apache and PHP.

[ server log snippet ]

  7166:20220615:155355.128 server #35 started [trapper #5]

  2705:20220615:155355.129 server #37 started [alert syncer #1]

  7015:20220615:155355.299 item "Zabbix server:zabbix[vmware,buffer,pused]" became not supported: No "vmware collector" processes started.

   111:20220615:155401.351 Zabbix agent item "system.cpu.num" on host "Zabbix server" failed: first network error, wait for 15 seconds

The VMware message makes sense, but the CPU message is one of those works on Linux, maybe on FreeBSD, but not on NetBSD. No matter, as enough CPU and system metrics are being collected for my purposes. The clips below are from the 5.0 and 5.4 systems, with the former including a more readable (to me) font color change between time marks. I suppose that is out-of-the-box behavior since I didn't tweak anything customized for charts.

I started to set up dual connect agents; realized "zabbix server" was a bad name choice, now. Each agent already is configure to believe the central system has that name, so I can't have 2 of them. I'll need to come up with a naming scheme and update all of the agents. I would have to do that anyway if I wanted to keep dual sources; this just makes it more challenging.