Showing posts with label GIS. Show all posts
Showing posts with label GIS. Show all posts

Monday, July 20, 2026

QGIS 3x3 on NetBSD 10/11 + PostgreSQL 14/16/18

 

Started making QGIS version 3.44.8 early June 2026 on NetBSD processors

jump ahead:

$ ps auxww | head -2

USER        PID %CPU %MEM      VSZ    RSS TTY    STAT STARTED     TIME COMMAND

root      24310 93.3  1.1   427668 355252 pts/6  O+   12:18AM  0:02.14 /usr/pkg/gcc12/libexec/gcc/x86_64--netbsd/12.4.0/cc1plus -quiet -I /usr/pkgsrc/wip/qgis/work/QGIS-final-4_0_3/cmake-pkgsrc-build/src/core/qgis_core_autogen/include

Wed Jun 10 00:25:37 UTC 2026

$ TZ=EST date

Tue Jun  9 19:25:45 EST 2026

=> Checking file-check results for qgis-3.44.8
=> Creating binary package /usr/pkgsrc/geography/qgis/work/.packages/qgis-3.44.8.tgz


/// pkg_delete postgresql18-client qgis-3.44.8  py313-psycopg2-2.9.11


$ pkgin  show-deps "qgis" | grep -i client
        postgresql16-client>=16.0nb1


https://pkgsrc.se/wip/qgis

Version history: (Expand)

    (2026-06-02) Updated to version: qgis-4.0.3
    (2026-05-10) Updated to version: qgis-4.0.2.18
    (2026-03-09) Updated to version: qgis-4.0.0.0
    (2026-02-22) Updated to version: qgis-3.99.3
    (2026-02-21) Updated to version: qgis-3.99.2

FREEBSD ~~~

foobar:/etc # pkg search qgis
qgis-3.44.11                   Free and Open Source Geographic Information System
qgis-ltr-3.40.15_1             Free and Open Source Geographic Information System

something bollixed. Back to NetBSD.
~~~

amd64-12-core
$ ls -ltr /usr/pkgsrc/packages/All/qg*
-rw-r--r--  1 root  wheel  123201876 Jun  9 17:15 /usr/pkgsrc/packages/All/qgis-3.44.8.tgz
-rw-r--r--  2 root  wheel  124614138 Jun 26 16:10 /usr/pkgsrc/packages/All/qgis-3.44.11.tgz
-rw-r--r--  2 root  wheel  127333035 Jul 13 11:34 /usr/pkgsrc/packages/All/qgis-4.0.3.tgz
-rw-r--r--  2 root  wheel  131910253 Jul 15 23:27 /usr/pkgsrc/packages/All/qgis-4.2.0.tgz

amd64-4-core
5.3$ ls -l /root/pkg-local/qg*
-rw-r--r--  1 root  wheel  127333035 Jul 13 20:02 /root/pkg-local/qgis-4.0.3.tgz

amd64-2-core
[     1.000004] cpu0 at mainbus0 apid 0
[     1.000004] cpu0: Use lfence to serialize rdtsc
[     1.000004] cpu0: Intel(R) Core(TM)2 Duo CPU     L9400  @ 1.86GHz, id 0x1067a
[     1.000004] cpu0: node 0, package 0, core 0, smt 0
[     1.000004] cpu1 at mainbus0 apid 1
[     1.000004] cpu1: Intel(R) Core(TM)2 Duo CPU     L9400  @ 1.86GHz, id 0x1067a
[     1.000004] cpu1: node 0, package 0, core 1, smt 0
[     1.000004] acpi0 at mainbus0: Intel ACPICA 20221020



2 core had faults, see:

RuntimeError: NumPy was built with baseline optimizations:
(X86_V2) but your machine doesn't support:
(X86_V2).

Pictures and exhibitions

Kept track of long compile times (~18 hours) on the slowest machine, as well as faster, looking at CPU temperature and process load (distributed across 1 or many "cores").

[
mastodon cc me
https://assets.chaos.social/media_attachments/files/116/846/959/384/561/931/original/96c70f8d89d6e49f.png

Go Speed Racer Go!


Pre 3-way build side quest:


Initial launch 4.0.3 June 2026


-rw-r--r--  1 jim  jim 20978 Jun 10 12:46 GIS/spots-rooms-q4.qgz



Side quest:

QGIS 3.44 on the machine I chose did not build, so the check with PG SQL 14 was inadequate. I changed the SQL version on that machine to 18, and it runs with the broken CPU message disabling various plugins (yet to be found).

This is the override of the database version when building from source, not installing pre-fabs.

$ pwd
/usr/pkgsrc/mk

$ diff pg*
28,30c28
< # PGSQL_VERSION_DEFAULT?=             16
< # Fri Jun 26 04:32:43 UTC 2026
< PGSQL_VERSION_DEFAULT?=               18
---
> PGSQL_VERSION_DEFAULT?=               16


Screengrab needs to be built from scratch; what I previously did with an X capture app has fallen into disrepair. At least xli still works.

$ ls -l `which qgis`
-rwxr-xr-x  1 root  wheel     390320 Jul  3 12:43 /usr/pkg/bin/qgis

-rw-r--r--  1 root  wheel  124615383 Jul  3 12:45 ./pkg-local/qgis-3.44.11.tgz

$ psql --version
psql (PostgreSQL) 18.4


Post 3-way build side quest of 3x3, going to 4:

Back to the fast machine and the newest (l+g). The work-in-progress version went from 4.0.3 to 4.2.0. Wow, quick change!

Build time ~8 hours with or without fumbles. Needs to be redone with a clean slate and only 2026Q4 pkgsrc to get things level. But it starts and connects to databases as it should.

$ time qgis --version
QGIS 4.2.0-Belém do Pará 'Belém do Pará' (exported)

real    0m2.522s
user    0m2.458s
sys     0m0.039s







Quick duck back to 3.44.11



Comparison table of the 12-core with 4.2.0 and the 2-core with 3.44.11.



Sidebar issue with Python caused the active plugins to be empty on the 2-core, but otherwise they are pretty much the same beyond QGIS itself and the Qt version going from 5 to 6.

Neither system shows a value for SFCGAL or GeographicLib. Not sure what impact that has.

Working on a list of projects for QGIS 4.2, including waypoints of Historic Trails, paths in the woods, and geocaching at the Bengies Drive-In Theatre.

Sunday, January 11, 2026

Note to Self, Mainly, on Shrinking Map Colors

 I am helping a theatre redo their website, from a post-Y2K relic to one that promises responsiveness (their phrase). They make a big deal about putting warning flags on images over 1MB, or a certain pixel size (like 800x1340). On the "directions and parking" section the old site had text directions, probably copied from Mapquest in 2002 or so. I kept the narrative and added static maps, since I didn't want to try a "live" google map image, figuring most folks would put the address in the own GPS/map app.

  The OSMAND app works great, so I figured I'd do screenshots and show the suggested routes, whether they lined up with the old text or not. They looked good, full screen PNG files with nice colors. Only they ended up way too big for a quick page load.

-rw-rw-rw-  1 user  users  1586661 Dec 14 22:19 map_from_westminster.png

When I saved them as JPEG, they didn't shrink much, just got a little fuzzier. Smaller size images didn't look great either. So I turned to the old standby library of graphics fixer-uppers, NetPBM [ netpbm.sourceforge.net/doc/ ]

The classic set of 216 colors (AKA "Web Safe") made sense as a target palette, as long as the altered image looked decent. I recall doing color reduction in the past and making some pictures that didn't. The docs said create a palette file, like so:

pamseq 3 5 >websafe.pam

Now, I had a 3 color band file, 5 values deep, just like Netscape likes. Applying that to a large image took almost no time:

pnmremap -map=websafe.pam map_from_westminster.pnm >websafe_west.ppm

Note this skips the PNG to PNM and PPM to PNG steps, which I added in a shell script that rotated through the 8 compass points of map/directions I wanted (N, NE, E, etc.)

#!/bin/sh

for img in \

 img1 \

 img2 \


do

  pngtopnm $img.png >$img.pnm

  pnmremap -map=websafe.pam $img.pnm | pnmtopng > ${img}_c216.png

done


Even on a slow drive, this is quick.

$ time pngtopnm map_from_westminster.png | pnmremap -map=websafe.pam - | pnmtopng >west_c26.png
pnmremap: 216 colors found in colormap

real    0m0.246s
user    0m0.260s
sys     0m0.017s

$ ls -l west_c26.png
-rw-r--r--  1 user  users  282243 Jan 12 00:01 west_c26.png


From 1.5 MB to 276 KB: good shrink!


$ perl $EXIFTOOL *west* | egrep "File Size|Image Size|Color Type"
File Size                       : 1549 KiB
Color Type                      : RGB
Image Size                      : 800x1340
File Size                       : 276 KiB
Color Type                      : Palette
Image Size                      : 800x1340

Afterward, I reapplied metadata such as Author and Copyright to the PNG file with the awesome exiftool.

Yet to do: add an overlay showing the OpenStreetMap sources used.




Tuesday, October 17, 2023

Saga of QGIS Tutorial--NetBSD stumble at the finish line

Previously*, I wrangled QGis version 3.28 or better on Raspberry Pi's trying to complete pertinent tutorial lessons and hitting local software application limits. I had success with OpenSUSE and FreeBSD (on x86 hardware). My next focus was NetBSD on a Pi4 (with the full 8GB), after upgrading the package system (in fits and starts) to the Q3 2023 release. 

https://jspath55.blogspot.com/2023/09/tiling-for-geo-map-referencing-in-qgis.html

https://jspath55.blogspot.com/2023/10/qgis-tutorial-torture-track.html

Just because it says BETA doesn't mean this is a flaky OS. The tiny box just takes everything I can throw at it; my main challenge is getting all the pieces talking nice (and doing a valid GEO PDF at the end).

$ uname -a
NetBSD nb 10.0_BETA NetBSD 10.0_BETA (GENERIC64) #0: Sat Sep  9 15:02:43 UTC 2023  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC64 evbarm

[     1.000000] NetBSD 10.0_BETA (GENERIC64) #0: Sat Sep  9 15:02:43 UTC 2023
[     1.000000]         mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC64
[     1.000000] total memory = 8024 MB
[     1.000000] avail memory = 7735 MB

[     1.000000] simplebus0 at armfdt0: Raspberry Pi Foundation Raspberry Pi 4 Model B

[     2.425653] bwfm0: Firmware file model-spec: brcmfmac43455-sdio.Raspberry Pi 4 Model B.bin

The PkgSrc has a current enough version (between Rasbian and Windows), and all the ducks included.

$ ldd /usr/pkg/bin/qgis
/usr/pkg/bin/qgis:
        -lqgis_app.3.28.7 => /usr/pkg/lib/libqgis_app.so.3.28.7
        -lqwt.6 => /usr/pkg/qwt-6.1.5/lib/libqwt.so.6
        -lexecinfo.0 => /usr/lib/libexecinfo.so.0
        -lelf.2 => /usr/lib/libelf.so.2
        -lc.12 => /usr/lib/libc.so.12
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1
        -lQt5PrintSupport.5 => /usr/pkg/qt5/lib/libQt5PrintSupport.so.5
        -lQt5Widgets.5 => /usr/pkg/qt5/lib/libQt5Widgets.so.5
        -lQt5Gui.5 => /usr/pkg/qt5/lib/libQt5Gui.so.5
        -lQt5Core.5 => /usr/pkg/qt5/lib/libQt5Core.so.5
        -lz.1 => /usr/lib/libz.so.1
        -ldouble-conversion.3 => /usr/pkg/lib/libdouble-conversion.so.3
        -lstdc++.9 => /usr/lib/libstdc++.so.9


Summing up:

$ qgis --version

QGIS 3.28.7-Firenze 'Firenze' (exported)

Steps for tutorial section 14 (part: "14.3.6. basic Follow Along: Joining the Forest Stand Data")

1. Clicking on the table on the right side lights up the areas on the left. Huzzah. Tiny sliver of badly-drawn border there as I could not (did not) locate the "fix overlaps" routine). Next lesson.

QGIS Attributes table - Pi 4 NetBSD 10

The attributes link is a huge step in correlating pictures to numbers, as it were.

2. Filter on the far left (funnel icon) where I selected by a routine just 2 subsets. Heads above manually clicking to select, and saved for future changes.

QGIS Filter - Pi4 NetBSD 10


3. A tree view of attributes rather than a grid view, 


QGIS attributes / Tutorial document

Zoom

The lessons worked, as far as an academic follow-the-book exercise went. I knew the parts I skipped or didn't complete, learning enough to see applicability to a few real-world projects.

The export to a georeferenced PDF is almost there but the NetBSD packages I installed omitted a GDAL subset.

GDAL PDF driver was not built with PDF read support. A build with PDF read support is required for GeoPDF creation.

Back to the package manual. And the mailing support lists with a reasonable question.

Next lesson: Elevations with topographic or other sources, or a dive into database storage/retrieval.



Monday, October 9, 2023

QGIS tutorial torture track

 I tried to follow the QGIS Forestry tutorial (link below) on a couple Raspberry Pi's (would have liked to use only one but meh). I got up to vector-to-polygon and then stalled on the first day hands-on. In 2 weeks there's a QGIS conference in town I'm attending; consider this my study hall.

Previously: jspath55.blogspot.com/2023/09/tiling-for-geo-map-referencing-in-qgis.html

Wherein I list the QGIS versions within reach. Odd to have this many flavors, so let's do the taste test, starting with the Pi with the newest version, running on OpenSUSE. 


The OS is missing something in the Perl layer which results in error messages and missing menus. Until I can track them down or they reappear with an unrelated app upgrade.

Tutorial link:

The learning here is use the filter (search) field to enter the coordinate reference system (the other CRS, not the can't remember stuff one) with numbers. I missed one of the transformations as the version I used had slightly different field text.

Geo-referencing was easy until I misread something, throwing off the mark. Easily seen by the result, and I skipped to the next lesson by keying in the exact values instead of trying the visual drop the claw-grab protocol.




Very nice to have the answers in the margins, so to speak.
The skewed first version:



Maxwell Smart voice: "missed it by that much."

Gnu Imp

Sigh. Crashed on tutorial section trying to select by color.

Bang, Zoom, to the Moon, or as my Brit friends might say, "Cloud-cuckoo-land."


12:33:55



12:34:01

Fortunately, only the user session/window manager died, not the whole OS as it might look.

Switched over to another PI running Rasbian (but QGIS 3.10) that had Gnu Imp which worked, to a fault.

Gnu Imp - Lower Left of Map



More results in the green goo.



Not the ideal output the tutorial has, but the steps are workable. Later attempts to move to polygonal spaces failed as the Raspbian was missing parts, or I cannot find them yet, and the OpenSUSE needs the python hooks re-hooked to shape up the shape files.

SQL FUN

Something was in the wrong mode.


ERROR 1: sqlite3_exec(DROP TRIGGER rtree_states_provinces_geom_update3) failed: attempt to write a readonly database

ERROR 1: sqlite3_exec(CREATE TRIGGER "rtree_disputed_borders_geom_update3" AFTER UPDATE ON "disputed_borders" WHEN OLD."fid" != NEW."fid" AND (NEW."geom" NOTNULL AND NOT ST_IsEmpty(NEW."geom")) BEGIN DELETE FROM "rtree_disputed_borders_geom" WHERE id = OLD."fid"; INSERT OR REPLACE INTO "rtree_disputed_borders_geom" VALUES (NEW."fid",ST_MinX(NEW."geom"), ST_MaxX(NEW."geom"),ST_MinY(NEW."geom"), ST_MaxY(NEW."geom")); END) failed: trigger "rtree_disputed_borders_geom_update3" already exists


FILE TIFFing


tutorial-pi400/rautjarvi_green_georef.tif: TIFF image data, little-endian, direntries=29, height=4561, bps=362, compression=none, PhotometricIntepretation=RGB, name=U:\Dropbox\GISMOOD_KESKUS_prj2014\0-13004-002-Simosol-online_gis_materials\1-mets\303\244materiaalit\0-mets\303\244materiaalit\ex1-map digi, description=Groovy Green Goo, orientation=upper-left, width=2014



C.R.S.


"Multiple operations are possible for converting coordinates between these two Coordinate Reference Systems. Please select the appropriate conversion operation, given the desired area of use, origins of your data, and any other constraints which may alter the "fit for purpose" for particular transformation operations."


"An alternative, ballpark-only transform was used when transforming coordinates between EPSG:2392 - KKJ / Finland zone 2 and EPSG:3067 - ETRS89 / TM35FIN(E,N). The results may not match those obtained by using the preferred operation:
Possibly an incorrect choice of operation was made for transformations between these reference systems. Check the Project Properties and ensure that the selected transform operations are applicable over the whole extent of the current project."