Getting Booted Up with Python for OpenOversight et al

At the hackathon earlier this month [ see: Legal Hacking 2018 ] I found several "areas for improvement" in the code underlying the developing BPD Watch .  As I said I'd contribute, it made sense to set up a similar environment at home. After all, it's "just a database and some glue code" right?

https://lucyparsonslabs.com/

https://lucyparsonslabs.com/projects/openoversight/

https://openoversight.com/

> OpenOversight is released as free and open source software so others can launch similar police accountability projects in their own cities. The software is available for download on GitHub.

https://github.com/lucyparsons/OpenOversight/

https://github.com/lucyparsons/OpenOversight/issues

It sounded easy:

---

Developer Quickstart

Make sure you have Docker installed and then:

• git clone https://github.com/lucyparsons/OpenOversight.git
• cd OpenOversight
• make dev
• And open http://localhost:3000 in your favorite browser!

---

Only 4 steps. Oh, and a prerequisite. Which I didn't meet. Off to docker.org.

=

Note: If your system does not meet the requirements to run Docker for Windows, you can install Docker Toolbox, which uses Oracle Virtual Box instead of Hyper-V.

=

Ah, right, need to have Windows Professional

=

Requires Microsoft Windows 10 Professional or Enterprise 64-bit. For previous versions get Docker Toolbox.

=

So, fine, Docker Toolbox sounds good. Searching...

= 

(default) Downloading C:\[]\.docker\machine\cache\boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v18.09.0/boot2docker.iso...

...

(default) Creating VirtualBox VM...

(default) Creating SSH key...

...

Checking connection to Docker...

Docker is up and running!

...

Hello from Docker!

This message shows that your installation appears to be working correctly.

=

Too easy, obviously.  Trying out the first step (cloning with git), 

==

$ git clone https://github.com/lucyparsons/OpenOversight.git
Cloning into 'OpenOversight'...

==

Erp.

= =

MINGW64 /c/mingw64/[]/OpenOversight (develop)
$ make dev

bash: make: command not found

= =

So, a running Linux flavor inside a Windows app, which doesn't have the bells and whistles of a full "tool chain" that needs must include make.  Having read comments on hooking the Docker environment up to a CygWin install (which I already had) I diverted into a short cul de sac with apt-get (: command not found) then tried a cross-compile using the make from cygwin.

=


/c/cygwin64/bin/make dev
docker-compose build
WARNING: The AWS_ACCESS_KEY_ID variable is not set. Defaulting to a blank string.
WARNING: The AWS_SECRET_ACCESS_KEY variable is not set. Defaulting to a blank string.
WARNING: The AWS_DEFAULT_REGION variable is not set. Defaulting to a blank string.
WARNING: The S3_BUCKET_NAME variable is not set. Defaulting to a blank string.
postgres uses an image, skipping
Building web

=

I was off to the races!  For a short time, at least.

=

docker-compose up -d
Pulling postgres

..

Postgres is up

## Creating database

= =

Sidebar - I had already looked at postgres for Cygwin, installed it on NetBSD, then decided the Windows version should be native, per some web hints. But I guess Docker doesn't know that database is running.

Next hurdle - building and populating the DB:

=

  File "../create_db.py", line 2, in

    from OpenOversight.app import create_app

ImportError: No module named OpenOversight.app

=

Here where it gets trickier.  Using make from cygwin to MingW wasn't complex, hooking in python turned out to be a mess.

I had already installed Python with CygWin, and a native Windows version (and, as I later found, another python distro is bundled with LibreOffice).

=

$ ./create_db.py
bash: ./create_db.py: /usr/bin/python^M: bad interpreter: No such file or directory

=

CRLF conflict, can be overcome with an outer shell instead of running directly:

$ /c/cygwin64/bin/python3.6 create_db.py

Down the overlapping install (due to the non-native python) rabbit hole.

$ pip3.6 install flask

ModuleNotFoundError: No module named 'flask_bootstrap'

ImportError: cannot import name 'config'

That last was due to Python 2/3 differences, I think, again "easily" fixed with a short edit

https://stackoverflow.com/questions/12172791/changes-in-import-statement-python3

Back to the drawing board, and the bootstrap tools for Python and friends.


=
 /cygdrive/c/mingw64/[]/OpenOversight
$ pip3.6 install -U pip setuptools


$ pip3.6 install python-dotenv
Collecting python-dotenv

=

More errors.

=
$ make dev
docker-compose build
make: docker-compose: Command not found
make: *** [Makefile:5: build] Error 127

...

ImportError: No module named OpenOversight.app

...

$ cp create_db.py test_data.py /usr/src/app

...

$ docker-compose run --rm web /c/cygwin64/bin/python3.6  ../create_db.py

ERROR:

        Can't find a suitable configuration file in this directory or any

        parent. Are you in the right directory?

=

Well, no, I'm not sure I'm in the right directory. I got the database processes running, the build code, and everything else I can think of.

Versions of Python, at the moment:

---

CygWin

 $ python --version
Python 2.7.14

$ python3 --version
Python 3.6.4

---

Windows

C:\Users\Owner\AppData\Local\Programs\Python\Python37>.\python.exe --version
Python 3.7.1

c:\Program Files\LibreOffice\program> python --version
Python 3.5.5

wait, found one more:

/cygdrive/c/emacs/emacs-26.1/bin
$ ./python.exe --version
Python 2.7.14

---

 
MingW64

[]@ MINGW64 /c/Program Files/Docker Toolbox
$ python --version
Python 3.7.1

---

 
LibreOffice

APSO python console [LibreOffice 6.1]
3.5.5 (default, Oct 30 2018, 00:49:44) [MSC v.1900 64 bit (AMD64)]

---

My next attempt will be to pull the python build chain into the Docker shell terminal, which I found out how to do when I noted the LibreOffice Python drop was also bereft of any tools.

=

https://github.com/pypa/get-pip

!  wget https://bootstrap.pypa.io/get-pip.py

c:\[]\python\tmp>..\python.exe get-pip.py

=

Bengies Top Ten 2018 -- It Was A Very Wet Year (not 2019 yet)

I did see Waterworld at the Bengies once.  This year was fiction come to life as the Baltimore are got a record amount of rain. Or enough above average to make a dent. I mean a splash.

The Top Ten, and then, the Explanations

1. Bad Times At The El Royale
2. Bohemian Rhapsody
3. Remember The Night (1940)
4. Home Alone (1990)
5. Venom
6. The Grinch
7. House with a Clock in the Walls
8. The Mask (1994)
9. Incredibles (2)
10. Jurassic World (2)

Also Rans:

• Teen Titans Go!
• Skyscraper
• Crazy Rich Asians
• The Greatest Showman
• Mama Mia (2)

Wish I Had Seen:

• Black Panther
• First Man
• Avengers 3
• AntMan & The Wasp
• Mission Impossible 6
• Top Gun
• Tomb Raider

Seven movies I wish I had seen at the Bengies is a lot this year. In some cases, those shows might have run only a week, or they played with something else that I wished to avoid, or we were away, or the weather was just disheartening.  I'm a loyal patron (hence the many tweets and fan photos) but can't always follow through.

The top movie this year, in my book, was Bad Times at the El Royale.  As the third feature, it could easily have been skipped once we reached the midnight hour and were saturated, but something told me to get caffeinated and stick with it. I was not disappointed.  The film noir sense, and the just on the edge of a slasher movie but always veering from macabre to bizarre just in time made this a top notch post moderne grindhouse flick.  Highly recommended, if you have the stomach, and the accelerants. I'll rent if from Netflix if only to pick up missed details from the flash forward/flashback weaving.

Second and third were close.  Bohemian Rhaposdy edged out mainly as it is in current release. The 1940 Fred MacMurray and Barbara Stanwyck holiday season co-feature was a rough cut diamond; the bad girl partially reformed by the good guy/tough (A)DA with a soft heart. Given the time period, the heroine had to be punished for her wicked ways, a reminder how times have changed.

Got quite a kick out of Home Alone on the really big screen, particularly the appearance of polka king John Candy.

/

Jim Spath‏ @jspath55 Dec 8

Forgot that Chuck Berry's Run Run Rudolph was the theme music to leaving Kevin home alone.
1 reply0 retweets0 likes

Jim Spath‏ @jspath55

John Candy as the polka king. Miss him.

6:34 PM - 8 Dec 2018

0 replies0 retweets0 likes

View Tweet activity
/

At the Number Five spot is Venom.  With a name like that, I was thinking scary movie, like Snakes on a Plane, yet unfunny and unironic.  Wasn't thinking a SpiderMan type character, edgier and apocalyptic. So I was pleasantly surprised.  Don't need to see again, probably, but high up in the list for the unexpected jolts.

 Slots 6 through 10 are really just OK, and the order is immaterial.  I'm sure if I had been able to see any of the once I missed listed above that they'd be in the top 10. My list, my rules.

The Also Rans were ones I saw, and liked well enough to remember somewhat. Only Teen Titans might have made it in the top ten, as a replacement for one of the bottom 5.  The others had some wince factor, whether the over-the-top musicality of Showman or Mamma Mia, or the unfortunately scheduled Skyscraper (so late in the Scout weekend series I totally fell asleep during the whole flick). Crazy Rich Asians had fascinating scenery and sets, yet the plot was so gag-inducing I wish I had been able to fast forward to The Mask (but, no, reality is the show goes on in slow crawl real time).

Sprinkled below are my usual marquee shots, and a few related images thrown in, such as the Boy Scouts and Girl Scouts patches for their respective camp-ins, and the marquee-in-progress I like to stumble upon.

Links to prior year posts

2017: https://jspath55.blogspot.com/2017/12/2017-bengies-best-10-of-and-others-less.html
2016: https://jspath55.blogspot.com/2016/11/bengies-top-ten-movies-for-2016.html
2015: https://jspath55.blogspot.com/2015/12/bengies-top-ten-for-2015-in-my-eye.html
2014: https://jspath55.blogspot.com/2014/12/bengies-top-ten-movies-for-2014.html
2013: https://jspath55.blogspot.com/2013/12/bengies-top-ten-2013.html

Earlier year reviews back 2001 linked in one or more of these posts.

Legal Hacking 2018

Last night I attended a presentation at the University of Baltimore "Legal Hacker" event from Open Justice Baltimore concerning police oversight and open data. Stimulating ideas in the midst of a tempest of issues from corrupt public officials to lead poisoning to violence to information  accessibility.

I won't try to capture the entire presentation and discussion (lots of views, and we had to leave early) but focus on some data topics I will try to assist with.  The audience didn't need to be techies or lawyers; I'm the former and the meeting was in the UB Law School arena.  Short background.

Image above via Jason Tashea on Twitter (cropped for effect) https://twitter.com/jtashea/status/1070462755512217600.
General twitter accounts related to this effort are @LegalHackBalt, @OpenJusticeBalt, and @BOvrsight; more specific accounts are out there.
The online locations of groups, code repositories and data structures, more or less:

• https://openjusticebaltimore.org/
• https://github.com/openjusticebaltimore
• https://github.com/lucyparsons/OpenOversight
• https://disman.tl/oo-docs/tables/incident_license_plates.html
• https://www.projectcomport.org
• https://justicetech.info/tags/police-accountability/

To get my feet wet on the data architecture, given the case harvester project is not fully visible yet, I looked at the table documentation and drilled into the license plate data structure, where incident_license_plates looks like it might be pulled from a police report, and matching incident_id to license_plate_id.  The latter points to a license_plates table, with scant fields:

Column Type Size Nulls Auto Default

id serial 10 √ nextval('license_plates_id_seq'::regclass)

number varchar 8 null

state varchar 2 √ null

-

-

-

Several thoughts on this - (1) there could be more than one vehicle responding or involved in an incident, so it might be useful to know if one was the primary responder, backup, etc. (2) if an incident included vehicles but the license plate was not known (or illegible/redacted) what other identifiers might be helpful, and (3) while it seems as if 8 characters are adequate for typical DMV plates (number being a misnomer perhaps), there are special characters or images that might need to be recorded, as in vanity plates (not usual for police I suppose).

Another thread is whether a license plate might be only one indication of a specific vehicle, not to mention the records of who might be driving or a passenger. Typically, there are highly visible numbers placed on the vehicle roof or trunk to aid airborne units in directing traffic.  Here's an example culled from Google Street Views (image (c) google):

Google removes license plate details from all vehicles generally before making the images public, though these roof numbers seems to be intact (though not totally readable).

Record dates would also be important to know, since these expire, are transferred, etc.

Some areas to consider should the municipal fleet data be public (other than through crowdsourcing):

• High speed pursuits are risky, and should be on the record. Witnesses to these might or might not note ID info. If cruisers had dash cams, do they also have GPS waypoints that might establish speeding trends?
• Vehicles not in motion for long periods might indicate problematic behavior, as recently discussed in social media.
• Red light camera records may show violations that need to be justified by pursuit, and thus on a public record.
• Likewise for speed camera violations.
• City asset tags/inventory numbers
• Vehicle service records

More to follow as I get up to speed, virtually.