Thursday, December 27, 2018

Getting Booted Up with Python for OpenOversight et al


At the hackathon earlier this month [ see: Legal Hacking 2018 ] I found several "areas for improvement" in the code underlying the developing BPD Watch .  As I said I'd contribute, it made sense to set up a similar environment at home. After all, it's "just a database and some glue code" right?

https://lucyparsonslabs.com/

https://lucyparsonslabs.com/projects/openoversight/

https://openoversight.com/


> OpenOversight is released as free and open source software so others can launch similar police accountability projects in their own cities. The software is available for download on GitHub.



It sounded easy:


Developer Quickstart
Make sure you have Docker installed and then:

  • git clone https://github.com/lucyparsons/OpenOversight.git
  • cd OpenOversight
  • make dev
  • And open http://localhost:3000 in your favorite browser!



Only 4 steps. Oh, and a prerequisite. Which I didn't meet. Off to docker.org.

=

Note: If your system does not meet the requirements to run Docker for Windows, you can install Docker Toolbox, which uses Oracle Virtual Box instead of Hyper-V.

=

Ah, right, need to have Windows Professional

=

Requires Microsoft Windows 10 Professional or Enterprise 64-bit. For previous versions get Docker Toolbox.

=

So, fine, Docker Toolbox sounds good. Searching...


(default) Downloading C:\[]\.docker\machine\cache\boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v18.09.0/boot2docker.iso...
...
(default) Creating VirtualBox VM...
(default) Creating SSH key...
...
Checking connection to Docker...
Docker is up and running!
...
Hello from Docker!
This message shows that your installation appears to be working correctly.

=

Too easy, obviously.  Trying out the first step (cloning with git), 


==

$ git clone https://github.com/lucyparsons/OpenOversight.git
Cloning into 'OpenOversight'...

==

Erp.

= =

MINGW64 /c/mingw64/[]/OpenOversight (develop)
$ make dev

bash: make: command not found

= =

So, a running Linux flavor inside a Windows app, which doesn't have the bells and whistles of a full "tool chain" that needs must include make.  Having read comments on hooking the Docker environment up to a CygWin install (which I already had) I diverted into a short cul de sac with apt-get (: command not found) then tried a cross-compile using the make from cygwin.

=


/c/cygwin64/bin/make dev
docker-compose build
WARNING: The AWS_ACCESS_KEY_ID variable is not set. Defaulting to a blank string.
WARNING: The AWS_SECRET_ACCESS_KEY variable is not set. Defaulting to a blank string.
WARNING: The AWS_DEFAULT_REGION variable is not set. Defaulting to a blank string.
WARNING: The S3_BUCKET_NAME variable is not set. Defaulting to a blank string.
postgres uses an image, skipping
Building web

=

I was off to the races!  For a short time, at least.

=

docker-compose up -d
Pulling postgres

..
Postgres is up
## Creating database

= =

Sidebar - I had already looked at postgres for Cygwin, installed it on NetBSD, then decided the Windows version should be native, per some web hints. But I guess Docker doesn't know that database is running.

Next hurdle - building and populating the DB:

=

  File "../create_db.py", line 2, in
    from OpenOversight.app import create_app
ImportError: No module named OpenOversight.app

=

Here where it gets trickier.  Using make from cygwin to MingW wasn't complex, hooking in python turned out to be a mess.

I had already installed Python with CygWin, and a native Windows version (and, as I later found, another python distro is bundled with LibreOffice).

=

$ ./create_db.py
bash: ./create_db.py: /usr/bin/python^M: bad interpreter: No such file or directory

=

CRLF conflict, can be overcome with an outer shell instead of running directly:

$ /c/cygwin64/bin/python3.6 create_db.py


Down the overlapping install (due to the non-native python) rabbit hole.

$ pip3.6 install flask

ModuleNotFoundError: No module named 'flask_bootstrap'

ImportError: cannot import name 'config'

That last was due to Python 2/3 differences, I think, again "easily" fixed with a short edit

https://stackoverflow.com/questions/12172791/changes-in-import-statement-python3

Back to the drawing board, and the bootstrap tools for Python and friends.


=
 /cygdrive/c/mingw64/[]/OpenOversight
$ pip3.6 install -U pip setuptools


$ pip3.6 install python-dotenv
Collecting python-dotenv
=

More errors.

=
$ make dev
docker-compose build
make: docker-compose: Command not found
make: *** [Makefile:5: build] Error 127
...
ImportError: No module named OpenOversight.app
...
$ cp create_db.py test_data.py /usr/src/app
...

$ docker-compose run --rm web /c/cygwin64/bin/python3.6  ../create_db.py

ERROR:

        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

=

Well, no, I'm not sure I'm in the right directory. I got the database processes running, the build code, and everything else I can think of.

Versions of Python, at the moment:



CygWin

 $ python --version
Python 2.7.14

$ python3 --version
Python 3.6.4



Windows

C:\Users\Owner\AppData\Local\Programs\Python\Python37>.\python.exe --version
Python 3.7.1

c:\Program Files\LibreOffice\program> python --version
Python 3.5.5

wait, found one more:

/cygdrive/c/emacs/emacs-26.1/bin
$ ./python.exe --version
Python 2.7.14


 
MingW64

[]@ MINGW64 /c/Program Files/Docker Toolbox
$ python --version
Python 3.7.1


 
LibreOffice

APSO python console [LibreOffice 6.1]
3.5.5 (default, Oct 30 2018, 00:49:44) [MSC v.1900 64 bit (AMD64)]




My next attempt will be to pull the python build chain into the Docker shell terminal, which I found out how to do when I noted the LibreOffice Python drop was also bereft of any tools.

=

https://github.com/pypa/get-pip

!  wget https://bootstrap.pypa.io/get-pip.py

c:\[]\python\tmp>..\python.exe get-pip.py

=


Monday, December 24, 2018

Bengies Top Ten 2019 -- It Was A Very Wet Year

I did see Waterworld at the Bengies once.  This year was fiction come to life as the Baltimore are got a record amount of rain. Or enough above average to make a dent. I mean a splash.



The Top Ten, and then, the Explanations


  1. Bad Times At The El Royale
  2. Bohemian Rhapsody
  3. Remember The Night (1940)
  4. Home Alone (1990)
  5. Venom
  6. The Grinch
  7. House with a Clock in the Walls
  8. The Mask (1994)
  9. Incredibles (2)
  10. Jurassic World (2)

Also Rans:


  • Teen Titans Go!
  • Skyscraper
  • Crazy Rich Asians
  • The Greatest Showman
  • Mama Mia (2)



Wish I Had Seen:

  • Black Panther
  • First Man
  • Avengers 3
  • AntMan & The Wasp
  • Mission Impossible 6
  • Top Gun
  • Tomb Raider

Seven movies I wish I had seen at the Bengies is a lot this year. In some cases, those shows might have run only a week, or they played with something else that I wished to avoid, or we were away, or the weather was just disheartening.  I'm a loyal patron (hence the many tweets and fan photos) but can't always follow through.

The top movie this year, in my book, was Bad Times at the El Royale.  As the third feature, it could easily have been skipped once we reached the midnight hour and were saturated, but something told me to get caffeinated and stick with it. I was not disappointed.  The film noir sense, and the just on the edge of a slasher movie but always veering from macabre to bizarre just in time made this a top notch post moderne grindhouse flick.  Highly recommended, if you have the stomach, and the accelerants. I'll rent if from Netflix if only to pick up missed details from the flash forward/flashback weaving.

Second and third were close.  Bohemian Rhaposdy edged out mainly as it is in current release. The 1940 Fred MacMurray and Barbara Stanwyck holiday season co-feature was a rough cut diamond; the bad girl partially reformed by the good guy/tough (A)DA with a soft heart. Given the time period, the heroine had to be punished for her wicked ways, a reminder how times have changed.

Got quite a kick out of Home Alone on the really big screen, particularly the appearance of polka king John Candy.

/
View Tweet activity
/

At the Number Five spot is Venom.  With a name like that, I was thinking scary movie, like Snakes on a Plane, yet unfunny and unironic.  Wasn't thinking a SpiderMan type character, edgier and apocalyptic. So I was pleasantly surprised.  Don't need to see again, probably, but high up in the list for the unexpected jolts.

 Slots 6 through 10 are really just OK, and the order is immaterial.  I'm sure if I had been able to see any of the once I missed listed above that they'd be in the top 10. My list, my rules.

The Also Rans were ones I saw, and liked well enough to remember somewhat. Only Teen Titans might have made it in the top ten, as a replacement for one of the bottom 5.  The others had some wince factor, whether the over-the-top musicality of Showman or Mamma Mia, or the unfortunately scheduled Skyscraper (so late in the Scout weekend series I totally fell asleep during the whole flick). Crazy Rich Asians had fascinating scenery and sets, yet the plot was so gag-inducing I wish I had been able to fast forward to The Mask (but, no, reality is the show goes on in slow crawl real time).

Sprinkled below are my usual marquee shots, and a few related images thrown in, such as the Boy Scouts and Girl Scouts patches for their respective camp-ins, and the marquee-in-progress I like to stumble upon.

Links to prior year posts

2017: https://jspath55.blogspot.com/2017/12/2017-bengies-best-10-of-and-others-less.html
2016: https://jspath55.blogspot.com/2016/11/bengies-top-ten-movies-for-2016.html
2015: https://jspath55.blogspot.com/2015/12/bengies-top-ten-for-2015-in-my-eye.html
2014: https://jspath55.blogspot.com/2014/12/bengies-top-ten-movies-for-2014.html
2013: https://jspath55.blogspot.com/2013/12/bengies-top-ten-2013.html

Earlier year reviews back 2001 linked in one or more of these posts.



























Thursday, December 6, 2018

Legal Hacking 2018

Last night I attended a presentation at the University of Baltimore "Legal Hacker" event from Open Justice Baltimore concerning police oversight and open data. Stimulating ideas in the midst of a tempest of issues from corrupt public officials to lead poisoning to violence to information  accessibility.

I won't try to capture the entire presentation and discussion (lots of views, and we had to leave early) but focus on some data topics I will try to assist with.  The audience didn't need to be techies or lawyers; I'm the former and the meeting was in the UB Law School arena.  Short background.


Image above via Jason Tashea on Twitter (cropped for effect) https://twitter.com/jtashea/status/1070462755512217600.
General twitter accounts related to this effort are @LegalHackBalt, @OpenJusticeBalt, and @BOvrsight; more specific accounts are out there.
The online locations of groups, code repositories and data structures, more or less:


  • https://openjusticebaltimore.org/
  • https://github.com/openjusticebaltimore
  • https://github.com/lucyparsons/OpenOversight
  • https://disman.tl/oo-docs/tables/incident_license_plates.html
  • https://www.projectcomport.org
  • https://justicetech.info/tags/police-accountability/

To get my feet wet on the data architecture, given the case harvester project is not fully visible yet, I looked at the table documentation and drilled into the license plate data structure, where incident_license_plates looks like it might be pulled from a police report, and matching incident_id to license_plate_id.  The latter points to a license_plates table, with scant fields:

Column Type Size Nulls Auto Default
id serial 10 √ nextval('license_plates_id_seq'::regclass)
number varchar 8 null
state varchar 2 null


-

-

-

Several thoughts on this - (1) there could be more than one vehicle responding or involved in an incident, so it might be useful to know if one was the primary responder, backup, etc. (2) if an incident included vehicles but the license plate was not known (or illegible/redacted) what other identifiers might be helpful, and (3) while it seems as if 8 characters are adequate for typical DMV plates (number being a misnomer perhaps), there are special characters or images that might need to be recorded, as in vanity plates (not usual for police I suppose).

Another thread is whether a license plate might be only one indication of a specific vehicle, not to mention the records of who might be driving or a passenger. Typically, there are highly visible numbers placed on the vehicle roof or trunk to aid airborne units in directing traffic.  Here's an example culled from Google Street Views (image (c) google):


Google removes license plate details from all vehicles generally before making the images public, though these roof numbers seems to be intact (though not totally readable).

Record dates would also be important to know, since these expire, are transferred, etc.

Some areas to consider should the municipal fleet data be public (other than through crowdsourcing):


  • High speed pursuits are risky, and should be on the record. Witnesses to these might or might not note ID info. If cruisers had dash cams, do they also have GPS waypoints that might establish speeding trends?
  • Vehicles not in motion for long periods might indicate problematic behavior, as recently discussed in social media.
  • Red light camera records may show violations that need to be justified by pursuit, and thus on a public record.
  • Likewise for speed camera violations.
  • City asset tags/inventory numbers
  • Vehicle service records
More to follow as I get up to speed, virtually.

Sunday, December 31, 2017

2017, Bengies, Best 10 Of, And Others Less Fortunate


Bengies 2017 Top 10 Movies and More



This year there's as long a list of movies not watched at the Bengies than watched.  Previously (e.g. top ten movies for 2016 ) I posted ten movies in the Letterman style.  In 2017, I'm adding an addendum of those that didn't make the cut, either because I didn't watch the show on purpose, or conflicting events got in the way.  Whether the weather prevented my attendance on one of those lost weekends.

May


  • Boss Baby
  • Hidden Figures
  • Wonder Woman
 I didn't make the opening weekend this year, though by May there were several features of interest.  Of the 3 listed, Wonder Woman definitely hit the target, great special effects and action, good anti-Nazi catharsis, and women first.

Boss Baby could have been one note or even been tone deaf.  It was enjoyable if only by surpassing minimal expectations.  Probably would have not seen it without a strong second feature.

Hidden Figures contrasts with Wonder Woman with a similarly strong plot, made stronger by being historically accurate.  With Hollywood twists such as mild caricatures and compressed time lapses.


June 







OK, maybe we didn't see WW until June.

July


  •  Despicable Me 3
  •  Spider Man Homecoming

 Sequels can degrade the original experience, or just about break even.  Despicable Me is funny enough, depending on the Minions to sustain the humor.  The group singing an other-worldly version of a Modern Major General was worth the admission price.

Another Spider Day, another Spider Man.  I've lost track of the reboots, honestly.  This year's model was entertaining even with the deja vu.

August



September


Scout Camp-In


  • Cars 3
  • Pirates
  • Ferris Buellers Day Off

For the sixth year, local area Scouts (and a few from the DC area) crowded the Bengies on the first weekend after Labor Day, filling the field with tents, the snack bar line with new fans, and a crowd-pleasing line-up.  Can't please everyone of course, putting out a survey picks that can't all show, and tenting plans that mostly line up with the house rules and sight lines.



October


 Season Ending
  • Sandlot
  • Back to the Future
  • Executive Action

If you're keeping count, the above 3 take the season total to 11.  Which works for me even if the entire bill was pre-2017 productions.  The sum is closer to 10 since we left before the third feature ended.  I had not seen Sandlot, and so learned the origins of an oft-heard phrase (among Scouts) "you''re killing me. Smalls.").






December 


Holiday Special
 - Holiday Affair



Icing on the cake this year was an after the fact Double Feature. Almost missed this due to being out-of-town for the originally planned date, a squallish snowstorm pushed the Bengies management to reschedule for the week after.

Always like Robert Mitchum, and this lesser-known feature did not fail to entertain. For a rare exception to standard practice, the print was shown at an aspect ration differing from the original release.  Anything cut-off was not missed.

So, a Baker's Dozen of best for 2017.

Let's leave Chevy Chase for a future rant. Not in the top.




Previously:
2016 | 2015 | 2014 | 2013 | 2012 | 2011 | 2010 | 2009 - 2001

MISSED

The marquess below, and a few of the movies shown above that I saw part of the bill at least, are shows I did not see in 2017.  I think a few would have made the top 10, pushing one or more out of the top billing.  Others I'm good to pass.

Recent twitter thread ("Get Out":
/
https://twitter.com/krutika

Dec 29
pray 4 me my mom asked me what the best movie of the year was and i said get out so now....we're watching get out

\
 \- follow this thread.  Ir's hilarious.  Not maybe safe for work. R.

April


 Kong Skull Island
 Lego Batman

May

 

King Arthur
 Alien Covenant

ALL NIGHT SHOW

July

 Guardians of the Galaxy 2
 War for the Planet of the Apes


GG2, sure, would be great on the big screen.  Would have sat through an Apes movie, maybe.  Have seen a lifetime supply, starting with Technicolor, swearing, Charlton Heston.


 Baby Driver
 The Emoji Movie

Yeah, definitely would have missed anything with Emoji in the title if I wanted to see another show on the same bill.

Baby Driver had a lot of positive feedback. Darn. But Netflix DVD for the rerun.

September


ALL NIGHT SHOW

Now I can't remember a year where I didn't get to at least one of the all-night shows for 2 or more flicks, even staying through to the second playing of the National Anthem. But 2017 was an exception.

 Atomic Blonde

October


  Lego Ninjago

Two Lego movies in one year?  Yeah, peak 2017 is movies about children's building blocks.


 Blade Runner 2049

Of all the above, Blade Runer 2049 is the one I would have liked to catch.  Only played one weekend I believe, so poof and it's gone by.

Honorable Mention

On the return from total eclipse viewing, we caught a double feature at the Starlite Drive-In near Christiansburg, Virginia. The "kiddie" feature was Nut Job 2 ("nutty by nature") and the second feature the remake of The Beguiled.  Small crowd due to local sports events apparently, but a friendly atmosphere and decent snack bar food.


Thursday, March 23, 2017

Workmate renewed

This gem was given to me by my parents sometime in the late 70s I would say. 

It's a Black & Decker Workmate, fairly early model thought probably not the first generation.  It folds up neatly and has a metal frame that one can lean on for small or medium cuts.  Larger ones I'd want saw horses.

The top was pressed sawdust or something and while it lasted decades, wasn't as durable as the steel frame, threaded rods, and plastic handles (though one of those died at the very end of the rejuventation when I turned it wrongway up.

I bought slices of red oak and proceded to mostly duplicate the original top in dimensions and utility.

Happy to have it workable again, and thanks to Mom and Dad!











2
2 2


2 2
2