Thursday, December 27, 2018

Getting Booted Up with Python for OpenOversight et al


At the hackathon earlier this month [ see: Legal Hacking 2018 ] I found several "areas for improvement" in the code underlying the developing BPD Watch .  As I said I'd contribute, it made sense to set up a similar environment at home. After all, it's "just a database and some glue code" right?

https://lucyparsonslabs.com/

https://lucyparsonslabs.com/projects/openoversight/

https://openoversight.com/


> OpenOversight is released as free and open source software so others can launch similar police accountability projects in their own cities. The software is available for download on GitHub.



It sounded easy:


Developer Quickstart
Make sure you have Docker installed and then:

  • git clone https://github.com/lucyparsons/OpenOversight.git
  • cd OpenOversight
  • make dev
  • And open http://localhost:3000 in your favorite browser!



Only 4 steps. Oh, and a prerequisite. Which I didn't meet. Off to docker.org.

=

Note: If your system does not meet the requirements to run Docker for Windows, you can install Docker Toolbox, which uses Oracle Virtual Box instead of Hyper-V.

=

Ah, right, need to have Windows Professional

=

Requires Microsoft Windows 10 Professional or Enterprise 64-bit. For previous versions get Docker Toolbox.

=

So, fine, Docker Toolbox sounds good. Searching...


(default) Downloading C:\[]\.docker\machine\cache\boot2docker.iso from https://github.com/boot2docker/boot2docker/releases/download/v18.09.0/boot2docker.iso...
...
(default) Creating VirtualBox VM...
(default) Creating SSH key...
...
Checking connection to Docker...
Docker is up and running!
...
Hello from Docker!
This message shows that your installation appears to be working correctly.

=

Too easy, obviously.  Trying out the first step (cloning with git), 


==

$ git clone https://github.com/lucyparsons/OpenOversight.git
Cloning into 'OpenOversight'...

==

Erp.

= =

MINGW64 /c/mingw64/[]/OpenOversight (develop)
$ make dev

bash: make: command not found

= =

So, a running Linux flavor inside a Windows app, which doesn't have the bells and whistles of a full "tool chain" that needs must include make.  Having read comments on hooking the Docker environment up to a CygWin install (which I already had) I diverted into a short cul de sac with apt-get (: command not found) then tried a cross-compile using the make from cygwin.

=


/c/cygwin64/bin/make dev
docker-compose build
WARNING: The AWS_ACCESS_KEY_ID variable is not set. Defaulting to a blank string.
WARNING: The AWS_SECRET_ACCESS_KEY variable is not set. Defaulting to a blank string.
WARNING: The AWS_DEFAULT_REGION variable is not set. Defaulting to a blank string.
WARNING: The S3_BUCKET_NAME variable is not set. Defaulting to a blank string.
postgres uses an image, skipping
Building web

=

I was off to the races!  For a short time, at least.

=

docker-compose up -d
Pulling postgres

..
Postgres is up
## Creating database

= =

Sidebar - I had already looked at postgres for Cygwin, installed it on NetBSD, then decided the Windows version should be native, per some web hints. But I guess Docker doesn't know that database is running.

Next hurdle - building and populating the DB:

=

  File "../create_db.py", line 2, in
    from OpenOversight.app import create_app
ImportError: No module named OpenOversight.app

=

Here where it gets trickier.  Using make from cygwin to MingW wasn't complex, hooking in python turned out to be a mess.

I had already installed Python with CygWin, and a native Windows version (and, as I later found, another python distro is bundled with LibreOffice).

=

$ ./create_db.py
bash: ./create_db.py: /usr/bin/python^M: bad interpreter: No such file or directory

=

CRLF conflict, can be overcome with an outer shell instead of running directly:

$ /c/cygwin64/bin/python3.6 create_db.py


Down the overlapping install (due to the non-native python) rabbit hole.

$ pip3.6 install flask

ModuleNotFoundError: No module named 'flask_bootstrap'

ImportError: cannot import name 'config'

That last was due to Python 2/3 differences, I think, again "easily" fixed with a short edit

https://stackoverflow.com/questions/12172791/changes-in-import-statement-python3

Back to the drawing board, and the bootstrap tools for Python and friends.


=
 /cygdrive/c/mingw64/[]/OpenOversight
$ pip3.6 install -U pip setuptools


$ pip3.6 install python-dotenv
Collecting python-dotenv
=

More errors.

=
$ make dev
docker-compose build
make: docker-compose: Command not found
make: *** [Makefile:5: build] Error 127
...
ImportError: No module named OpenOversight.app
...
$ cp create_db.py test_data.py /usr/src/app
...

$ docker-compose run --rm web /c/cygwin64/bin/python3.6  ../create_db.py

ERROR:

        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

=

Well, no, I'm not sure I'm in the right directory. I got the database processes running, the build code, and everything else I can think of.

Versions of Python, at the moment:



CygWin

 $ python --version
Python 2.7.14

$ python3 --version
Python 3.6.4



Windows

C:\Users\Owner\AppData\Local\Programs\Python\Python37>.\python.exe --version
Python 3.7.1

c:\Program Files\LibreOffice\program> python --version
Python 3.5.5

wait, found one more:

/cygdrive/c/emacs/emacs-26.1/bin
$ ./python.exe --version
Python 2.7.14


 
MingW64

[]@ MINGW64 /c/Program Files/Docker Toolbox
$ python --version
Python 3.7.1


 
LibreOffice

APSO python console [LibreOffice 6.1]
3.5.5 (default, Oct 30 2018, 00:49:44) [MSC v.1900 64 bit (AMD64)]




My next attempt will be to pull the python build chain into the Docker shell terminal, which I found out how to do when I noted the LibreOffice Python drop was also bereft of any tools.

=

https://github.com/pypa/get-pip

!  wget https://bootstrap.pypa.io/get-pip.py

c:\[]\python\tmp>..\python.exe get-pip.py

=


No comments: