Fixing Jussi Build

Since we moved away from Steem, it became essential that the tools needed migration to Hive. One of those tools is Jussi, a reverse proxy that only speaks json-rpc 2.0, which acts as a bridge between the users and the backend servers (steemd, hivemind). First thing in the migration was to ensure that the tools can be built. Unfortunately, as we speak, as far as I'm concerned, we're still using an old (functional) docker image of jussi built by Steemit 6 months ago (steemit/jussi:latest). I tried to build it but it failed. I created an issue but nobody had addressed it yet https://gitlab.syncad.com/hive/jussi/issues/1. So I decided to look into it myself, although I'm not a Python programmer at all. Armed with patience, resolve and lots of googling, I started digging into the problem.

Preliminary trials and errors

To begin, it's pretty straight forward:
git clone https://gitlab.syncad.com/hive/jussi
cd jussi
docker build --tag jussi:test .

Step 24/28 : RUN pipenv install --dev
 ---> Running in 0f0bed24815f
Creating a virtualenv for this project…
Pipfile: /app/Pipfile
Using /usr/local/bin/python3.6m (3.6.5) to create virtualenv…
⠼ Creating virtual environment...created virtual environment CPython3.6.5.final.0-64 in 253ms
  creator CPython3Posix(dest=/app/.venv, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /app/.venv
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
⠴ Locking..✘ Locking Failed! 
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/lib/python3.6/site-packages/pipenv/resolver.py", line 69, in resolve
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
[pipenv.exceptions.ResolutionFailure]:       Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
There are incompatible versions in the resolved dependencies.
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/usr/local/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
[pipenv.exceptions.ResolutionFailure]:       Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0
Tried: 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.2.4, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.3.5, 2.4.0, 2.4.1, 2.4.2, 2.5.0, 2.5.1, 2.5.2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.7.0, 2.7.1, 2.7.1, 2.7.2, 2.7.2, 2.7.3, 2.7.3, 2.8.0, 2.8.0, 2.8.1, 2.8.1, 2.8.2, 2.8.2, 2.8.3, 2.8.3, 2.8.4, 2.8.4, 2.8.5, 2.8.5, 2.8.6, 2.8.6, 2.8.7, 2.8.7, 2.9.0, 2.9.0, 2.9.1, 2.9.1, 2.9.2, 2.9.2, 3.0.0, 3.0.0, 3.0.1, 3.0.1, 3.0.2, 3.0.2, 3.0.3, 3.0.3, 3.0.4, 3.0.4, 3.0.5, 3.0.5, 3.0.6, 3.0.6, 3.0.7, 3.0.7, 3.1.0, 3.1.0, 3.1.1, 3.1.1, 3.1.2, 3.1.2, 3.1.3, 3.1.3, 3.2.0, 3.2.0, 3.2.1, 3.2.1, 3.2.2, 3.2.2, 3.2.3, 3.2.3, 3.2.4, 3.2.4, 3.2.5, 3.2.5, 3.3.0, 3.3.0, 3.3.1, 3.3.1, 3.3.2, 3.3.2, 3.4.0, 3.4.0, 3.4.1, 3.4.1, 3.4.2, 3.4.2, 3.5.0, 3.5.0, 3.5.1, 3.5.1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.6.2, 3.6.2, 3.6.3, 3.6.3, 3.6.4, 3.6.4, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.7.2, 3.7.2, 3.7.3, 3.7.3, 3.7.4, 3.7.4, 3.8.0, 3.8.0, 3.8.1, 3.8.1, 3.8.2, 3.8.2, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.9.3, 3.9.3, 3.10.0, 3.10.0, 3.10.1, 3.10.1, 4.0.0, 4.0.0, 4.0.1, 4.0.1, 4.0.2, 4.0.2, 4.1.0, 4.1.0, 4.1.1, 4.1.1, 4.2.0, 4.2.0, 4.2.1, 4.2.1, 4.3.0, 4.3.0, 4.3.1, 4.3.1, 4.4.0, 4.4.0, 4.4.1, 4.4.1, 4.4.2, 4.4.2, 4.5.0, 4.5.0, 4.6.0, 4.6.0, 4.6.1, 4.6.1, 4.6.2, 4.6.2, 4.6.3, 4.6.3, 4.6.4, 4.6.4, 4.6.5, 4.6.5, 4.6.6, 4.6.6, 4.6.7, 4.6.7, 4.6.8, 4.6.8, 4.6.9, 4.6.9, 5.0.0, 5.0.0, 5.0.1, 5.0.1, 5.1.0, 5.1.0, 5.1.1, 5.1.1, 5.1.2, 5.1.2, 5.1.3, 5.1.3, 5.2.0, 5.2.0, 5.2.1, 5.2.1, 5.2.2, 5.2.2, 5.2.3, 5.2.3, 5.2.4, 5.2.4, 5.3.0, 5.3.0, 5.3.1, 5.3.1, 5.3.2, 5.3.2, 5.3.3, 5.3.3, 5.3.4, 5.3.4, 5.3.5, 5.3.5, 5.4.0, 5.4.0, 5.4.1, 5.4.1
There are incompatible versions in the resolved dependencies.
The command '/bin/sh -c pipenv install --dev' returned a non-zero code: 1

As you can see, it was having trouble locking the dependencies versions. I spent hours trying to find a solution for that but nothing worked.

Next, I switched my focus to this pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches pytest==3.10.0,>=2.7,>=2.9,>=3.0.0,>=3.0.6,>=3.6.0,>=5.0. It was weird because pytest 3.10.0 still exists on the python repository, so it's not obsolete or discontinued.

So I looked into the Pipfile, where pytest = "==3.10.0". So I set pytest = ">3.10.0", which would install a higher version of it.

Fire up the docker build again; this time, progress:

Step 24/28 : RUN pipenv install --dev
 ---> Running in 455f3152c983
Creating a virtualenv for this project…
Pipfile: /app/Pipfile
Using /usr/local/bin/python3.6m (3.6.5) to create virtualenv…
⠼ Creating virtual environment...created virtual environment CPython3.6.5.final.0-64 in 277ms
  creator CPython3Posix(dest=/app/.venv, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /app/.venv
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
⠴ Locking..✔ Success! 
Locking [packages] dependencies…
⠼ Locking..✔ Success! 
Updated Pipfile.lock (606260)!
Installing dependencies from Pipfile.lock (606260)…
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

The docker build was resuming normally, until it ran into another error, where pipenv run pytest was failing:

Step 26/28 : RUN pipenv run pytest
 ---> Running in 592b4803f184
Traceback (most recent call last):
  File "/app/.venv/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 105, in main
    config = _prepareconfig(args, plugins)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 258, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/app/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/helpconfig.py", line 90, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 836, in pytest_cmdline_parse
    self.parse(args)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 1044, in parse
    self._preparse(args, addopts=addopts)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 992, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 299, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/app/.venv/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 95, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "/app/.venv/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 152, in exec_module
    exec(co, module.__dict__)
  File "/app/.venv/lib/python3.6/site-packages/pytest_asyncio/plugin.py", line 8, in <module>
    from _pytest.python import transfer_markers
ImportError: cannot import name 'transfer_markers'
The command '/bin/sh -c pipenv run pytest' returned a non-zero code: 1

Looks like we have a nasty ImportError: cannot import name 'transfer_markers' related to pytest_asyncio. Back to Pipfile, the version installed was pytest-asyncio = "==0.9.0".

According to https://stackoverflow.com/questions/54064971/importerror-cannot-import-name-transfer-markers-when-testing-with-pytest

pytest-asyncio is using transfer_markers which is taken out in pytest 4.1.0. Looks like it's fixed and released. Issue#105
EDIT: You can now upgrade pip install pytest-asyncio==0.10.0 to fix this issue.
Or update pytest-asyncio==0.10.0 in your requirements.txt

This gave me a clue that we're in pytest 4 (vs 3.10.0 where we started). Alright, let's try pytest-asyncio = ">0.9.0" in Pipfile. And.... another error:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 495, in _importconftest
    return self._conftestpath2mod[key]
KeyError: PosixPath('/app/tests/conftest.py')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.venv/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 105, in main
    config = _prepareconfig(args, plugins)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 258, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/app/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/helpconfig.py", line 90, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 836, in pytest_cmdline_parse
    self.parse(args)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 1044, in parse
    self._preparse(args, addopts=addopts)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 1002, in _preparse
    early_config=self, args=args, parser=self._parser
  File "/app/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/app/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/app/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 899, in pytest_load_initial_conftests
    self.pluginmanager._set_initial_conftests(early_config.known_args_namespace)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 444, in _set_initial_conftests
    self._try_load_conftest(current)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 452, in _try_load_conftest
    self._getconftestmodules(x)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 473, in _getconftestmodules
    mod = self._importconftest(conftestpath)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/config/__init__.py", line 501, in _importconftest
    mod = conftestpath.pyimport()
  File "/app/.venv/lib/python3.6/site-packages/py/_path/local.py", line 701, in pyimport
    __import__(modname)
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "/app/.venv/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 152, in exec_module
    exec(co, module.__dict__)
  File "/app/tests/conftest.py", line 1870, in <module>
    @pytest.fixture(params=translatable_steemd_requests_and_responses())
  File "/app/.venv/lib/python3.6/site-packages/_pytest/fixtures.py", line 993, in result
    fail(message, pytrace=False)
  File "/app/.venv/lib/python3.6/site-packages/_pytest/outcomes.py", line 158, in fail
    raise Failed(msg=msg, pytrace=pytrace)
Failed: Fixture "translatable_steemd_requests_and_responses" called directly. Fixtures are not meant to be called directly,
but are created automatically when test functions request them as parameters.
See https://docs.pytest.org/en/latest/fixture.html for more information about fixtures, and
https://docs.pytest.org/en/latest/deprecations.html#calling-fixtures-directly about how to update your code.
The command '/bin/sh -c pipenv run pytest' returned a non-zero code: 1

Failed: Fixture "translatable_steemd_requests_and_responses" called directly. Fixtures are not meant to be called directly, Ok so now I need to learn Python and try to figure wth is happening in the code itself. Fuck no, already wasted too much time on this, and I'm not going to learn python. I need a break. Screw this chain of unhappy packages.

Let's try something else

How about checking which packages/versions are being installed? There's a nifty command pipenv graph that outputs all installed packages with their version, requirements and dependencies.
Since pytest = ">3.10.0" initially solved the Pipfile locking problem, I retried it with this added in Dockerfile
RUN pipenv install --dev && pipenv graph. The graph command gave a very long output (I won't paste it) that was extremely helpful! By looking at which packages were using pytest, I found:

pytest-asyncio==0.9.0
  - pytest [required: >=3.0.6, installed: 5.4.1]
...
pytest-console-scripts==0.1.7
  - mock [required: >=2.0.0, installed: 4.0.2]
  - pytest [required: >=3.0.0, installed: 5.4.1]
...
pytest-cov==2.6.0
  - coverage [required: >=4.4, installed: 5.0.4]
  - pytest [required: >=2.9, installed: 5.4.1]
...
pytest-mock==1.10.0
  - pytest [required: >=2.7, installed: 5.4.1]
...
pytest-profiling==1.7.0
  - gprof2dot [required: Any, installed: 2019.11.30]
  - pytest [required: Any, installed: 5.4.1]
...
pytest-pylint==0.15.1
  - pylint [required: >=2.0.0, installed: 2.2.3]
    - astroid [required: >=2.0,<2.2.0, installed: 2.1.0]
      - lazy-object-proxy [required: Any, installed: 1.4.3]
      - six [required: Any, installed: 1.14.0]
      - typed-ast [required: Any, installed: 1.4.1]
      - wrapt [required: Any, installed: 1.12.1]
    - isort [required: >=4.2.5, installed: 4.3.21]
    - mccabe [required: Any, installed: 0.6.1]
  - pytest [required: >=5.0, installed: 5.4.1]
...
pytest-sanic==0.1.13
  - aiohttp [required: Any, installed: 3.6.2]
    - async-timeout [required: >=3.0,<4.0, installed: 3.0.1]
    - attrs [required: >=17.3.0, installed: 19.1.0]
    - chardet [required: >=2.0,<4.0, installed: 3.0.4]
    - idna-ssl [required: >=1.0, installed: 1.1.0]
      - idna [required: >=2.0, installed: 2.9]
    - multidict [required: >=4.5,<5.0, installed: 4.7.5]
    - typing-extensions [required: >=3.6.5, installed: 3.7.4.2]
    - yarl [required: >=1.0,<2.0, installed: 1.4.2]
      - idna [required: >=2.0, installed: 2.9]
      - multidict [required: >=4.0, installed: 4.7.5]
  - async-generator [required: Any, installed: 1.10]
  - pytest [required: Any, installed: 5.4.1]
...
pytest-timeout==1.3.4
  - pytest [required: >=3.6.0, installed: 5.4.1]

Notice that all those packages work fine with up to pytest 3.x or any version of it, except pytest-pylint==0.15.1 that needs pytest [required: >=5.0, installed: 5.4.1]!!! Could it be that pytest-pylint was causing all this mayhem?

Rememeber I mentioned that the last jussi docker image built by Steemit was 6 months ago? That puts pytest-pylint 0.14.1 right about that period.Sure enough, it does https://pypi.org/project/pytest-pylint/#history

Ok, one last try with Pipfile modified only with pytest-pylint = "==0.14.1".

SUCCESS. Jussi builds at last, although with many test warning like this

tests/test_jussi_request_response_middleware.py::test_urn_parts_not_in_options_response_headers
  /app/.venv/lib/python3.6/site-packages/sanic/testing.py:26: DeprecationWarning: verify_ssl is deprecated, use ssl=False instead
    conn = aiohttp.TCPConnector(verify_ssl=False)

But that's beyond the scope of this fix. It appears to be a benign problem, I'll let others look into it if it's critical.

Updating PyPi URL

And while we're at it, the Python Package Index (PyPI) has moved to a new domain. https://packaging.python.org/guides/migrating-to-pypi-org/

Uploads through pypi.python.org were switched off on July 3, 2017. As of April 13th, 2018, pypi.org is the URL for PyPI.

So fixing Pipfile
url = "https://pypi.python.org/simple"
to
url = "https://pypi.org/simple"

The final fix is at:
~~https://gitlab.syncad.com/hive/jussi/-/tree/fix-build~~
EDIT: Merged it into the autoclave
branch https://gitlab.syncad.com/hive/jussi/-/tree/autoclave https://gitlab.syncad.com/hive/jussi/-/commit/420af54d5d0d725045843e91c2ab027b06739221

Available & Reliable. I am your Witness. I want to represent You.

🗳 If you like what I do, consider voting for me 🗳

Check my guide about Voting for Witnesses.

Go to https://wallet.hive.blog/~witnesses. Find my name and click the vote button.

Alternatively you can use Hivesigner to vote for me or set me as proxy

Sort:

Trending

[-]

mytechtrail (63) 5 years ago

Thank you for the update and the hard work. I always look forward to hearing from witnesses and what they are doing. I try to stay as informed as I am able to, so as to try and make informed witness votes.

$0.31

1 vote

filotasriza3 (71) 5 years ago

i don't understand shit about programming languages :P but i know that you are working hard since the start of steemit so once again thanks for your hard work!

godact (48) 5 years ago

Chill bro, it will be sorted out soon...

$0.00

jasonrussell (69) 5 years ago

beep bop beep beep beep code beep...

rockor (60) 5 years ago

Oh I really wished I know all these coding stuffs. One of my heart desires and wish