python 2.7 - Differences in sys.path when python2.7 is invoked normally or via subprocess -

python 2.7 - Differences in sys.path when python2.7 is invoked normally or via subprocess -

question

why python2.7 when called using subprocess via python3 not have same sys.path python2.7 called normally? specifically, python2.7 via subprocess not have "/path/to/site-packages/" directory in sys.path.

context

i'd utilize fabric deploy django app i'm writing. problem i've written app in python3, fabric doesn't have explicit python3 back upwards yet. workaround, until fabric compatible python3, phone call fab script using subprocess.

for reason when phone call python2.7 using subprocess via python3, don't have access modules in site-packages.

python2.7 checks

i've got python2.7 , fabric==1.10.0 installed via enthought.

$ python /users/.../library/enthought/canopy_32bit/user/bin/python $ python --version python 2.7.6 -- 32-bit $ fab /users/.../library/enthought/canopy_32bit/user/bin/fab $ fab --version fabric 1.10.0 paramiko 1.15.1 subprocess checks

i have no problem calling fab within python2.7 using subprocess.

$ python enthought canopy python 2.7.6 | 32-bit | (default, apr 11 2014, 12:06:39) [gcc 4.2.1 (apple inc. build 5666) (dot 3)] on darwin type "help", "copyright", "credits" or "license" more information. >>> import subprocess >>> subprocess.check_output('fab --version', shell=true) 'fabric 1.10.0\nparamiko 1.15.1\n'

i have no problem calling python2.7 within python3 using subprocess.

$ python3 python 3.4.1 (v3.4.1:c0e311e010fc, may 18 2014, 00:54:21) [gcc 4.2.1 (apple inc. build 5666) (dot 3)] on darwin type "help", "copyright", "credits" or "license" more information. >>> import subprocess >>> subprocess.check_output('which python', shell=true) b'/users/.../library/enthought/canopy_32bit/user/bin/python\n' >>> subprocess.check_output('python --version', shell=true) python 2.7.6 -- 32-bit b'' distributionnotfound: fabric==1.10.0

however, though subprocess of python2.7 can "find" fab script, can't phone call it.

# python3 >>> subprocess.check_output(['which', 'fab']) b'/users/.../library/enthought/canopy_32bit/user/bin/fab\n' >>> subprocess.check_output(['fab', '--version']) traceback (most recent phone call last): file "/users/.../library/enthought/canopy_32bit/user/bin/fab", line 5, in <module> pkg_resources import load_entry_point file "/applications/canopy.app/appdata/canopy-1.4.0.1938.macosx-x86/canopy.app/contents/lib/python2.7/site-packages/pkg_resources.py", line 2877, in <module> working_set.require(__requires__) file "/applications/canopy.app/appdata/canopy-1.4.0.1938.macosx-x86/canopy.app/contents/lib/python2.7/site-packages/pkg_resources.py", line 698, in require needed = self.resolve(parse_requirements(requirements)) file "/applications/canopy.app/appdata/canopy-1.4.0.1938.macosx-x86/canopy.app/contents/lib/python2.7/site-packages/pkg_resources.py", line 596, in resolve raise distributionnotfound(req) pkg_resources.distributionnotfound: fabric==1.10.0 traceback (most recent phone call last): file "<stdin>", line 1, in <module> file "/library/frameworks/python.framework/versions/3.4/lib/python3.4/subprocess.py", line 620, in check_output raise calledprocesserror(retcode, process.args, output=output) subprocess.calledprocesserror: command '['fab', '--version']' returned non-zero exit status 1 site-packages not in sys.path

it appears python2.7 when called using subprocess via python3 not have same sys.path python2.7 called normally.

as expected, sys.path did not have enthought "site-packages" directory, contains fabric module.

# python3 >>> subprocess.check_output('python -c "import sys; print sys.path"', shell=true) ## not contain '/path/to/enthought/python2.7/site-packages' manually add together site-packages sys.path

to confirm it's possible: when manually add together right "site-packages" directory, can import fabric.

# python3 >>> subprocess.check_output('python -c\ "import sys; sys.path.append(\'/path/to/enthought/site-packages\');\ fabric import version; print version.get_version()"',\ shell = true) b'1.10.0\n' other options?

there's got improve way create sure python2.7, when invoked via subprocess python3, has same sys.path python2.7 invoked normally. can more familiar subprocess weigh in?

additional thoughts

it's interesting python2.7 can spawn python2.7 via subprocess , subprocess has right site-packages dir in sys.path.

$ python >>> import subprocess >>> subprocess.check_output('python -c "import sys; print sys.path"', shell=true) ## contains "/path/to/enthought/python2.7/site-packages"

i compared sys.path's python3, python3 subprocessed python3, , python3 subprocessed python2.7, , bit surprised find 3 resulted in same sys.path.

subprocess supports env parameter that, if given, environment called command -- create re-create of it, remove troublesome variables, , pass re-create subprocess:

my_env = os.environ.copy() del my_env['__pyenv_launcher__'] subprocess.check_output(..., env=my_env)

python-2.7 subprocess python-3.4 sys.path