Bundle-python » History » Revision 43
Revision 42 (Adrian Georgescu, 03/04/2018 03:05 PM) → Revision 43/44 (Adrian Georgescu, 03/04/2018 03:47 PM)
h1. Building a Python Framework to bundle inside Blink In order to avoid using the system Python a custom Framework build is needed. Using a bundled Python version will make the package bigger in size, but all package versions are controlled and not up to the environment. Also, we can use the latest Python version, with latest bugfixes and features, since Apple only updates the system Python version on every major OS release. The following instructions only apply for 64bit builds, 32bit builds are no longer supported. Blink dependencies must be installed under the following directory structure: * Distribution/Frameworks/ * Distribution/Resources/lib h2. Building the Python Framework itself * Install it using Homebrew <pre> brew install python (this may install python 3) To install 2.7: brew install python2 or https://stackoverflow.com/questions/18671253/how-can-i-use-homebrew-to-install-both-python-2-and-3-on-mac brew install pyenv pyenv install 2.7 </pre> The framework will be installed and linked with Homebrew supplied OpenSSL and SQLite versions. Those libraries will need to be copied too. *NOTE*: Be careful when copying the framework around, it contains symlinks and if @cp -r@ is used the size will we doubled, use @cp -a@ instead. The Python framework is found in <pre> cp -a /usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework ~/work/blink/Distribution/Frameworks/ </pre> * Reduce the size of the Python Framework: There are a number of things that can (and must when submitting a sandbox app to Mac App Store) be removed from the framework directory to make it smaller in size: <pre> cd ~/work/blink/Distribution/Frameworks/Python.framework ~/work/blink/Distribution/Frameworks//Python.framework cd Versions/2.7 ln -sf Current 2.7 cd ~/work/blink/Distribution/Frameworks/Python.framework ~/work/blink/Distribution/Frameworks//Python.framework ln -sf Versions/Current/Headers . ln -sf Versions/Current/Python . ln -sf Versions/Current/Resources . find . -name *.pyc -exec rm -r "{}" \; find . -name *.pyo -exec rm -r "{}" \; rm -r Versions/Current/lib/python2.7/config/python.o rm -r Versions/Current/bin rm -r Versions/Current/Resources/* rm -r Versions/Current/lib/python2.7/test rm -r Versions/Current/lib/python2.7/plat-* rm -r Versions/Current/lib/python2.7/idlelib rm -r Versions/Current/lib/python2.7/curses rm -r Versions/Current/lib/python2.7/lib2to3 rm -r Versions/Current/lib/python2.7/lib-tk rm -r Versions/Current/lib/python2.7/bsddb rm -r Versions/Current/lib/python2.7/lib-dynload/gdbm.so rm -r Versions/Current/lib/python2.7/lib-dynload/readline.so rm -r Versions/2.7/lib/python2.7/site-packages </pre> Replace @Versions/Current/lib/python2.7/site.py@ with an empty file. <pre> rm ~/work/blink/Distribution/Frameworks//Python.framework/Versions/Current/lib/python2.7/site.py touch ~/work/blink/Distribution/Frameworks//Python.framework/Versions/Current/lib/python2.7/site.py </pre> Python Framework needs file a Info.plist file under Resources in order to be compatible with latest OSX bundle structure: <pre> cp build_scripts/PythonFramework.plist Distribution/Frameworks/Python.framework/Resources/Info.plist </pre> h2. Compiling PyObjC In order to get a PyObjC version that will work with the framework created above (Python 2.7, 64bits) an equivalent Python must be used to compile it. That is, if has to be a Python 2.7 version (it doesn't have to be the exact version) and it has to be a 64bit version. The MACOSX_DEPLOYMENT_TARGET must also be set to the appropriate value. PyObjcC can be installed with easy_install or pip. We install it in 2 steps to save some compilation time due to a bug in the build system: <pre> pip install pyobjc-core pip install pyobjc pip install pycrypto </pre> When compiling PyObjC a Python package will be created for every system framework, but not all of them are needed (at the moment), so just pick the ones we use: <pre> AddressBook AppKit Cocoa CoreFoundation Foundation JavaScriptCore LaunchServices PyObjCTools Quartz ScriptingBridge StoreKit WebKit objc </pre> For example this is the content of a Resources/lib bundled with Blink Cocoa as of November 3rd, 2016 (including sipsimple dependencies & all): <pre> AVFoundation AddressBook AppKit Cocoa CoreFoundation Crypto Foundation LaunchServices PyObjCTools Quartz ScriptingBridge WebKit _cffi_backend.so _ldap.so _markerlib application cffi cjson.so cryptography cryptography-1.5.1.dist-info dateutil dns dsml.py enum eventlib formencode gmpy2.so gnutls greenlet.so idna ipaddress.py ldap ldapurl.py ldif.py lxml msrplib objc otr pkg_resources pyasn1 pycparser pydispatch pytz service_identity sipsimple six.py sqlobject twisted xcaplib </pre> *NOTE:* The _objc_ package is located inside a _PyObjC_ directory, just copy it from there, without the parent directory. <pre> __import__('pkg_resources').declare_namespace(__name__) </pre> h2. Fix library paths All libraries must have their relative path change to the Framework path bundled within Blink.app <pre> #!/bin/sh old_path="local/lib/\|local/Cellar/\|/usr/local/opt/libmpc/lib/\|/usr/local/opt/mpfr/lib/\|Frameworks/Frameworks/\|/Users/adigeo/work/ag-projects/video/local/lib/" new_path="@executable_path/../Frameworks/" for library in $@; do install_name_tool -id $new_path$library $library dependencies=$(otool -L $library | grep $old_path | awk '{print $1}') for dependency in $dependencies; do new_basename=$(basename $dependency) new_name="$new_path$new_basename" echo $dependency $new_name $library install_name_tool -change $dependency $new_name $library done done </pre> A script is available in ./build_scripts/ directory <pre> ./build_scripts/change_lib_names.sh Distribution/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/*.so chmod +w Distribution/Frameworks/Python.framework/Versions/Current/Python ./build_scripts/change_lib_names.sh Distribution/Frameworks/Python.framework/Versions/Current/Python </pre> *NOTE*: Python.framework as well as all other libraries must be signed using command line tools. Make sure when building Blink that "Code sign on copy" option is disabled for Python.framework. This script can be used to sign all libraries and frameworks <pre> sos=`find ./Resources/lib -name *.so`; for s in $sos; do codesign -f -s '3rd Party Mac Developer Application: AG Projects' $s; done sos=`find ./Frameworks -name *.dylib`; for s in $sos; do codesign -f -s '3rd Party Mac Developer Application: AG Projects' $s; done sos=`find ./Frameworks -name *.so`; for s in $sos; do codesign -f -s '3rd Party Mac Developer Application: AG Projects' $s; done sos=`find ./Frameworks -name *.o`; for s in $sos; do codesign -f -s '3rd Party Mac Developer Application: AG Projects' $s; done sos=`find ./Frameworks -name *.a`; for s in $sos; do codesign -f -s '3rd Party Mac Developer Application: AG Projects' $s; done </pre> A script is available in ./build_scripts/ directory <pre> ./build_scripts/codesign.sh </pre> h2. Module exceptions When copying built Python modules into the distribution folder, care must be taken with the 2 following packages: * zope: an empty @__init__.py@ file must be created in the @zope@ directory * cryptography: the @*-dist.info@ must be copied too * _PyObjCTools_ is not a valid Python package, as it lacks a @__init__.py@ file, an empty one needs to be manually created with this content: h1. Creating a sandbox (Python virtualenv) <pre> sudo easy_install pip sudo pip install virtualenv virtualenvwrapper </pre> Add to ~.bashrc <pre> # Virtualenv export WORKON_HOME=$HOME/.virtualenvs export PIP_VIRTUALENV_BASE=$WORKON_HOME export PIP_RESPECT_VIRTUALENV=true export VIRTUALENVWRAPPER_SCRIPT=/usr/local/bin/virtualenvwrapper.sh [[ -f /usr/local/bin/virtualenvwrapper_lazy.sh ]] && source /usr/local/bin/virtualenvwrapper_lazy.sh </pre> Creating a sandbox: <pre> mkvirtualenv -p $(which python2.7) sandbox </pre> Exiting the sandbox: <pre> deactivate </pre> Entering the sandbox: <pre> workon sandbox </pre>