16.2. Building the CamCOPS client¶
The CamCOPS client is written in C++11 using the Qt cross-platform framework.
16.2.1. Prerequisites¶
16.2.1.1. Linux¶
- Linux should come with Python and the necessary build tools.
- To build Android programs under Linux, you will also need a Java development
kit (JDK), such as OpenJDK:
sudo apt install openjdk-8-jdk
16.2.1.2. Windows¶
Install a recent version of Python. Make sure it’s on your
PATH
.Install a Microsoft Visual C++ compiler. A free one is Visual Studio Community. As you install Visual Studio, don’t forget to tick the C++ options.
Install these other tools:
- CMake. (We’ll use this version of cmake to build CamCOPS.)
- Cygwin and its packages
cmake
,gcc-g++
, andmake
. (If you missed them out during initial installation, just re-run the Cygwin setup program, such assetup-x86_64.exe
. SQLCipher requiresmake
.) - NASM, the Netwide Assembler for x86-family processors.
- ActiveState TCL. (SQLCipher requires
tclsh
.) - ActiveState Perl. (OpenSSL requires
perl
.) - Optionally, Debugging Tools for Windows (including CDB), such as from the Windows SDK.
Add everything to the
PATH
.In Windows 10, persistent environment variable settings are accessible by searching the Start menu for “environment variables”, or
.You can use either the User or the System settings, as you see fit.
PATH elements are separated with semicolons, if you edit the path manually.
For example, you may want these:
C:\cygwin64\bin C:\Program Files\NASM C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build -- These are usually added automatically by installers: C:\Program Files\Git\cmd C:\ActiveTcl\bin C:\Perl64\bin
Do make sure that the
PATH
doesn’t have an unquoted ampersand in; this is technically legal but it causes no end of trouble (seebuild_qt.py
). (The usual culprit is MySQL.) Thebuild_qt.py
script will check this.
Tested in July 2018 with:
ActivePerl 5.24.3 build 2404 (64-bit) ActiveTcl 8.6.7 build 0 (64-bit) CMake 3.12.0 (64-bit) Cygwin Setup 2.889 (64-bit) Microsoft Visual Studio Community 2017 NASM 2.13.03 (64-bit) Python 3.6 Qt Creator 4.7.0 Windows 10 (64-bit)
16.2.1.3. All operating systems¶
Install the open-source edition of Qt, with Qt Creator. (You only really need the Tools component. We will fetch Qt separately.)
Make sure you have Git installed.
Set some environment variables, so we can be consistent in these instructions. Specimen values:
Environment variable Example value (Linux) Example value (Windows) Notes CAMCOPS_QT_BASE_DIR ~/dev/qt_local_build
%USERPROFILE%\dev\qt_local_build
Read by build_qt.py
.CAMCOPS_SOURCE_DIR ~/dev/camcops
%USERPROFILE%\dev\camcops
Used in these instructions. CAMCOPS_VENV ~/dev/camcops_venv
%USERPROFILE%\dev\camcops_venv
Used in these instructions. Fetch CamCOPS. For example, for the GitHub version:
# Linux git clone https://github.com/RudolfCardinal/camcops $CAMCOPS_SOURCE_DIR
REM Windows git clone https://github.com/RudolfCardinal/camcops %CAMCOPS_SOURCE_DIR%
Create a virtual environment and install some Python tools:
# Linux python3 -m virtualenv $CAMCOPS_VENV . $CAMCOPS_VENV/bin/activate pip install cardinal_pythonlib==1.0.23
REM Windows python -m virtualenv %CAMCOPS_VENV% %CAMCOPS_VENV%\Scripts\activate pip install cardinal_pythonlib==1.0.23
16.2.2. Build OpenSSL, SQLCipher, Qt¶
Build a copy of Qt and supporting tools (OpenSSL, SQLCipher) from source using the CamCOPS build_qt.py tool (q.v.). For example:
# Linux
$CAMCOPS_SOURCE_DIR/tablet_qt/tools/build_qt.py --build_all
REM Windows
python %CAMCOPS_SOURCE_DIR%/tablet_qt/tools/build_qt.py --build_all
16.2.2.1. Troubleshooting¶
Problem (Windows): fatal error C1041: cannot open program database
'...\openssl-1.1.0g\app.pdb'; if multiple CL.EXE write to the same .PDB file,
please use /FS
… even when -FS
is in use via jom.
Solution: just run build_qt.py
again; this error usually goes away.
Presumably the Qt jom tool doesn’t always get things quite right with Visual
C++, and this error reflects parallel compilation processes clashing
occasionally. It’s definitely worth persisting, because Jom saves no end of
time.
If it fails repeatedly, add the --nparallel 1
option. (It seems to be the
OpenSSL build that’s prone to failing; you can always interrupt the program
after OpenSSL has finished, and use the full number of CPU cores for the much
longer Qt build.)
16.2.3. Run and set up Qt Creator¶
- Run Qt Creator.
- If you are compiling for Android:
- Install the Android SDK and the Android NDK.
- Configure your Android SDK/NDK and Java JDK at: , or in newer versions of Qt Creator, .
- Proceed with the instructions below.
16.2.4. Qt versions¶
See
.Assuming you set your qt_local_build directory to ~/dev/qt_local_build
, the
build_qt.py
script should have generated a series of qmake
(or, under
Windows, qmake.exe
) files within that directory:
Operating system qmake Linux 64-bit qt_linux_x86_64_install/bin/qmake Android (ARM) qt_android_armv7_install/bin/qmake Android emulator qt_android_x86_32_install/bin/qmake Mac OS/X 64-bit qt_osx_x86_64_install/bin/qmake iOS (ARM) qt_ios_armv8_64_install/bin/qmake iOS Simulator qt_ios_x86_64_install/bin/qmake Windows 32-bit qt_windows_x86_32_install/bin/qmake Windows 64-bit qt_windows_x86_64_install/bin/qmake
16.2.5. Qt kits¶
See
.Options last checked against Qt Creator 4.6.2 (built June 2018).
Note
If you did not install a version of Qt with Qt Creator, pick one of your
own kits and choose “Make Default”. Otherwise you will get the error
Could not find qmake spec 'default'.
(e.g. in the General Messages tab
when you open your application) and the ..pro
(project) file will not
parse. See https://stackoverflow.com/questions/27524680.
Custom_Linux_x86_64
Option Setting Name Custom_Linux_x86_64
File system name Device type Desktop Device Local PC (default for Desktop) Sysroot Compiler: C GCC (C, x86 64bit in /usr/bin
)Compiler: C++ GCC (x86 64bit in /usr/bin
)Environment [not editable: “No changes to apply.”] Debugger System GDB at /usr/bin/gdb
Qt version THE “LINUX 64-BIT” ONE FROM QT VERSIONS, ABOVE Qt mkspec CMake Tool System CMake at /usr/bin/cmake
CMake Generator CodeBlocks - Unix Makefiles CMake Configuration [not editable] Additional Qbs Profile Settings
Custom_Android_ARM
Option Setting Name Custom_Android_ARM
File system name Device type Android Device Device Run on Android (default for Android) Sysroot Compiler: C <No compiler> Compiler: C++ Android GCC (arm-4.9) Environment [not editable: “No changes to apply.”] Debugger Android Debugger for Android GCC (arm-4.9) Qt version THE “ANDROID” ONE FROM QT VERSIONS, ABOVE Qt mkspec CMake Tool System CMake at /usr/bin/cmake
CMake Generator CodeBlocks - Unix Makefiles CMake Configuration [not editable] Additional Qbs Profile Settings
Custom_Android_x86 – NOT FULLY TESTED
Option Setting Name Custom_Android_x86
File system name Device type Android Device Device Run on Android (default for Android) Sysroot Compiler: C <No compiler> Compiler: C++ Android GCC (i686-4.9) Environment [not editable: “No changes to apply.”] Debugger Android Debugger for Android GCC (i686-4.9) Qt version THE “ANDROID EMULATOR” ONE FROM QT VERSIONS, ABOVE Qt mkspec CMake Tool System CMake at /usr/bin/cmake
CMake Generator CodeBlocks - Unix Makefiles CMake Configuration [not editable] Additional Qbs Profile Settings
Custom_Windows_x86_64
Option Setting Name Custom_Windows_x86_64
File system name Device type Desktop Device Local PC (default for Desktop) Sysroot [...]\qt_local_build\qt_windows_x86_64_install\bin
Compiler: C Microsoft Visual C++ Compiler 14.0 (amd64) Compiler: C++ Microsoft Visual C++ Compiler 14.0 (amd64) Environment [not editable: “No changes to apply.”] Debugger Auto-detected CDB at C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe
Qt version THE “WINDOWS 64-BIT” ONE FROM QT VERSIONS, ABOVE Qt mkspec CMake Tool System CMake at C:\Program Files (x86)\CMake\bin\cmake.exe
CMake Generator CodeBlocks - MinGW Makefiles, Platform: <none>, Toolset: <none> CMake Configuration CMAKE_CXX_COMPILER:STRING=%{Compiler:Executable:Cxx}
CMAKE_C_COMPILER:STRING=%{Compiler:Executable:C}
CMAKE_PREFIX_PATH:STRING=%{Qt:QT_INSTALL_PREFIX}
QT_QMAKE_EXECUTABLE:STRING=%{Qt:qmakeExecutable}
Additional Qbs Profile Settings
16.2.6. Build settings¶
… let’s put them in a camcops.pro.shared
file:
http://doc.qt.io/qtcreator/creator-sharing-project-settings.html
16.2.6.1. General¶
- Open the
camcops.pro
project file in Qt Creator. - Add your chosen kit(s) to the CamCOPS project.
- Use defaults, except everywhere you see
-j 8
for an 8-CPU machine to get it compiling in parallel.- To save this effort, set
MAKEFLAGS="-j8"
in your user environment (e.g. in~/.bashrc
, or~/.profile
); see https://stackoverflow.com/questions/8860712/setting-default-make-options-for-qt-creator. HOWEVER, Qt Creator doesn’t seem to read that environment variable for me. Not sure why!
, add - To save this effort, set
- Build.
16.2.6.2. Android¶
Under
:
Option Setting PREVIOUSLY: android-23 [= default]. NOW: android-28 [= default]. ~/Documents/CamCOPS/android_keystore/CAMCOPS_ANDROID_KEYSTORE.keystore
[NB not part of published code, obviously!]Yes (at least for release versions) Do NOT tick. (Formerly, before 2018-06-25, this was . The objective remains to bundle Qt, not to install it via Ministro.)Additional libraries ~/dev/qt_local_build/openssl_android_armv7_build/openssl-1.1.0g/libcrypto.so
~/dev/qt_local_build/openssl_android_armv7_build/openssl-1.1.0g/libssl.so
Then in the file AndroidManifest.xml
(which Qt Creator has a custom editor
for):
Option Setting Package name org.camcops.camcops Version code [integer; may as well use consecutive] Version name [string] Minimum required SDK API 16: Android 4.1, 4.1.1 [default] Target SDK WAS: API 23: Android 6.0 [default]. AS OF 2018-06-25: API 26: Android 8.0 [Google Play Store requires this soon]. DOWNGRADED AGAIN 2018-07-16: OpenSSL problems. Probably because you have to rebuild OpenSSL for Android (see DEFAULT_ANDROID_API_NUM
inbuild_qt.py
).Application name CamCOPS Activity name CamCOPS Run camcops Application icon [icon] Include default permissions for Qt modules [tick] Include default features for Qt modules [tick] Boxes for other permissions [no other specific permission requested] But then you must also edit
AndroidManifest.xml
manually to include the line:<meta-data android:name="android.app.load_local_libs" android:value="-- %%INSERT_LOCAL_LIBS%% --:lib/libssl.so:lib/libcrypto.so"/> Note this bit: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For versions, see:
- https://developer.android.com/guide/topics/manifest/manifest-element.html
- https://developer.android.com/studio/publish/versioning.html
If you run this without a keystore, it produces a debug build (e.g.
QtApp-debug.apk
). If you run it with a keystore/signature, it produces
android-build-release-signed.apk
(formerly QtApp-release-signed.apk
).
The APK name is fixed at this point
(https://forum.qt.io/topic/43329/qt-5-3-1-qtcreator-rename-qtapp-debug-apk-to-myapp).
We can rename the APK if we want, or just upload to Google Play, distribute,
etc.
16.2.6.3. Linux¶
Under LD_LIBRARY_PATH=/home/rudolf/dev/qt_local_build/openssl_linux_x86_64_build/openssl-1.1.0g/
16.2.7. Google Play Store settings¶
Developer URL is https://play.google.com/apps/publish
App category: “Utility/other”.
Content rating: by Google’s definitions, CamCOPS hits criteria for references to illegal drugs (e.g. Deakin1HealthReview, and when strings are available, the various drug abuse scoring scales). Did not meet Google Play’s criteria for sex, violence, etc.
Note that “Pending publication” means you’re waiting for Google Play to sort itself out, not that you have to do anything.
Note re versions:
- As above, the AndroidManifest.xml has an INTEGER version, so we may as well use consecutive numbers. See the release history below.
The Google Developer site will check the version codes. Failed uploads can sometimes block that version number.
You upload a new version with
.Note also that if you try to install the .apk directly to a device that’s had an installation from Google Play Store, you’ll get the error INSTALL_FAILED_UPDATE_INCOMPATIBLE (I think). Or if you mix debug/release versions.
Finally, note that there can be a significant delay between uploading a new release and client devices seeing it on Google Play (or even being able to see it at https://play.google.com/store, or via the direct link at https://play.google.com/store/apps/details?id=org.camcops.camcops). Perhaps 10 minutes to the main web site?
16.2.8. Google Play Store release history¶
Google Play Store release name | AndroidManifest.xml version code | AndroidManifest.xml name | To Play Store on | Minimum Android API | Target Android API |
---|---|---|---|---|---|
2.0.1 (beta) | 2 | 2.0.1 | 2017-08-04 | 16 | 23 |
2.0.4 (beta) | 3 | 2.0.4 | 2017-10-22 | 16 | 23 |
2.2.3 (beta) | 5 | 2.2.3 | 2018-06-25 | 16 | 26 |
2.2.4 (beta) | 6 | 2.2.4 | 2018-07-18 | 23 | 26 |
2.2.6 (beta) | 7 | 2.2.6 | 2018-07-31 | 23 | 26 |
16.2.9. Notes¶
16.2.9.1. Version constraints for third-party software¶
- OpenSSL 1.0.x has long-term support and 1.1.x is the current release.
- OpenSSL 1.0.2h didn’t compile under 64-bit Windows, whereas OpenSSL 1.1.x did.
- OpenSSL 1.1.x requires Qt 5.10 or higher (https://bugreports.qt.io/browse/QTBUG-52905).
- SQLCipher supports OpenSSL 1.1.0 as of SQLCipher 3.4.1 (https://discuss.zetetic.net/t/sqlcipher-3-4-1-release/1962).
- Qt requires Android API ≥16 (http://doc.qt.io/qt-5/android-support.html).
- Qt 5.11.1 does not compile with the
android-16
toolchain (specifically its Bluetooth components). Qt looks for a Java packageandroid.bluetooth.le
, which is the Bluetooth Low Energy component that comes with Android SDK 18. So let’s try 18 as the minimum. That does compile. - Android libraries should be compiled for the same SDK version as
minSdkVersion
inAndroidManifest.xml
(see https://stackoverflow.com/questions/21888052/what-is-the-relation-between-app-platform-androidminsdkversion-and-androidtar/41079462#41079462, and https://developer.android.com/ndk/guides/stable_apis). - For whatever reasons, CamCOPS (v2.2.3-2.2.4) doesn’t run on Android 4.4.x (API 18) but does run on 6.0 (API 23); intermediates untested.
- Google Play store will require
targetSdkVersion
to be at least 26 from 2018-11-01 (https://developer.android.com/distribute/best-practices/develop/target-sdk). - Qt favour Android NDK r10e (the May 2015 release) (http://doc.qt.io/qt-5/androidgs.html) but r11c also seems to work fine.
16.2.9.2. Android¶
Above Android API 23, linking to non-public libraries is prohibited, possibly with exceptions for SSL/crypto.
- https://android-developers.googleblog.com/2016/06/android-changes-for-ndk-developers.html
- https://developer.android.com/about/versions/nougat/android-7.0-changes#ndk
I think this caused fatal problems for CamCOPS in 2018-07; not sure, but this might explain it.
Error: "unsupported_android_version" is not translated
: see https://bugreports.qt.io/browse/QTBUG-63952. This error does not prevent you from continuing.
16.2.9.3. Debugging¶
DON’T FORGET to set up both Debug and Release (+/- Profile) builds.
Phone USB debugging negotiation sometimes takes a while. On the Samsung Galaxy phone, the alert light goes red when in Debug mode.
If a USB Android device appears not to connect (via
adb devices
), appears then disappears as you connect it (vialsusb | wc
), and gives thedmesg
errordevice descriptor read/64, error -71
or similar, try a different cable (see https://stackoverflow.com/questions/9544557/debian-device-descriptor-read-64-error-71); try also plugging it directly into the computer’s USB ports rather than through a hub.If you lose the debugger windows in Qt Creator midway through a debug session, press Ctrl-4.
This error (with a variety of compiler names):
.../mkspecs/features/toolchain.prf(50): system(execute) requires one or two arguments. Project ERROR: Cannot run compiler 'g++'. Maybe you forgot to setup the environment?
means that you need to re-run qmake manually. It usually occurs if you delete your build* directories.
For debugging, consider install Valgrind:
sudo apt install valgrind
16.2.9.4. Oddities¶
- Sometimes you have to restart Qt creator after creating new build settings; it loses its .pro file and won’t show the project, or complains of a missing .pro file when you try to build.
- The first build can be very slow as it compiles all the resources; this usually looks like a process stuck compiling qrc_camcops.cpp to qrc_camcops.o
- If builds are very slow, you may have forgotten to use all your CPU cores;
try e.g.
-j 8
(for 8 cores) as an argument to make, as above. - If an Android build fails for a bizarre reason (like garbage in a .java file that looks like it’s been pre-supplied), delete the whole build directory, which is not always removed by cleaning.