16.2. Building the CamCOPS client

The CamCOPS client is written in C++11 using the Qt cross-platform framework.

16.2.1. Prerequisites

16.2.1.1. Linux

  • Linux should come with Python and the necessary build tools.
  • To build Android programs under Linux, you will also need a Java development kit (JDK), such as OpenJDK: sudo apt install openjdk-8-jdk

16.2.1.2. Windows

  • Install a recent version of Python. Make sure it’s on your PATH.

  • Install a Microsoft Visual C++ compiler. A free one is Visual Studio Community. As you install Visual Studio, don’t forget to tick the C++ options.

  • Install these other tools:

    • CMake. (We’ll use this version of cmake to build CamCOPS.)
    • Cygwin and its packages cmake, gcc-g++, and make. (If you missed them out during initial installation, just re-run the Cygwin setup program, such as setup-x86_64.exe. SQLCipher requires make.)
    • NASM, the Netwide Assembler for x86-family processors.
    • ActiveState TCL. (SQLCipher requires tclsh.)
    • ActiveState Perl. (OpenSSL requires perl.)
    • Optionally, Debugging Tools for Windows (including CDB), such as from the Windows SDK.
  • Add everything to the PATH.

    • In Windows 10, persistent environment variable settings are accessible by searching the Start menu for “environment variables”, or Start ‣ Control Panel ‣ System and Security ‣ System ‣ Advanced System Settings ‣ Environment Variables.

    • You can use either the User or the System settings, as you see fit.

    • PATH elements are separated with semicolons, if you edit the path manually.

    • For example, you may want these:

      C:\cygwin64\bin
      C:\Program Files\NASM
      C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build
      
      -- These are usually added automatically by installers:
      
      C:\Program Files\Git\cmd
      C:\ActiveTcl\bin
      C:\Perl64\bin
      
    • Do make sure that the PATH doesn’t have an unquoted ampersand in; this is technically legal but it causes no end of trouble (see build_qt.py). (The usual culprit is MySQL.) The build_qt.py script will check this.

  • Tested in July 2018 with:

    ActivePerl 5.24.3 build 2404 (64-bit)
    ActiveTcl 8.6.7 build 0 (64-bit)
    CMake 3.12.0 (64-bit)
    Cygwin Setup 2.889 (64-bit)
    Microsoft Visual Studio Community 2017
    NASM 2.13.03 (64-bit)
    Python 3.6
    Qt Creator 4.7.0
    Windows 10 (64-bit)
    

16.2.1.3. All operating systems

  • Install the open-source edition of Qt, with Qt Creator. (You only really need the Tools component. We will fetch Qt separately.)

  • Make sure you have Git installed.

  • Set some environment variables, so we can be consistent in these instructions. Specimen values:

    Environment variable Example value (Linux) Example value (Windows) Notes
    CAMCOPS_QT_BASE_DIR ~/dev/qt_local_build %USERPROFILE%\dev\qt_local_build Read by build_qt.py.
    CAMCOPS_SOURCE_DIR ~/dev/camcops %USERPROFILE%\dev\camcops Used in these instructions.
    CAMCOPS_VENV ~/dev/camcops_venv %USERPROFILE%\dev\camcops_venv Used in these instructions.
  • Fetch CamCOPS. For example, for the GitHub version:

    # Linux
    git clone https://github.com/RudolfCardinal/camcops $CAMCOPS_SOURCE_DIR
    
    REM Windows
    git clone https://github.com/RudolfCardinal/camcops %CAMCOPS_SOURCE_DIR%
    
  • Create a virtual environment and install some Python tools:

    # Linux
    python3 -m virtualenv $CAMCOPS_VENV
    . $CAMCOPS_VENV/bin/activate
    pip install cardinal_pythonlib==1.0.23
    
    REM Windows
    python -m virtualenv %CAMCOPS_VENV%
    %CAMCOPS_VENV%\Scripts\activate
    pip install cardinal_pythonlib==1.0.23
    

16.2.2. Build OpenSSL, SQLCipher, Qt

Build a copy of Qt and supporting tools (OpenSSL, SQLCipher) from source using the CamCOPS build_qt.py tool (q.v.). For example:

# Linux
$CAMCOPS_SOURCE_DIR/tablet_qt/tools/build_qt.py --build_all
REM Windows
python %CAMCOPS_SOURCE_DIR%/tablet_qt/tools/build_qt.py --build_all

16.2.2.1. Troubleshooting

Problem (Windows): fatal error C1041: cannot open program database '...\openssl-1.1.0g\app.pdb'; if multiple CL.EXE write to the same .PDB file, please use /FS

… even when -FS is in use via jom.

Solution: just run build_qt.py again; this error usually goes away. Presumably the Qt jom tool doesn’t always get things quite right with Visual C++, and this error reflects parallel compilation processes clashing occasionally. It’s definitely worth persisting, because Jom saves no end of time.

If it fails repeatedly, add the --nparallel 1 option. (It seems to be the OpenSSL build that’s prone to failing; you can always interrupt the program after OpenSSL has finished, and use the full number of CPU cores for the much longer Qt build.)

16.2.3. Run and set up Qt Creator

  • Run Qt Creator.
  • If you are compiling for Android:
    • Install the Android SDK and the Android NDK.
    • Configure your Android SDK/NDK and Java JDK at: Tools ‣ Options ‣ Android, or in newer versions of Qt Creator, Tools ‣ Options ‣ Devices ‣ Android ‣ Android Settings.
  • Proceed with the instructions below.

16.2.4. Qt versions

See Tools ‣ Options ‣ Kits ‣ Qt Versions.

Assuming you set your qt_local_build directory to ~/dev/qt_local_build, the build_qt.py script should have generated a series of qmake (or, under Windows, qmake.exe) files within that directory:

Operating system qmake
Linux 64-bit qt_linux_x86_64_install/bin/qmake
Android (ARM) qt_android_armv7_install/bin/qmake
Android emulator qt_android_x86_32_install/bin/qmake
Mac OS/X 64-bit qt_osx_x86_64_install/bin/qmake
iOS (ARM) qt_ios_armv8_64_install/bin/qmake
iOS Simulator qt_ios_x86_64_install/bin/qmake
Windows 32-bit qt_windows_x86_32_install/bin/qmake
Windows 64-bit qt_windows_x86_64_install/bin/qmake

16.2.5. Qt kits

See Tools ‣ Options ‣ Kits ‣ Kits.

Options last checked against Qt Creator 4.6.2 (built June 2018).

Note

If you did not install a version of Qt with Qt Creator, pick one of your own kits and choose “Make Default”. Otherwise you will get the error Could not find qmake spec 'default'. (e.g. in the General Messages tab when you open your application) and the ..pro (project) file will not parse. See https://stackoverflow.com/questions/27524680.

Custom_Linux_x86_64

Option Setting
Name Custom_Linux_x86_64
File system name  
Device type Desktop
Device Local PC (default for Desktop)
Sysroot  
Compiler: C GCC (C, x86 64bit in /usr/bin)
Compiler: C++ GCC (x86 64bit in /usr/bin)
Environment [not editable: “No changes to apply.”]
Debugger System GDB at /usr/bin/gdb
Qt version THE “LINUX 64-BIT” ONE FROM QT VERSIONS, ABOVE
Qt mkspec  
CMake Tool System CMake at /usr/bin/cmake
CMake Generator CodeBlocks - Unix Makefiles
CMake Configuration [not editable]
Additional Qbs Profile Settings  

Custom_Android_ARM

Option Setting
Name Custom_Android_ARM
File system name  
Device type Android Device
Device Run on Android (default for Android)
Sysroot  
Compiler: C <No compiler>
Compiler: C++ Android GCC (arm-4.9)
Environment [not editable: “No changes to apply.”]
Debugger Android Debugger for Android GCC (arm-4.9)
Qt version THE “ANDROID” ONE FROM QT VERSIONS, ABOVE
Qt mkspec  
CMake Tool System CMake at /usr/bin/cmake
CMake Generator CodeBlocks - Unix Makefiles
CMake Configuration [not editable]
Additional Qbs Profile Settings  

Custom_Android_x86 – NOT FULLY TESTED

Option Setting
Name Custom_Android_x86
File system name  
Device type Android Device
Device Run on Android (default for Android)
Sysroot  
Compiler: C <No compiler>
Compiler: C++ Android GCC (i686-4.9)
Environment [not editable: “No changes to apply.”]
Debugger Android Debugger for Android GCC (i686-4.9)
Qt version THE “ANDROID EMULATOR” ONE FROM QT VERSIONS, ABOVE
Qt mkspec  
CMake Tool System CMake at /usr/bin/cmake
CMake Generator CodeBlocks - Unix Makefiles
CMake Configuration [not editable]
Additional Qbs Profile Settings  

Custom_Windows_x86_64

Option Setting
Name Custom_Windows_x86_64
File system name  
Device type Desktop
Device Local PC (default for Desktop)
Sysroot [...]\qt_local_build\qt_windows_x86_64_install\bin
Compiler: C Microsoft Visual C++ Compiler 14.0 (amd64)
Compiler: C++ Microsoft Visual C++ Compiler 14.0 (amd64)
Environment [not editable: “No changes to apply.”]
Debugger Auto-detected CDB at C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\cdb.exe
Qt version THE “WINDOWS 64-BIT” ONE FROM QT VERSIONS, ABOVE
Qt mkspec  
CMake Tool System CMake at C:\Program Files (x86)\CMake\bin\cmake.exe
CMake Generator CodeBlocks - MinGW Makefiles, Platform: <none>, Toolset: <none>
CMake Configuration CMAKE_CXX_COMPILER:STRING=%{Compiler:Executable:Cxx} CMAKE_C_COMPILER:STRING=%{Compiler:Executable:C} CMAKE_PREFIX_PATH:STRING=%{Qt:QT_INSTALL_PREFIX} QT_QMAKE_EXECUTABLE:STRING=%{Qt:qmakeExecutable}
Additional Qbs Profile Settings  

16.2.6. Build settings

… let’s put them in a camcops.pro.shared file: http://doc.qt.io/qtcreator/creator-sharing-project-settings.html

16.2.6.1. General

  • Open the camcops.pro project file in Qt Creator.
  • Add your chosen kit(s) to the CamCOPS project.
  • Use defaults, except everywhere you see Build Settings ‣ Build Steps ‣ Make ‣ Make arguments, add -j 8 for an 8-CPU machine to get it compiling in parallel.
  • Build.

16.2.6.2. Android

Under Project ‣ Build Settings ‣ Build Steps ‣ Build Android APK:

Option Setting
Application ‣ Android build SDK PREVIOUSLY: android-23 [= default]. NOW: android-28 [= default].
Sign package ‣ Keystore ~/Documents/CamCOPS/android_keystore/CAMCOPS_ANDROID_KEYSTORE.keystore [NB not part of published code, obviously!]
Sign package ‣ Sign package Yes (at least for release versions)
Advanced actions ‣ Use Ministro service to install Qt Do NOT tick. (Formerly, before 2018-06-25, this was Qt deployment ‣ Bundle Qt libraries in APK. The objective remains to bundle Qt, not to install it via Ministro.)
Additional libraries ~/dev/qt_local_build/openssl_android_armv7_build/openssl-1.1.0g/libcrypto.so ~/dev/qt_local_build/openssl_android_armv7_build/openssl-1.1.0g/libssl.so

Then in the file AndroidManifest.xml (which Qt Creator has a custom editor for):

Option Setting
Package name org.camcops.camcops
Version code [integer; may as well use consecutive]
Version name [string]
Minimum required SDK API 16: Android 4.1, 4.1.1 [default]
Target SDK WAS: API 23: Android 6.0 [default]. AS OF 2018-06-25: API 26: Android 8.0 [Google Play Store requires this soon]. DOWNGRADED AGAIN 2018-07-16: OpenSSL problems. Probably because you have to rebuild OpenSSL for Android (see DEFAULT_ANDROID_API_NUM in build_qt.py).
Application name CamCOPS
Activity name CamCOPS
Run camcops
Application icon [icon]
Include default permissions for Qt modules [tick]
Include default features for Qt modules [tick]
Boxes for other permissions [no other specific permission requested]

But then you must also edit AndroidManifest.xml manually to include the line:

<meta-data android:name="android.app.load_local_libs" android:value="-- %%INSERT_LOCAL_LIBS%% --:lib/libssl.so:lib/libcrypto.so"/>
Note this bit:                                                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For versions, see:

If you run this without a keystore, it produces a debug build (e.g. QtApp-debug.apk). If you run it with a keystore/signature, it produces android-build-release-signed.apk (formerly QtApp-release-signed.apk). The APK name is fixed at this point (https://forum.qt.io/topic/43329/qt-5-3-1-qtcreator-rename-qtapp-debug-apk-to-myapp). We can rename the APK if we want, or just upload to Google Play, distribute, etc.

16.2.6.3. Linux

Under Build Settings ‣ Build Environment`, set e.g. LD_LIBRARY_PATH=/home/rudolf/dev/qt_local_build/openssl_linux_x86_64_build/openssl-1.1.0g/

16.2.7. Google Play Store settings

  • Developer URL is https://play.google.com/apps/publish ‣ pick your application ‣ e.g. Release management / App releases

  • App category: “Utility/other”.

  • Content rating: by Google’s definitions, CamCOPS hits criteria for references to illegal drugs (e.g. Deakin1HealthReview, and when strings are available, the various drug abuse scoring scales). Did not meet Google Play’s criteria for sex, violence, etc.

  • Note that “Pending publication” means you’re waiting for Google Play to sort itself out, not that you have to do anything.

  • Note re versions:

    • As above, the AndroidManifest.xml has an INTEGER version, so we may as well use consecutive numbers. See the release history below.

    The Google Developer site will check the version codes. Failed uploads can sometimes block that version number.

  • You upload a new version with App releases ‣ Create Release.

  • Note also that if you try to install the .apk directly to a device that’s had an installation from Google Play Store, you’ll get the error INSTALL_FAILED_UPDATE_INCOMPATIBLE (I think). Or if you mix debug/release versions.

  • Finally, note that there can be a significant delay between uploading a new release and client devices seeing it on Google Play (or even being able to see it at https://play.google.com/store, or via the direct link at https://play.google.com/store/apps/details?id=org.camcops.camcops). Perhaps 10 minutes to the main web site?

16.2.8. Google Play Store release history

Google Play Store release name AndroidManifest.xml version code AndroidManifest.xml name To Play Store on Minimum Android API Target Android API
2.0.1 (beta) 2 2.0.1 2017-08-04 16 23
2.0.4 (beta) 3 2.0.4 2017-10-22 16 23
2.2.3 (beta) 5 2.2.3 2018-06-25 16 26
2.2.4 (beta) 6 2.2.4 2018-07-18 23 26
2.2.6 (beta) 7 2.2.6 2018-07-31 23 26

16.2.9. Notes

16.2.9.1. Version constraints for third-party software

16.2.9.2. Android

16.2.9.3. Debugging

  • DON’T FORGET to set up both Debug and Release (+/- Profile) builds.

  • Phone USB debugging negotiation sometimes takes a while. On the Samsung Galaxy phone, the alert light goes red when in Debug mode.

  • If a USB Android device appears not to connect (via adb devices), appears then disappears as you connect it (via lsusb | wc), and gives the dmesg error device descriptor read/64, error -71 or similar, try a different cable (see https://stackoverflow.com/questions/9544557/debian-device-descriptor-read-64-error-71); try also plugging it directly into the computer’s USB ports rather than through a hub.

  • If you lose the debugger windows in Qt Creator midway through a debug session, press Ctrl-4.

  • This error (with a variety of compiler names):

    .../mkspecs/features/toolchain.prf(50): system(execute) requires one or two arguments.
    Project ERROR: Cannot run compiler 'g++'. Maybe you forgot to setup the environment?
    

    means that you need to re-run qmake manually. It usually occurs if you delete your build* directories.

  • For debugging, consider install Valgrind: sudo apt install valgrind

16.2.9.4. Oddities

  • Sometimes you have to restart Qt creator after creating new build settings; it loses its .pro file and won’t show the project, or complains of a missing .pro file when you try to build.
  • The first build can be very slow as it compiles all the resources; this usually looks like a process stuck compiling qrc_camcops.cpp to qrc_camcops.o
  • If builds are very slow, you may have forgotten to use all your CPU cores; try e.g. -j 8 (for 8 cores) as an argument to make, as above.
  • If an Android build fails for a bizarre reason (like garbage in a .java file that looks like it’s been pre-supplied), delete the whole build directory, which is not always removed by cleaning.