OpenCV on Android — Tiny with Optimization Enabled

Subset of OpenCV on Android

This article is also published on LearnOpenCV

My OpenCV Android SDK = small size library

If you choose OpenCV for production, your primary goal is to bring down the size of the library and also make it performance packed. OpenCV is an awesome library with tons of algorithms but you must be using a very small subset of these algorithm in your application, hence it makes perfect sense to include what is required and leave out the rest.

Library can be compiled statically along with your application code or can be dynamically linked at runtime and this is completely application development decision. Here we will create shared object (.so) for this demo.

Android is a versatile OS and can run on multiple hardwares, from mobile phones to IOT devices, Raspberry Pi, to various single board computers and therefore it become important to cross compile the code for that particular computer architecture called ISA. As we are building it for Android, we would need its built tools. Android has a fancy name for its build tools called, NDK (Native Development Kit), anyways, but that’s what we need.

Following are the list of software and their respective version we are going to use:

  1. cmake — 3.7.2
  2. NDK — r14b
  3. OpenCV — 3.4.1
  4. Host — Ubuntu — 16.4
  5. Target — armeabi-v7a (ARM based)
  6. Android SDK 25.2.5 tools.

Download NDK and unzip it to your work area. There are various version of NDK, hence it is important to read the release notes and pick the one you need for your project. But for this exercise its Android NDK Release 14b on Ubuntu inside Docker container. But this can be on any Linux box.

root@dc:/opt/android-ndk-r14b# ls
CHANGELOG.md ndk-build ndk-gdb ndk-which prebuilt shader-tools source.properties sysroot
build ndk-depends ndk-stack platforms python-packages simpleperf sources toolchains

We will build the standalone toolchain for OpenCV compilation,

root@dc:/opt/android-ndk-r14b# ./build/tools/make_standalone_toolchain.py \
--arch arm \
--api 23 \
--install-dir /tmp/my-android-toolchain

This command will create toolchain based on --arch and --api that i need for my project and install them in /tmp/my-android-toolchain directory.

Often ANDROID_NDK path is set to the prebuilt toolchain, that comes along with the NDK. But we will use the one that we built in the previous step, we need to let cmake know this important fact and hence set the environment variable.

$ export ANDROID_STANDALONE_TOOLCHAIN=/tmp/my-android-toolchain/

Thats pretty much our build environment needs to know, there are no other variables required. If you already have ANDROID_NDK set, please unset it.

Building for Android also needs few important packages called Ninja and Ant, we would install them as well:

$ sudo apt-get install ninja-build ant

Download OpenCV and there is no better place to find download steps than Satya Mallick’s blog. Either you can compile the way blog suggested or just download all the packages, OpenCV repository and leave it there. Because we are compiling for Android so we can skip compilation part mentioned in the blog. If you need contrib modules, download them for this article i am skipping the,

Anyways, let us move to OpenCV directory.

$ cd opencv/
$ mkdir build
$ cd build
$ cmake \
-DCMAKE_TOOLCHAIN_FILE=../platforms/android/android.toolchain.cmake\
-DANDROID_STL=gnustl_shared \
-DANDROID_NATIVE_API_LEVEL=23 ..

Note: make a fail if the SDK is higher than 25.2.5 for OpenCV 3.0. Please make sure you have appropriate Android SDK in place.

The output will show the configuration for this build and this is the step where you can change the configuration of your likes.

For ARM based devices, optimization i usually enable using:

  1. NEON
  2. VFPV3

Check out preamble of platforms/android/android.toolchain.cmake for various configuration options.

$ make -j $nproc

Hopefully the build goes fine and you have ready ELF output in the form of .so (shared object), you can also generate output as .a, for this you need to enable appropriate flag in the cmake. The output would look like following:

root@dc:/opencv/build/lib/armeabi-v7a# ls
libopencv_calib3d.a libopencv_flann.a libopencv_java3.so libopencv_shape.a libopencv_video.a
libopencv_core.a libopencv_highgui.a libopencv_ml.a libopencv_stitching.a libopencv_videoio.a
libopencv_dnn.a libopencv_imgcodecs.a libopencv_objdetect.a libopencv_superres.a libopencv_videostab.a
libopencv_features2d.a libopencv_imgproc.a libopencv_photo.a libopencv_ts.a

libopencv_java3.so is what you need for your android project, but if you look at the size of it, its around 9.5MB:

root@dc:/opencv/build/lib/armeabi-v7a# du -h libopencv_java3.so
9.4M libopencv_java3.so

That is because, it has all the modules included. To verify this, run following command:

$ strings -a libopencv_java3.so | grep .cpp

Lets trim the library for our needs. Let us say we need just two modules core and imageproc:

root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ \
-shared -o libopencv_tiny.so \
--sysroot=/tmp/my-android-toolchain/sysroot/ \
-Wl,--whole-archive libopencv_core.a \
libopencv_imgproc.a -Wl,--no-whole-archive

now check the file type and size of linopencv_tiny.so

root@dc:/opencv/build/lib/armeabi-v7a# file libopencv_tiny.so
libopencv_tiny.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, not stripped
root@dc:/opencv/build/lib/armeabi-v7a# du -h libopencv_tiny.so
8.3M libopencv_tiny.so

This is non-striped, let us strip it:

root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-strip --strip-unneeded libopencv_tiny.soroot@dc:/opencv/build/lib/armeabi-v7a# file libopencv_tiny.so
libopencv_tiny.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, stripped
root@dc:/opencv/build/lib/armeabi-v7a# du -h libopencv_tiny.so
3.4M libopencv_tiny.so

You can further drill down and pick each .o file rather than .a file, that can be achieved using ar tool:

$ /tmp/my-android-toolchain/bin/arm-linux-androideabi-ar x  libopencv_core.a

ar tool extract all the .o from .a file and later you can make .so from them the same way we created before, the command would look like following:

root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ \
-shared -o libopencv_tiny.so \
--sysroot=/tmp/my-android-toolchain/sysroot/ \
-Wl, — whole-archive *.o -Wl, — no-whole-archive

Linking is where the efforts ends, and let us cross that milestone too. As this is cross compiled for a specific platform, it is important to include few extract libraries in the .so, these extra libraries are completely 3rdparty and platform specific. If you do not include them you will get linking error, hence let us do it:

root@dc:/opencv/build/lib/armeabi-v7a# /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ -L /opencv/build/3rdparty/lib/armeabi-v7a  -lz -llog -ljnigraphics -ltegra_hal -lcpufeatures -lm -lgnustl_shared -fexceptions -frtti -fpic  -shared -Wl,-soname,libopencv_java.so -o libopencv_java.so  -Wl,--whole-archive libopencv_core.a libopencv_imgproc.a  -Wl,--no-whole-archive

Dependences should look like following, few of these extra libraies are .a so they won’t show up in the dependences.

root@dc:/opencv/build/lib/armeabi-v7a# readelf -d libopencv_tiny.soDynamic section at offset 0x3615d8 contains 33 entries:
Tag Type Name/Value
0x00000003 (PLTGOT) 0x362b18
0x00000002 (PLTRELSZ) 2488 (bytes)
0x00000017 (JMPREL) 0x3f674
0x00000014 (PLTREL) REL
0x00000011 (REL) 0x28cb4
0x00000012 (RELSZ) 92608 (bytes)
0x00000013 (RELENT) 8 (bytes)
0x6ffffffa (RELCOUNT) 11570
0x00000006 (SYMTAB) 0x148
0x0000000b (SYMENT) 16 (bytes)
0x00000005 (STRTAB) 0xaf78
0x0000000a (STRSZ) 97135 (bytes)
0x00000004 (HASH) 0x22ae8
0x00000001 (NEEDED) Shared library: [libz.so]
0x00000001 (NEEDED) Shared library: [liblog.so]
0x00000001 (NEEDED) Shared library: [libjnigraphics.so]
0x00000001 (NEEDED) Shared library: [libm.so]
0x00000001 (NEEDED) Shared library: [libc.so]
0x00000001 (NEEDED) Shared library: [libdl.so]
0x0000000e (SONAME) Library soname: [libopencv_tiny.so]
0x0000001a (FINI_ARRAY) 0x35e7b4
0x0000001c (FINI_ARRAYSZ) 8 (bytes)
0x00000019 (INIT_ARRAY) 0x362554
0x0000001b (INIT_ARRAYSZ) 132 (bytes)
0x00000010 (SYMBOLIC) 0x0
0x0000001e (FLAGS) SYMBOLIC BIND_NOW
0x6ffffffb (FLAGS_1) Flags: NOW
0x6ffffff0 (VERSYM) 0x27690
0x6ffffffc (VERDEF) 0x28c58
0x6ffffffd (VERDEFNUM) 1
0x6ffffffe (VERNEED) 0x28c74
0x6fffffff (VERNEEDNUM) 2
0x00000000 (NULL) 0x0

Make sure you strip the newly created .so before checking the size.

Now, your shared object ( libopencv_tiny.so ) is ready for android deployment. Often it is given libopencv_java3.so name where 3 is the version but do not worry its just a file name, it will still work or make it android complaint and name it as libopencv_java3.so.

Let us say we have following code in the file demo.cpp inside directory ./demo, that you want to link with created .so:

#include "opencv2/opencv.hpp"
#include<iostream>
int main()
{
std::cout << "OpenCV Version " << CV_VERSION << std::endl;
}

We MUST use the same toolchain to compile and link demo.cpp, can you guess why?

Ok! now that you are thinking, let me quickly start the compilation:

root@dc: /tmp/my-android-toolchain/bin/arm-linux-androideabi-g++ -L/demo  -I /demo/include/ -Wall    -o demo demo.cpp -lopencv_tiny

I have moved the generated libopencv_tiny.so to folder ./demo and all the opencv includes (i mean .hpp), this is just to demonstrated that we are in isolation and not using /usr/local/include

Now, coming back to my question, why same toolchain?

root@5c7fe9a72d74:/demo# readelf -h libopencv_tiny.so
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: DYN (Shared object file)
Machine: ARM
Version: 0x1
Entry point address: 0x0
Start of program headers: 52 (bytes into file)
Start of section headers: 3561860 (bytes into file)
Flags: 0x5000200, Version5 EABI, soft-float ABI
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
Number of program headers: 8
Size of section headers: 40 (bytes)
Number of section headers: 27
Section header string table index: 26

Did you get it? Ok, let me just say it: The .so is cross compiled for a specific arch type and that is ARM where as out host machine is x86 based linux. This is the reason, you need to run the executable file only on the same target machine type.

If you build it on x86, you must also tell where your .so resides, you can keep it anywhere you like, i mean in any folder of your project or workarea, as this run time linking, ld should be able to find it. The best way to do this is by updating an environment variable:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<your .so path>

Now, say you have become very generous and want to share it with everyone on the system. For that you need to have root privileges. You will need this for two reasons:

  • To put the library in a standard system location, probably /usr/lib or /usr/local/lib, which normal users don’t have write access to.
  • You will need to modify the ld.so config file and cache. As root, do the following:
cp /demo/libopencv_tiny.so /usr/local/lib
chmod 0755 /usr/local/lib/libopencv_tiny.so

We need to inform ld that it’s available for use, hence let’s update the cache:

ldconfig

Now, we do not need LD_LIBRARY_PATH, so let’s clear it too. CAUTION: you clear your own path from LD_LIBRARY_PATH only.

unset LD_LIBRARY_PATH

I am including reply suggested by Yk Order. Beautify of using OpenCVMakefile, is that all dependences are taken care and you need to pass the appropriate build tools for your platform.

Example for Android :

cmake -DCMAKE_TOOLCHAIN_FILE=../platforms/android/android.toolchain.cmake \
-D ANDROID_ABI=arm64-v8a \
-D CMAKE_BUILD_TYPE=Release \
-D ANDROID_NATIVE_API_LEVEL=23 \
-D WITH_CUDA=OFF \
-D WITH_MATLAB=OFF \
-D BUILD_ANDROID_EXAMPLES=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_TESTS=OFF \
-D ANDROID_STL=c++_shared \
-D BUILD_SHARED_LIBS=ON \
-D BUILD_opencv_objdetect=OFF \
-D BUILD_opencv_video=OFF \
-D BUILD_opencv_videoio=OFF \
-D BUILD_opencv_features2d=OFF \
-D BUILD_opencv_flann=OFF \
-D BUILD_opencv_highgui=OFF \
-D BUILD_opencv_ml=OFF \
-D BUILD_opencv_photo=OFF \
-D BUILD_opencv_python=OFF \
-D BUILD_opencv_shape=OFF \
-D BUILD_opencv_stitching=OFF \
-D BUILD_opencv_superres=OFF \
-D BUILD_opencv_ts=OFF \
-D BUILD_opencv_videostab=OFF \
..

On your Linux or MacOS: (It will include imgproc and highgui modules in the build)

cmake
-D CMAKE_BUILD_TYPE=Release \
-D WITH_CUDA=OFF \
-D WITH_MATLAB=OFF \
-D BUILD_ANDROID_EXAMPLES=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_SHARED_LIBS=ON \
-D BUILD_opencv_objdetect=OFF \
-D BUILD_opencv_video=OFF \
-D BUILD_opencv_videoio=OFF \
-D BUILD_opencv_features2d=OFF \
-D BUILD_opencv_flann=OFF \
-D BUILD_opencv_ml=OFF \
-D BUILD_opencv_photo=OFF \
-D BUILD_opencv_python=OFF \
-D BUILD_opencv_shape=OFF \
-D BUILD_opencv_stitching=OFF \
-D BUILD_opencv_superres=OFF \
-D BUILD_opencv_ts=OFF \
-D BUILD_opencv_videostab=OFF \
-D BUILD_opencv_highgui=ON \
-D BUILD_opencv_imgproc=ON \
..

You can pick your modules you need in final build, in fact i may have missed quite a few.

Enjoy the power of dynamic linking!

Co-founder Logits Systems https://logits.systems. Biometric on the edge. https://tomdeore.wixsite.com/epoch

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store