Building Flowty¶
Flowty depends on OpenCV and FFmpeg. To build flowty outside of docker you will need to build and install these things yourself. This is surprisingly challenging given the finicky nature of OpenCV builds. It is also necessary to build OpenCV with CUDA support.
We provide detailed instructions on using conda to build an environment suitable for flowty. We rely on CUDA 9.2+ due to constraints of the relatively modern compiler conda-forge use. If you need to use an older CUDA version, then you’re best off compiling everything from scratch (including ffmpeg).
Building with conda¶
Warning
Note that we use gcc
7.3.0, which is fairly recent and only compatible with
CUDA 9.2+. If you need to use a previous version, you’re on your own and will have to
build everything from scratch (the conda packages expect a recent C++ ABI and you
won’t be able to compile OpenCV and link to FFmpeg).
The first thing we need to do to produce a CUDA-equipped build of OpenCV is to find out what graphics card you have, and what driver you’re running. This will define what PTX and BIN files we’ll generate when building OpenCV.
$ nvidia-smi
Fri May 10 21:33:15 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.56 Driver Version: 418.56 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Quadro K620 Off | 00000000:01:00.0 Off | N/A |
| 34% 39C P8 1W / 30W | 1MiB / 2001MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
Now look up the max CUDA version support available for your driver: https://docs.nvidia.com/deploy/cuda-compatibility/index.html
For 418.56, we can use up to CUDA 10.1 (as specified in the top right corner of
nvidia-smi
’s output, previous versions of nvidia-smi
don’t display this though).
We also see the graphics card is a Quadro K620, we need to look up its CUDA compute capability here: https://developer.nvidia.com/cuda-gpus. It lists 5.0 compute capability for this model, so we’ll generate CUDA OpenCV binaries for this compute capability.
Now we need to set up our environment to build OpenCV and flowty. We’ll assume you already have the following installed, if not install them before proceeding:
nvcc
(CUDA 9.2+, versions earlier than 9.2 don’t support GCC 7.3.0, the compiler we’ll use installed from conda-forge)conda
Hint
Any of the following variable definitions with <something>
are placeholders and
should be replaced based on your setup.
$ conda create -n flowty -c conda-forge python=3.6 pip ffmpeg lapack openblas \
cmake libjpeg-turbo libpng zlib cython numpy pytest gxx_linux-64=7.3.0
$ conda activate flowty
$ export OPENCV_VERSION=4.1.0
$ export PKG_CONFIG_PATH=$CONDA_PREFIX/lib/pkgconfig:$CONDA_PREFIX/lib64/pkgconfig
$ export CC=x86_64-conda_cos6-linux-gnu-gcc CXX=x86_64-conda_cos6-linux-gnu-gcc
$ export CUDA_PATH="<path-to-cuda>" # typically is /usr/local/cuda or similar
$ wget https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.tar.gz \
-O opencv-${OPENCV_VERSION}.tar.gz
$ wget https://github.com/opencv/opencv_contrib/archive/${OPENCV_VERSION}.tar.gz \
-O opencv_contrib-${OPENCV_VERSION}.tar.gz
$ mkdir -p opencv/build
$ cd opencv
$ tar -xvf ../opencv-${OPENCV_VERSION}.tar.gz
$ tar -xvf ../opencv_contrib-${OPENCV_VERSION}.tar.gz
$ cd build
$ unset CXXFLAGS CFLAGS # conda sets these by default when gxx_linux-64 is installed. These break .cu file compilation
$ export CUDA_COMPUTE_CAPABILITY=<your-gpu-compute-capability> # 5.0 in our example.
$ cmake \
-D CMAKE_PREFIX_PATH="$CONDA_PREFIX" \
-D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=$CONDA_PREFIX \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib-${OPENCV_VERSION}/modules \
-D BUILD_opencv_optflow=ON \
-D BUILD_opencv_cudaoptflow=ON \
-D BUILD_opencv_tracking=ON \
-D BUILD_opencv_calib3d=ON \
-D BUILD_opencv_features2d=ON \
-D BUILD_opencv_cudafeatures2d=ON \
-D BUILD_opencv_flann=ON \
-D BUILD_opencv_plot=ON \
-D BUILD_opencv_highgui=ON \
-D WITH_PNG=ON \
-D WITH_FFMPEG=ON \
-D WITH_CUDA=ON \
-D WITH_CUBLAS=ON \
-D WITH_OPENMP=ON \
-D WITH_OPENCL=ON \
-D WITH_LAPACK=ON \
-D WITH_JPEG=ON \
-D WITH_IPP=ON \
-D WITH_MKL=ON \
-D CUDA_TOOLKIT_ROOT_DIR="$CUDA_PATH" \
-D CUDA_FAST_MATH=ON \
-D CUDA_ARCH_PTX="$CUDA_COMPUTE_CAPABILITY" \
-D CUDA_ARCH_BIN="$CUDA_COMPUTE_CAPABILITY" \
-D ENABLE_FAST_MATH=ON \
-D WITH_ADE=OFF \
-D WITH_ARAVIS=OFF \
-D WITH_CLP=OFF \
-D WITH_EIGEN=OFF \
-D WITH_GDAL=OFF \
-D WITH_GDCM=OFF \
-D WITH_GPHOTO2=OFF \
-D WITH_GTK=OFF \
-D WITH_ITT=OFF \
-D WITH_JASPER=OFF \
-D WITH_LIBREALSENSE=OFF \
-D WITH_MFX=OFF \
-D WITH_OPENEXR=OFF \
-D WITH_OPENGL=OFF \
-D WITH_OPENNI=OFF \
-D WITH_OPENNI1=OFF \
-D WITH_OPENVX=OFF \
-D WITH_PROTOBUF=OFF \
-D WITH_PTHREADS_PF=OFF \
-D WITH_PVAPI=OFF \
-D WITH_QT=OFF \
-D WITH_QUIRC=OFF \
-D WITH_TBB=OFF \
-D WITH_TIFF=OFF \
-D WITH_V3L=OFF \
-D WITH_VA=OFF \
-D WITH_VA_INTEL=OFF \
-D WITH_VTK=OFF \
-D WITH_VULKAN=OFF \
-D WITH_WEBP=OFF \
-D WITH_XIMEA=OFF \
-D WITH_XINE=OFF \
-D WITH_HALIDE=OFF \
-D WITH_GSTREAMER=OFF \
-D WITH_V4L=OFF \
-D BUILD_EXAMPLES=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_opencv_apps=OFF \
-D BUILD_opencv_aruco=OFF \
-D BUILD_opencv_bgsegm=OFF \
-D BUILD_opencv_bioinspired=OFF \
-D BUILD_opencv_cudabgsegm=OFF \
-D BUILD_opencv_cudaobjdetect=OFF \
-D BUILD_opencv_cudastereo=OFF \
-D BUILD_opencv_datasets=OFF \
-D BUILD_opencv_dnn=OFF \
-D BUILD_opencv_dnn_objdetect=OFF \
-D BUILD_opencv_dpm=OFF \
-D BUILD_opencv_face=OFF \
-D BUILD_opencv_fuzzy=OFF \
-D BUILD_opencv_gapi=OFF \
-D BUILD_opencv_hfs=OFF \
-D BUILD_opencv_img_hash=OFF \
-D BUILD_opencv_java_bindings_generator=OFF \
-D BUILD_opencv_js=OFF \
-D BUILD_opencv_legacy=OFF \
-D BUILD_opencv_line_descriptor=OFF \
-D BUILD_opencv_ml=OFF \
-D BUILD_opencv_objdetect=OFF \
-D BUILD_opencv_phase_unwrapping=OFF \
-D BUILD_opencv_photo=OFF \
-D BUILD_opencv_python3=OFF \
-D BUILD_opencv_python_bindings_generator=OFF \
-D BUILD_opencv_quality=OFF \
-D BUILD_opencv_reg=OFF \
-D BUILD_opencv_rgbd=OFF \
-D BUILD_opencv_saliency=OFF \
-D BUILD_opencv_shape=OFF \
-D BUILD_opencv_stereo=OFF \
-D BUILD_opencv_stitching=OFF \
-D BUILD_opencv_stitching=OFF \
-D BUILD_opencv_structured_light=OFF \
-D BUILD_opencv_superres=OFF \
-D BUILD_opencv_surface_matching=OFF \
-D BUILD_opencv_text=OFF \
-D BUILD_opencv_videostab=OFF \
-D BUILD_opencv_xfeatures2d=OFF \
-D BUILD_opencv_xobjdetect=OFF \
-D BUILD_opencv_xphoto=OFF \
../opencv-${OPENCV_VERSION}
Note
Note that most of the cmake
build options in the above console session are
disabling additional features of OpenCV unused by flowty; if these cause errors, then
feel free to drop the OFF
options, they’re just to speed up compilation time and save space.
Once configured, you need to double check the cmake
output for FFmpeg, checking it
was found. If you get something like this…
-- Video I/O:
-- DC1394: NO
-- FFMPEG: NO
-- avcodec: NO
-- avformat: NO
-- avutil: NO
-- swscale: NO
-- avresample: NO
… then cmake
has been unable to resolve the location of FFmpeg headers and libs.
You should be aiming for something like this:
-- Video I/O:
-- DC1394: NO
-- FFMPEG: YES
-- avcodec: YES (58.35.100)
-- avformat: YES (58.20.100)
-- avutil: YES (56.22.100)
-- swscale: YES (5.3.100)
-- avresample: YES (4.0.0)
Typically this is as a result of the ffmpeg
pkgconfig files not being installed, or
present within a directory on the $PKG_CONFIG_PATH
.
Finally make
and make install
:
$ make -j $(nproc)
$ make install
Now you should have all the dependencies installed ready to build flowty.
$ mkdir flowty && cd flowty
$ export FLOWTY_VERSION=0.0.2
$ wget https://github.com/willprice/flowty/archive/v${FLOWTY_VERSION}.tar.gz \
-O flowty.tar.gz
$ tar -xvf flowty.tar.gz
$ cd flowty-${FLOWTY_VERSION}
$ python setup.py build_ext --inplace
You will probably need to add the $CONDA_PREFIX/lib64
directory to your
$LD_LIBRARY_PATH
as this is where opencv will have installed its shared libraries.
Without setting this you will get error like…
ImportError: libopencv_core.so.4 .1: cannot open shared object file: No such file or directory
Resolve this like so:
$ export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$CONDA_PREFIX/lib64
Check you can run the tests without failures:
$ PYTHONPATH=src pytest tests
If you were able to run the tests without any import failures, congratulations, you’re now ready to install flowty and compute some flow!
$ python setup.py install
$ flowty --help
Building on Ubuntu¶
Your best bet for manually setting up an environment to run flowty is to look at the Dockerfile we build upon: willprice/opencv4. This is built on Ubuntu 18.04 (although very few changes are needed to go back to 16.04). You can see the flags we enable for building OpenCV. The key flags are:
OPENCV_GENERATE_PKGCONFIG=on
as we usepkgconfig
insetup.py
to get the OpenCV paths.WITH_FFMPEG=on
since FFmpeg is the default backendWITH_CUDA=on
as some of the algorithms are CUDA acceleratedOPENCV_EXTRA_MODULES_PATH=<path/to/opencv_contrib/modules>
since theoptflow
module isn’t in core OpenCV.
Pay close attention to the output of cmake
as the configuration step won’t
crash if FFmpeg isn’t found, this will result in an OpenCV build incapable of
reading videos (upon attempting to read a video it will just return no frames
rather than raising an exception).
Once you’ve managed to build OpenCV and install it, then have a look at the flowty dockerfile to see how to build and install flowty. Basically it’s just:
$ pip3 install Cython numpy pytest
$ python setup.py build_ext --inplace
$ python setup.py install