OpenCV

OpenCV is a very popular computer vision and image processing library.
There are bindings for C++, Java, JavaScript, and Python

Installation

• Download Python 3

Free algorithms only

pip install opencv-contrib-python

• Missing SURF and SIFT.
  • The patent for SIFT expired on March 6, 2020 so it might be added to opencv-contrib-python in a future update.
  • The patent for SURF should expire around 2029.
• You can use opencv-python if you do not need extra contrib modules.

Non-free Algorithms

You need to compile OpenCV from source alongside OpenCV-contrib.
The easiest way is to use the automated script to compile opencv-contrib-python.

Compile opencv-contrib-python

You can build a wheel to install which includes NONFREE modules as follows:

• Install all dependencies listed in the build tutorial

sudo apt install -y libavcodec-dev libavformat-dev libswscale-dev
sudo apt install -y libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev
sudo apt install -y libpng-dev libjpeg-dev libopenexr-dev libtiff-dev libwebp-dev
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
sudo apt update
sudo apt install -y libjasper1 libjasper-dev
# sudo apt install -y tesseract-ocr* liblept5 leptonica-progs libleptonica-dev

• Setup a build environment.
  • Note that whatever version of qt you use to build, you will also need in the environment you install the wheel in. Ideally, build it in the same environment you want to use it in.

conda create -n opencvbuild python=3.7
conda activate opencvbuild
conda install -c conda-forge pyqt

• Make a wheel

git clone --recurse-submodules [email protected]:skvark/opencv-python.git
cd opencv-python
export ENABLE_CONTRIB=1
export CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON -DWITH_QT=5"
python setup.py bdist_wheel

Compilation Instructions

See https://www.pyimagesearch.com/2018/08/15/how-to-install-opencv-4-on-ubuntu for instructions.

cd ~
git clone [email protected]:opencv/opencv.git
git clone [email protected]:opencv/opencv_contrib.git
cd opencv
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
	-D CMAKE_INSTALL_PREFIX=/usr/local \
	-D INSTALL_PYTHON_EXAMPLES=ON \
	-D INSTALL_C_EXAMPLES=OFF \
	-D OPENCV_ENABLE_NONFREE=ON \
	-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
	-D PYTHON3_EXECUTABLE=~/anaconda3/envs/tf2/bin/python \
	-D PYTHON3_INCLUDE_DIRS=~/anaconda3/envs/tf2/python3.7m \
	-D PYTHON3_LIBRARIES=~/anaconda3/envs/tf2/lib/libpython3.7m.so \
	-D BUILD_EXAMPLES=ON ..
make -j16
sudo make install
sudo ldconfig

Usage

Getting Started

import cv2

# cv2.IMREAD_ANYCOLOR
# cv2.IMREAD_ANYDEPTH
# cv2.IMREAD_COLOR
# cv2.IMREAD_GRAYSCALE
# cv2.IMREAD_UNCHANGED # G or BGR or BGRA

# Use cv2.IMREAD_GRAYSCALE to read in grayscale
my_image = cv2.imread("my_image.png", cv2.IMREAD_GRAYSCALE)

# Perform some modification
# Do your machine learning here

# Write your output image
cv2.imwrite("my_modified_image.png", my_image)

Resizing an Image

Reference

# Resize to resolution
new_img = cv2.resize(img, (500,200), interpolation=v2.INTER_CUBIC)
# Resize by factor
cv2.resize(img, (0, 0), fx=1/4, fy=1/4, interpolation=cv2.INTER_AREA)

Interpolation options

• INTER_NEAREST
• INTER_LINEAR
• INTER_CUBIC
• INTER_AREA
• INTER_LANCZOS4

For downscaling, use `INTER_AREA` to avoid aliasing. However, `INTER_NEAREST` will give the the optimal speed.
For upscaling, use `INTER_CUBIC` for best results or `INTER_LINEAR` for best performance.

Face Detection

face detection in 2 minutes

1. Download haarcascade_frontalface_default.xml

face_cascade = cv2.CascadeClassifier(
            'haarcascade_frontalface_default.xml')
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(image_gray, 1.1, 4)

Video

Reading Video

video_capture = cv2.VideoCapture(path.join(videos_folder, video_filename))
if not video_capture.isOpened():
    print("Error opening video stream or file")
    sys.exit(0)

frame_num = 0
while video_capture.isOpened():
    ret, frame = video_capture.read()
    if ret:
        cv2.imshow('Frame', frame)

        frame_num = frame_num + 1
        if cv2.waitKey(25) & 0xFF == ord('q'):
            break
    else:
        break

Parameters

# Resolution
width = int(video_capture.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video_capture.get(cv2.CAP_PROP_FRAME_HEIGHT))

total_frames = int(video_capture.get(cv2.CAP_PROP_FRAME_COUNT))
frame_pos = int(video_capture.get(cv2.CAP_PROP_POS_FRAMES))
fps = video_capture.get(cv2.CAP_PROP_FPS)

Writing Video

C++ Video Write
Note that OpenCV does not handle audio.

output_video = cv2.VideoWriter()
codec = cv2.VideoWriter_fourcc(*"avc1")
fps = 15
image_size = (100, 100)
output_video.open("video_output.mp4", codec, fps, image_size, True)
if not output_video.isOpened():
    print("Error opening output video")

# Write all of your frames
# while have_frames:
#   output_video.write(my_frame)

# Release the video
output_video.release()

Algorithms

Stereo Rectify

This is a summary of the cv2.stereoRectify function which produces a rectification rotation and projection matrix for calibrated cameras (i.e. cameras with known intrinsics and extrinsics).

source code

Algorithm

1. Rotation calculation
2. Projection calculation
3. Bounding box calculation
4. Disparity-to-depth matrix calculation

Resources

• OpenCV Python Tutorial