OpenCV: Difference between revisions

From David's Wiki
 
(10 intermediate revisions by the same user not shown)
Line 13: Line 13:
** The [https://patents.google.com/patent/US6711293B1/en patent for SIFT] expired on March 6, 2020 so it might be added to opencv-contrib-python in a future update.
** The [https://patents.google.com/patent/US6711293B1/en patent for SIFT] expired on March 6, 2020 so it might be added to opencv-contrib-python in a future update.
** The [https://patents.google.com/patent/US20090238460A1/en patent for SURF] should expire around 2029.
** The [https://patents.google.com/patent/US20090238460A1/en patent for SURF] should expire around 2029.
* You can use <code>opencv-python</code> if you do not need extra contrib modules.


=== Non-free Algorithms ===
=== Non-free Algorithms ===


You need to compile [https://github.com/opencv/opencv OpenCV] from source alongside [https://github.com/opencv/opencv_contrib OpenCV-contrib].<br>
You need to compile [https://github.com/opencv/opencv OpenCV] from source alongside [https://github.com/opencv/opencv_contrib OpenCV-contrib].<br>
The easiest way is to use the automated script to compile opencv-contrib-python.<br>
The easiest way is to use [https://github.com/opencv/opencv-python#manual-builds the automated script] to compile opencv-contrib-python.<br>
Email me if you want a prebuilt wheel for Python 3.6 or 3.7.
{{hidden | Compile opencv-contrib-python |
{{hidden | Compile opencv-contrib-python |
You can build a wheel to install which includes NONFREE modules as follows:
You can build a wheel to install which includes NONFREE modules as follows:
Line 26: Line 26:
sudo apt install -y libavcodec-dev libavformat-dev libswscale-dev
sudo apt install -y libavcodec-dev libavformat-dev libswscale-dev
sudo apt install -y libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev
sudo apt install -y libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev
sudo apt install -y libpng-dev libjpeg-dev libopenexr-dev libtiff-dev libwebp-dev libjasper-dev
sudo apt install -y libpng-dev libjpeg-dev libopenexr-dev libtiff-dev libwebp-dev
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
sudo apt update
sudo apt update
Line 80: Line 80:
import cv2
import cv2


# cv2.IMREAD_ANYCOLOR = 4
# cv2.IMREAD_ANYCOLOR
# cv2.IMREAD_ANYDEPTH = 2
# cv2.IMREAD_ANYDEPTH
# cv2.IMREAD_COLOR = 1
# cv2.IMREAD_COLOR
# cv2.IMREAD_GRAYSCALE = 0
# cv2.IMREAD_GRAYSCALE
# cv2.IMREAD_UNCHANGED # G or BGR or BGRA


# Use cv2.IMREAD_GRAYSCALE to read in grayscale
# Use cv2.IMREAD_GRAYSCALE to read in grayscale
Line 97: Line 98:
===Resizing an Image===
===Resizing an Image===
[https://docs.opencv.org/master/da/d6e/tutorial_py_geometric_transformations.html Reference]
[https://docs.opencv.org/master/da/d6e/tutorial_py_geometric_transformations.html Reference]
<syntaxhighlight lang="bash">
<syntaxhighlight lang="python">
import numpy as np
# Resize to resolution
import cv2
new_img = cv2.resize(img, (500,200), interpolation=v2.INTER_CUBIC)
# Resize by factor
cv2.resize(img, (0, 0), fx=1/4, fy=1/4, interpolation=cv2.INTER_AREA)
</syntaxhighlight>
 
;Interpolation options
* INTER_NEAREST
* INTER_LINEAR
* INTER_CUBIC
* INTER_AREA
* INTER_LANCZOS4
 
For downscaling, use `INTER_AREA` to avoid aliasing. However, `INTER_NEAREST` will give the the optimal speed.<br>
For upscaling, use `INTER_CUBIC` for best results or `INTER_LINEAR` for best performance.


img = cv2.imread('messi5.jpg')
===Face Detection===
new_img = cv2.resize(img, (500,200), interpolation = cv.INTER_CUBIC)
[https://towardsdatascience.com/face-detection-in-2-minutes-using-opencv-python-90f89d7c0f81?gi=2fadb2dbc99d face detection in 2 minutes]
# Download [https://raw.githubusercontent.com/kipr/opencv/master/data/haarcascades/haarcascade_frontalface_default.xml haarcascade_frontalface_default.xml]
<syntaxhighlight lang="python">
face_cascade = cv2.CascadeClassifier(
            'haarcascade_frontalface_default.xml')
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(image_gray, 1.1, 4)
</syntaxhighlight>
</syntaxhighlight>


Line 156: Line 176:
output_video.release()
output_video.release()
</syntaxhighlight>
</syntaxhighlight>
==Algorithms==
===Stereo Rectify===
This is a summary of the [https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html#ga617b1685d4059c6040827800e72ad2b6 <code>cv2.stereoRectify</code>] function which produces a rectification rotation and projection matrix for calibrated cameras (i.e. cameras with known intrinsics and extrinsics).
[https://github.com/opencv/opencv/blob/4.x/modules/calib3d/src/calibration.cpp#LL2561C6-L2561C21 source code]
;Algorithm
# Rotation calculation
# Projection calculation
# Bounding box calculation
# Disparity-to-depth matrix calculation


==Resources==
==Resources==


*[https://docs.opencv.org/master/d6/d00/tutorial_py_root.html OpenCV Python Tutorial]
*[https://docs.opencv.org/master/d6/d00/tutorial_py_root.html OpenCV Python Tutorial]

Latest revision as of 19:16, 20 June 2023

OpenCV is a very popular computer vision and image processing library.
There are bindings for C++, Java, JavaScript, and Python

Installation

  • Download Python 3

Free algorithms only

pip install opencv-contrib-python
  • Missing SURF and SIFT.
    • The patent for SIFT expired on March 6, 2020 so it might be added to opencv-contrib-python in a future update.
    • The patent for SURF should expire around 2029.
  • You can use opencv-python if you do not need extra contrib modules.

Non-free Algorithms

You need to compile OpenCV from source alongside OpenCV-contrib.
The easiest way is to use the automated script to compile opencv-contrib-python.

Compile opencv-contrib-python

You can build a wheel to install which includes NONFREE modules as follows:

sudo apt install -y libavcodec-dev libavformat-dev libswscale-dev
sudo apt install -y libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev
sudo apt install -y libpng-dev libjpeg-dev libopenexr-dev libtiff-dev libwebp-dev
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
sudo apt update
sudo apt install -y libjasper1 libjasper-dev
# sudo apt install -y tesseract-ocr* liblept5 leptonica-progs libleptonica-dev
  • Setup a build environment.
    • Note that whatever version of qt you use to build, you will also need in the environment you install the wheel in. Ideally, build it in the same environment you want to use it in.
conda create -n opencvbuild python=3.7
conda activate opencvbuild
conda install -c conda-forge pyqt
  • Make a wheel
git clone --recurse-submodules [email protected]:skvark/opencv-python.git
cd opencv-python
export ENABLE_CONTRIB=1
export CMAKE_ARGS="-DOPENCV_ENABLE_NONFREE=ON -DWITH_QT=5"
python setup.py bdist_wheel
Compilation Instructions

See https://www.pyimagesearch.com/2018/08/15/how-to-install-opencv-4-on-ubuntu for instructions.

cd ~
git clone [email protected]:opencv/opencv.git
git clone [email protected]:opencv/opencv_contrib.git
cd opencv
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
	-D CMAKE_INSTALL_PREFIX=/usr/local \
	-D INSTALL_PYTHON_EXAMPLES=ON \
	-D INSTALL_C_EXAMPLES=OFF \
	-D OPENCV_ENABLE_NONFREE=ON \
	-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
	-D PYTHON3_EXECUTABLE=~/anaconda3/envs/tf2/bin/python \
	-D PYTHON3_INCLUDE_DIRS=~/anaconda3/envs/tf2/python3.7m \
	-D PYTHON3_LIBRARIES=~/anaconda3/envs/tf2/lib/libpython3.7m.so \
	-D BUILD_EXAMPLES=ON ..
make -j16
sudo make install
sudo ldconfig

Usage

Getting Started

import cv2

# cv2.IMREAD_ANYCOLOR
# cv2.IMREAD_ANYDEPTH
# cv2.IMREAD_COLOR
# cv2.IMREAD_GRAYSCALE
# cv2.IMREAD_UNCHANGED # G or BGR or BGRA

# Use cv2.IMREAD_GRAYSCALE to read in grayscale
my_image = cv2.imread("my_image.png", cv2.IMREAD_GRAYSCALE)

# Perform some modification
# Do your machine learning here

# Write your output image
cv2.imwrite("my_modified_image.png", my_image)

Resizing an Image

Reference

# Resize to resolution
new_img = cv2.resize(img, (500,200), interpolation=v2.INTER_CUBIC)
# Resize by factor
cv2.resize(img, (0, 0), fx=1/4, fy=1/4, interpolation=cv2.INTER_AREA)
Interpolation options
  • INTER_NEAREST
  • INTER_LINEAR
  • INTER_CUBIC
  • INTER_AREA
  • INTER_LANCZOS4

For downscaling, use `INTER_AREA` to avoid aliasing. However, `INTER_NEAREST` will give the the optimal speed.
For upscaling, use `INTER_CUBIC` for best results or `INTER_LINEAR` for best performance.

Face Detection

face detection in 2 minutes

  1. Download haarcascade_frontalface_default.xml
face_cascade = cv2.CascadeClassifier(
            'haarcascade_frontalface_default.xml')
image_gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
faces = face_cascade.detectMultiScale(image_gray, 1.1, 4)

Video

Reading Video

video_capture = cv2.VideoCapture(path.join(videos_folder, video_filename))
if not video_capture.isOpened():
    print("Error opening video stream or file")
    sys.exit(0)

frame_num = 0
while video_capture.isOpened():
    ret, frame = video_capture.read()
    if ret:
        cv2.imshow('Frame', frame)

        frame_num = frame_num + 1
        if cv2.waitKey(25) & 0xFF == ord('q'):
            break
    else:
        break
Parameters
# Resolution
width = int(video_capture.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video_capture.get(cv2.CAP_PROP_FRAME_HEIGHT))

total_frames = int(video_capture.get(cv2.CAP_PROP_FRAME_COUNT))
frame_pos = int(video_capture.get(cv2.CAP_PROP_POS_FRAMES))
fps = video_capture.get(cv2.CAP_PROP_FPS)

Writing Video

C++ Video Write
Note that OpenCV does not handle audio.

output_video = cv2.VideoWriter()
codec = cv2.VideoWriter_fourcc(*"avc1")
fps = 15
image_size = (100, 100)
output_video.open("video_output.mp4", codec, fps, image_size, True)
if not output_video.isOpened():
    print("Error opening output video")

# Write all of your frames
# while have_frames:
#   output_video.write(my_frame)

# Release the video
output_video.release()

Algorithms

Stereo Rectify

This is a summary of the cv2.stereoRectify function which produces a rectification rotation and projection matrix for calibrated cameras (i.e. cameras with known intrinsics and extrinsics).

source code

Algorithm
  1. Rotation calculation
  2. Projection calculation
  3. Bounding box calculation
  4. Disparity-to-depth matrix calculation

Resources