UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

Adam Taylor’s MicroZed Chronicles Part 142: OpenCV EV Object Tracking Part Deux

by Xilinx Employee ‎08-08-2016 09:51 AM - edited ‎08-08-2016 09:58 AM (19,764 Views)

 

By Adam Taylor

 

As I wrapped up the last blog, I had introduced the concepts that we are going to be using to develop the algorithm.

 

We can put these concepts together very simply to create the object-tracking algorithm. To do this we will use the following algorithm:

 

  • Attempt to open the Web camera
  • Wait 5 seconds – this delay takes Web delay into account to accommodate the Webcam I am using. This long latency might be required for the Webcam to supply its first frame.
  • Capture the first frame from the Webcam. This first frame serves as our reference background frame. We will detect any changes that occur in the scene from this first frame.
  • Convert the color space from RGB to grey scale.
  • Perform a Gaussian blur on the image.
  • Loop until any key is pressed and then perform the following:
    1. Capture another image from the web cam
    2. Convert the color space from RGB to grey scale
    3. Perform a Gaussian blur on the image
    4. Calculate the absolute difference between the reference frame and the most recent captured image
    5. Perform a thresholding operation to create the binary image
    6. Dilate the image to enlarge the differences
    7. Find the Contours remaining in the binary image
    8. Ignore any contours with too large an area
    9. Draw a box around each of the contours detected

 

What is very impressive is that regardless of our language of choice for implementation, we can implement this algorithm in very few lines of code.

 

We did not discuss the Gaussian blur last week. However the performance of many image-processing applications that detect objects or edges can be negatively affected by noise present within the grabbed frame. Blurring the image prior to further processing reduces noise within the frame and this technique is often used for image processing, particularly for edge-detection algorithms (e.g. the Laplacian of Gaussian edge-detection algorithm).

 

The sequence of frames shown below shows the results for each particular step in the above algorithm for points 2 through 6.

 

 

Image1.jpg

 

Frame converted to grey scale (2)

 

 

 

Image2.jpg

 

Grey scale image following blurring (3)

 

 

 

Image3.jpg

 

Absolute difference between reference and current frame (4)

 

 

 

Image4.jpg

 

Thresholded image (5)

 

 

 

Image5.jpg

 

Dilated image (6)

 

 

Once we have the binary image dilated, we run an OpenCV contour-finding algorithm on it. What are contours? They are curves joining all the points of an image with the same intensity or color. When we work with binary images, contours become the boundaries of the image.  Within OpenCV, contour finding falls within the structure analysis and shape descriptors functions.

 

There are a few things we need to know when using the find contours function. The first is that the function will change the image passed to it, so it’s often good to use a copy of the image and not the original image. Secondly, the find-contours function will only detect the contours of a white object in a binary image, so we must be sure to threshold appropriately.

 

The contour operation provides a list of vectors for each of the detected objects in the binary image passed to it. As the scene changes and as objects enter the frame, the number of objects within the dilated image and hence the number of returned contours could be large. We will need to eliminate contours with a small area. If we fail to do this, the performance of our algorithm will needlessly degrade and we will detect irrelevant things like small wind-induced motion, as can be seen in the image below where the shrubbery is moving in the breeze. This movement unnecessarily clutters the final image.

 

 

Image6.jpg

 

Image with no discrimination on contours

 

 

To ensure that we only track the objects of interest such as moving cars and people and to blot out fluctuations due to the breeze, we can use an OpenCV function that calculates the area of the contour. If the contour is below a certain threshold, we discard it and waste no more processing resource on it.

 

If the contour does have an area that’s above our threshold, then we want to calculate and draw a box around the area of interest. Again we can do this using an OpenCV function that calculates the bounded rectangle for each of the remaining contour points in the frame. We can then use this information to draw the number of rectangles we require on the original image to identify the objects that have changed since we took the reference frame.

 

This is a very simple image-detection algorithm, but it is one we can implement with relative ease.

 

Next week we will look at a few more image-processing algorithms before we venture into how we can use OpenCV with the Xilinx Vivado HLS, the High-Level Synthesis environment. (Note: To watch a 15-minute video about using OpenCV libraries with Xilinx Vivado HLx to create Zynq-based implementations, click here.)

 

 

 

 

Code is available on Github as always.

 

If you want E book or hardback versions of previous MicroZed chronicle blogs, you can get them below.

 

 

 

  • First Year E Book here
  • First Year Hardback here.

 

 

 MicroZed Chronicles hardcopy.jpg

 

 

 

  • Second Year E Book here
  • Second Year Hardback here

 

 

 

 MicroZed Chronicles Second Year.jpg

 

 

 

Labels
About the Author
  • Be sure to join the Xilinx LinkedIn group to get an update for every new Xcell Daily post! ******************** Steve Leibson is the Director of Strategic Marketing and Business Planning at Xilinx. He started as a system design engineer at HP in the early days of desktop computing, then switched to EDA at Cadnetix, and subsequently became a technical editor for EDN Magazine. He's served as Editor in Chief of EDN Magazine, Embedded Developers Journal, and Microprocessor Report. He has extensive experience in computing, microprocessors, microcontrollers, embedded systems design, design IP, EDA, and programmable logic.