Orange Tree Technologies’ ZestSC3 dev board pairs a Xilinx Artix-7 A35T FPGA with a Cypress Semiconductor FX3 SuperSpeed USB 3.0 peripheral controller. There’s also 512Mbytes of DDR3 SDRAM on board. The result is a small 40x50mm board that can accelerate just about any task while connected to a host over a very high speed serial link (360Mbytes/sec in each direction). The Cypress FX3 chip supplies the USB 3.0 connection and has an ARM9 microprocessor core on chip. There’s pre-installed firmware that causes the board to act like a USB peripheral so you don’t need detailed USB knowledge to get the host interface up and running. Also, you can program the board’s Flash configuration memory over the USB connection (no need for a programming cable) or via a JTAG port. Orange Tree Technologies supplies FPGA cores, example code, and OS drivers (Windows and Linux).
Orange Tree Technologies’ ZestSC3 dev board pairs a Xilinx Artix-7 A35T FPGA with a Cypress Semiconductor FX3 SuperSpeed USB 3.0 controller
Please contact Orange Tree Technologies for more information.
Mike Dini has been working with FPGAs for almost as long as there have been FPGAs to work with. His Dini Group specializes in taking the latest, greatest Xilinx FPGAs and putting them on “big,” high-performance dev, prototyping, and Ethernet engine boards with the most fanciful names in the industry. Here are just a few examples from the company’s large catalog:
So you just know that when Mike Dini speaks, he’ll say something interesting (perhaps “colorful” is more appropriate). You can have a chance to hear him speak at next month’s FPGAworld 2016 conference being held in Stockholm, Sweden. Mike’s keynote is at 3:45pm on September 13.
You’ll also have a chance to sit in on a technical overview for the Xilinx Zynq UltraScale+ MPSoC (starts at 10:30am).
Registration info is here. Registration is free.
If you don’t watch carefully, you’ll miss it. In this week’s 55-minute Mailbag video, EEVBlog’s Dave Jones thumbs through the July/August issue of Elektor magazine and I spotted an article titled “Filtering on the [Zynq-based] Red Pitaya” at 27:34 in the video. The Red Pitaya is a high-performance, open instrumentation platform based on the Xilinx Zynq-7000 All Programmable SoC. It was developed in Slovenia by a bunch of engineers who specialize in developing instrumentation for linear accelerators.
For more information about the Red Pitaya, see:
Like tens of thousands of other fans, I just love watching EEVBlog’s Dave Jones tear apart electronic products that people randomly send him in the mail. Dave’s motto: “Don’t turn it on. Take it apart!” (Be sure to use the appropriate Australian accent when you say it.) Sometimes, like yesterday, there’s a Xilinx-flavored Easter egg in the video. This week’s EEVBlog Mailbag segment includes a short review and teardown of the $399.95 Epiphan AV.io 4K HDMI-to-USB 3.0 Capture Card.
To my surprise, when the covers and the heat sink came off, there on the board was a Xilinx Artix-7 A50T FPGA paired with an Analog Devices ADV7619 HDMI receiver. Below is Dave’s 55-minute Mailbag video from this week. The AV.io 4K HDMI Video Capture Card review starts at 15:35 in the video. You’ll want to watch just to see the Crocodile Dundee knife Dave uses to open his mail.
(As Dave would say, none of this 3-minute YouTube video rubbish. Hour-long videos—those are real videos. The fact that more than 35,000 viewers have watched this video within the first 24 hours attest to Dave’s Ozzie wisdom.)
Let’s cut to the chase, shall we? Below is a screen capture from the video when the chips on the AV.io 4K HDMI Video Capture Card’s board are revealed under the thermal paste.
Epiphan AV.io 4K HDMI Video Capture Card circuit board with Xilinx Artix-7 A50T FPGA
So what’s the Artix-7 FPGA doing in the AV.io 4K HDMI Video Capture Card? The Analog Devices ADV7619 HDMI receiver translates and splits the HDMI signal—in any of twelve configurable capture resolutions ranging from 640x480 to 4096x2160 (it also adapts to other resolutions)—into component video and digital-audio streams. The Artix-7 FPGA then performs real-time video hardware scaling with “near-zero” latency to convert the digital video to any of ten output resolutions—from 640x360 to 4096x2160—and then sends the resulting video and audio streams into a USB 3.0 chip for capture by a USB video-recording device or PC. The FPGA also resamples the HDMI audio to output 16-bit stereo audio at 48KHz.
You can’t do that sort of real-time, near-zero-latency video scaling and conversion with a microprocessor. Way…too…slow. You need fast, “grunty” hardware like the programmable logic in the Artix-7 FPGA to perform “all the magic.”
Step right up folks, the Hot Chips show is unfolding this week at the Flint Center for the Performing Arts in Cupertino, California and DeePhi has rolled out a convolutional neural network (CNN) acceleration processor named Aristotle that is faster and more energy efficient than CPUs or GPUs performing the same tasks. Is DeePhi’s CNN Processor some sort of super ASIC based on carbon nanotubes or memristors and powered by Mr. Fusion? Nope.
DeePhi’s Aristotle CNN Processor is based on a Xilinx Zynq-7000 All Programmable SoC.
DeePhi Aristotle Processor Slide from Hot Chips 2016
According to this article on Nextplatform.com, DeePhi’s CEO and co-founder Song Yao said that “CPUs don’t have the energy efficiency, GPUs are great for training but lack ‘the efficiency in inference’, DSP approaches don’t have high enough performance and have a high cache miss rate and of course, ASICs are too slow to market—and even when produced, finding a large enough market to justify development cost is difficult.
“FPGA based deep learning accelerators meet most requirements. They have acceptable power and performance, they can support customized architecture and have high on-chip memory bandwidth and are very reliable.”
Yao reportedly said: “The FPGA based DPU platform achieves an order of magnitude higher energy efficiency over GPU on image recognition and speech detection.” He also said that “traditionally, FPGA based acceleration via hand coding took a few months; using OpenCL and the related tool chain brings that down to one month.”
DeePhi was launched in March, 2016 and leverages the results of work from teams at Stanford University and Tsinghua University.
One big obstruction between you and the finish line for your design is timing closure. As your design gets more complex and as you crank the clock rate to get more performance, timing closure gets more and more difficult to achieve.
Sound familiar? All too familiar? Want some help?
Got some for you.
There’s a new White Paper titled “A Methodology for Repeatable and Reliable Timing Closure” that will help you develop procedures to close timing in a repeatable and reliable fashion using the Xilinx Vivado HLx Design Suite. These procedures are based on the Xilinx UltraFast Design Methodology Guide and the process flow in the White Paper looks like this:
Vivado HLx Timing Closure Flow
The process makes use of tool directives for Vivado’s placer that generate statistical data over multiple runs. This data will lead you to a more optimal layout that closes timing more easily. The results of this statistical gathering process can answer questions such as:
Download the White Paper here.
For more information about the Xen hypervisor for the Xilinx Zynq UltraScale+ MPSoC, see:
For more information, contact DornerWorks directly.
Appropriately located in the Glenn Miller Ballroom on the University of Colorado at Boulder campus, the GNU Radio Conference (GRCon) 2016 will focus on the latest research and the newest products to implement software-defined radio based on GNU Radio, a free and open software radio ecosystem. There’s a 1-day New Users’ Intro Day (Monday, September 12) with a simultaneous Advanced Track that day for non-beginners, followed by four more days of tutorials, presentations, and receptions. There’s also an area set up for vendor booths and demos. The conference organizers include the GNU Radio Foundation and the University of Colorado at Boulder.
The GRCon Diamond sponsor for event is Ettus Research, a National Instruments company. Ettus makes the dc-to-6GHz USRP (Universal Software Radio Peripheral) line of SDR platforms including the:
Why use FPGAs for SDR? Because SDR requires a lot of fast DSP and FPGAs have that.
For more information about Ettus Research USRPs, see:
By Adam Taylor
So far on this journey, we have touched HLS (high-level synthesis) when we looked at the Xilinx SDSoC Development Environment. However, we have not directly used the standalone Vivado HLS, which comes as standard as part of the Vivado HLx Design Suite. It is time to correct this oversight. Over the next few blogs, I want to introduce the basics of HLS and then introduce how we can best use HLS for creating embedded-vision designs.
So what is wrong with plain, old HDL design? Why do we need a whole new design-capture paradigm? As devices get larger and as designs get more complicated, we need new to use new development methodologies to achieve the required results while keeping development time reasonable. This has always been the case for Programmable Logic (see the figure below).
Overview of Programmable Logic Development over the last 40 plus years
As the marketers like to say, HLS raises the level of design abstraction and that higher level allows us to create complex designs more quickly. The marketers like to say this, and in this instance they happen to be correct. HLS has been on the verge of breaking into mainstream development for a number of years. (Certainly I remember a demo when I was a graduate in 2000 of HandelC.) However, recent development-tool introductions including the Vivado HLx releases have pushed HLS firmly into mainstream design. I now use it a lot and like it.
That said there is still a place for HDL modules where handcrafted hardware design is needed to achieve maximum performance. The key is knowing what tool we as engineers should use for the task at hand so that we can deliver designs while meeting quality, time and budget goals.
The HLS environment within Vivado HLx should be very familiar to you if you are developing with the Xilinx Zynq SoC or Zynq UltraScale+ MPSoC using SDK or SDSoC. Vivado HLx and SDSoC are both based on the Eclipse platform.
We can use Vivado HLS to directly generate HDL from C, C++, or SystemC descriptions. The code for the function we want to compile with HLS must stand alone and must be contained within its own source file. (We will examine more HLS constraints in future blogs). Another big advantage of using HLS is that you also define the test bench in C, which allows for much quicker verification. What is really cool is we can use this high-level test bench to verify the HDL module compiled from the C/C++/SystemC code.
The Vivado HLS menu bar allows us to implement all of the steps we need. The menu bar shows C Simulation, Synthesis, Co-Simulation, Export RTL, Open Report and View Waveform—roughly the order in which they are used.
Vivado HLS menu bar
The Vivado HLS design flow is very simple, as we will see with a simple example (which adds two integers together). This flow is outlined below:
Source files and test bench files within Vivado HLS
C Simulation and Reported Results in the Console
HLS Synthesis Report & Output files
Report Utilization Estimate
Co-simulation dialog box allowing us to select Verilog or VHDL HDL
Screen Shot of Co-simulation waveform results
RTL Export Dialog
Overall the Vivado HLS flow is very simple and tightly integrated. However there are other areas we need to look at to use it effectively, which we will do over the next few blogs.
If you want to run through the example with the files from GitHub, you can establish the project going through the screen shots below.
Code is available on Github as always.
If you want E book or hardback versions of previous MicroZed chronicle blogs, you can get them below.
All of Adam Taylor’s MicroZed Chronicles are cataloged here.
Digilogic Systems in Hyderabad has developed a passive aircraft-detecting radar system using National Instruments’ (NI’s) FPGA-based FlexRIO processing modules, LabVIEW System Design Software, LabVIEW Real-Time, and LabVIEW FPGA. In the ELINT (electronic intelligence) world, this passive radar system falls into the ESM (electronic support measures) category. An adversary cannot detect passive radar because it’s, er, passive. The system receives electromagnetic signals emanating from the targets and no additional electromagnetic energy is injected into the monitored space. Digilogic’s passive radar system consists of five widely spaced receivers communicating with a central ESM Processor over point-to-point Gigabit Ethernet or microwave links. Here’s a block diagram:
Digilogic Passive Radar ESM System
The ESM processor consists of three NI PXIe-7975R PXIe FlexRIO FPGA Modules in an NI PXIe-1075 18-slot PXI chassis. There’s a Xilinx Kintex-7 FPGA in each FlexRIO module. The first FlexRIO module segregates targets. Data simultaneously streams to the other two FlexRIO modules using a peer-to-peer connection and those two FlexRIO modules implement TDOA (time difference of arrival) localization algorithms. The passive radar segregation and localization algorithms run on FPGAs because software-programmed processors are not fast enough. According to Digilogic, “Implementing the FPGA programming on VHDL/Verilog and custom hardware for the complexity of the application we were addressing would have taken us more than two years. LabVIEW FPGA made our FPGA programming both easy and faster, and we realized the complete system, from concept to predeployment, within eight months.”
Note: This project was a finalist in the Wireless and Mobile Communications category at this month’s NI Engineering Impact Awards during NI Week held in Austin, Texas. You can read more details in the full case study here.
Here’s a large-scale problem literally too hot to handle:
Global steelmaker and mining company ArcelorMittal has a 24/7 production line that continuously welds steel sheets unwound from giant coils in real time. If any part of the continuous weld tears due to a weak spot, the line quickly shuts down. If the tear occurs in an oven, there’s a 4-day cool-down period before the problem can be addressed. The cost of that delay is about €1 million in lost production, repair, and downtime. Something best avoided even if it only happens once or twice a year.
ArcelorMittal asked V2i, a Belgian engineering company specializing in structural dynamics, to address this production problem. V2i developed a reliable, autonomous system that detects low-quality welds and quickly warns the operator so there’s sufficient time to restart the welding process before the weld tears.
The system needed to monitor several real-time parameters including steel hardness, thickness, welding speed, and welding current. In the final design, a non-contact profilometer measures the weld shape precisely and quickly; a pyrometer measures the weld bead’s temperature; and several more sensors monitor parameters including pressures, current, voltage, positions, and speed. V2i used a National Instruments (NI) CompactRIO cRIO-9038 controller with appropriate snap-in I/O cards to collect and condition all of the sensor signals.
The NI cRIO-9038 controller—programmed with a combination of NI's LabVIEW System Design Software, LabVIEW Real-Time, and LabVIEW FPGA—has sufficient processing power to handle the complex tasks of combining all of the real-time sensor data and producing a go/no-go status display while ongoing result data feeds ArcelorMittal’s online data base via Ethernet. Processing power inside of the cRIO-9038 is provided by a combination of a dual-core, 1.33GHz Intel Atom processor and a Xilinx Kintex-7 160T FPGA.
Here’s a video of the welder in action:
Note: This project won the 2016 Intel Internet of Things Award and was a finalist in the Industrial Machinery and Control category at this month’s NI Engineering Impact Awards during NI Week held in Austin, Texas. You can read more details in the full case study here.
The University of Adelaide and Flinders University, both based in South Australia, jointly developed three testing robots with 6 DOF (degrees of freedom) capable of applying a variety of dynamic, 3D mechanical tests on loads as massive as 4 tonnes. The initial, smaller robotic platform was designed for biomechanics research with an absolute accuracy of 10µm and a load capacity of 1 tonne. Here’s a photo of that robotic test platform:
Based on the results of that first design, the university teams then developed two larger platforms with 4-tonne capacities and 1mm of absolute positioning accuracy based on hydraulic actuators.
One of two hydraulically-actuated 6 DOF robotic test platforms developed by the University of Adelaide and Flinders University
The robotic test systems are based on Stewart platforms with six mounted linear actuators driving a test baseplate. The actuators are controlled by a variety of equipment from National Instruments (NI) including several generations of PXI controllers, RIO (reconfigurable I/O) modules, and DAQ (data acquisition) devices. The designs use several NI controllers including the PXI-8106 RT, PXIe-8133, and PXIe-8135 to execute the higher-level control algorithms (trajectory generation, kinematics computation, state-machine control, safety-critical control, and data acquisition) while Xilinx All Programmable devices in various NI RIO products run six parallel real-time PID loops that control the movement of each robot’s six Stewart-platform legs.
All three robots are programmed with NI’s LabVIEW System Design Software, LabVIEW Real-Time, and LabVIEW FPGA, which specifically programs the Xilinx FPGAs or the Zynq-7000 SoC in the various NI RIO modules. Basing the robots’ electronic controls on NI’s hardware and LabVIEW software significantly boosted code reuse among the three robotic designs.
Here’s a short video of the robots in action conducting various mechanical tests:
Note: This project was a finalist in the Industrial Machinery and Control category at this month’s NI Engineering Impact Awards during NI Week held in Austin, Texas. You can read more details in the full case study here.
Sagemcom Technology NRJ needed to increase demand-based production volume and improve first-pass yield on an energy meter manufacturing line in Tunisia while maintaining high product quality and 99% up time on the test line. These meters are used to create Smart Grids. The existing meter test system, based on a Windows PC, was not up to the job so Sagemcom Technology turned to the National Instruments (NI) CompactRIO cRIO-9030 controller programmed with NI’s LabVIEW System Design Software. The controller’s integrated Xilinx Kintex-7 70T FPGA, which provided the real-time control and high-speed image processing required by the improved production-test program, can be programmed with NI’s LabVIEW FPGA and FPGA-based LabVIEW Vision Development Module. The result: a 38% improvement in test time, 50% cost reduction for the controller, 17% improvement in productivity, and 70% LabVIEW code reuse. Those are truly excellent numbers.
Testing takes place in a 4-station robotic testing cell with two robots to move the meters from one test station to the next with each station conducting different tests including a functional test, a machine-vision test to verify proper operation of the energy meter’s LCD, a 3.2KV HiPot test to verify fault immunity, and a test of the meter’s power-line communications capability. The NI CompactRIO controls the robot arm, runs the tests, and reports the results to a server via Ethernet.
Here’s a nicely done, minute-long video of the robotic test system in action:
Note: This project won in the Industrial Machinery and Control category at this month’s NI Engineering Impact Awards during NI Week held in Austin, Texas. You can read more details in the full case study here.
Xilinx will be attending this year’s Sparkfun AVC (Autonomous Vehicle Competition) in Colorado on September 17. Haven’t heard about the Sparkfun AVC? Incredibly, this is its eighth year and there are four different competitions this year:
Sparkfun’s AVC is taking place in the Sparkfun parking lot. Sparkfun is located in beautiful Niwot, Colorado. Where’s that? On the Diagonal halfway between Boulder and Longmont, of course.
Haven’t heard of Sparkfun? They’re an online electronics retailer at the epicenter of the maker movement. Sparkfun’s Web site is chock full of tutorials and just-plain-weird videos for all experience levels from beginner to engineer. I’m a regular viewer of the company’s Friday new-product videos. Also a long-time customer.
Xilinx will be exhibiting an embedded-vision demo in Maker’s Alley tent at AVC this year because Xilinx All Programmable devices like the Zynq-7000 SoC and Zynq UltraScale+ MPSoC give you a real competitive advantage when developing a quick, responsive autonomous vehicle.
If you are entering this year’s AVC and are using Xilinx All Programmable devices in your vehicle, please let me know in the comments below or come to see us in the tent at the event. We want to help make you famous for your effort!
Here’s an AVC video from Sparkfun to give you a preview of the AVC:
Here's the PRS video:
And here's the Robot Combat video:
Jaguar Land Rover (JLR) is developing electric vehicles and the company must ensure that the driving performance and range that customers expect from JLR brand vehicles is present even if the gasoline isn’t. That’s why JLR engineers spend a lot of time developing battery-management systems (BMS): to get the most life, the longest range, and maximum safety from lithium-ion vehicle batteries. Developing firmware for such battery-management systems to prevent over-voltage, over-current, over-temperature, and other extreme conditions for the battery packs is a tough job. If the JLR engineers worked with actual lithium-ion batteries during early development, they’d be dealing with a variety of problems including slow charge times, cell failures, and safety issues. Instead, they use a BMS HIL (hardware in the loop) Test System real-time battery pack simulator from Bloomy Controls to circumvent these problems during BMS development.
The Bloomy BMS HIL Test System simultaneously simulates 24 individual lithium-ion cells and includes real-time instrumentation so that the engineers can evaluate the state of the simulated battery during testing. Another key feature of using the BMS test platform instead of actual lithium-ion batteries is that the tests become deterministic. The system can recreate repeatable fault scenarios including over-voltage, under-voltage, over-current, over-temperature, and cell imbalances.
Bloomy BMS HIL Test System Control Panel
Bloomy’s BMS HIL Test System consists of a PXI chassis from National Instruments (NI) and a controller running NI’s LabVIEW Real-Time OS and VeriStand software. Along with PXI I/O and bus communication modules, the PXI chassis connects to two Bloomy Battery Simulator 1200 units based on NI sbRIO single-board controllers. The Bloomy Battery Simulator 1200’s sbRIO quickly and deterministically controls the output voltages at the end of the cable harness where it connects to the BMS, ensuring high accuracy and good response times. Bloomy developed the cell models using The MathWorks’ Simulink development environment.
JLR has used the Bloomy BMS HIL Test System to validate four different BMS hardware architectures including both centralized and master-slave topologies, with substantially different high-voltage architectures. JLR is able to quickly test different xEV programs during the same day using the test system’s standard connector interface.
Bloomy is an NI Embedded Control Specialty Alliance Partner and has developed substantial expertise in working with NI’s FPGA-based hardware including NI’s CompactRIO, sbRIO, sbRIO SOM (system on module), and FlexRIO. All of these NI hardware products are based on Xilinx All Programmable FPGAs or the Zynq-7000 SoC. All are programmable with NI’s LabVIEW and LabVIEW FPGA system-development tools.
Note: This project was a finalist in the Transportation and Heavy Equipment category at this month’s NI Engineering Impact Awards during NI Week held in Austin, Texas. You can read more details in the full case study here.
MaXentric Technologies develops cutting-edge wireless products ranging from simple, low-cost millimeter broadband wireless transceivers, to passive RFID readers, to a high-efficiency envelope-tracking power amplifiers (ETPAs) for the military defense and telecommunications/broadcast commercial markets. The cutting edge for RF power amplifiers is all about linearity without wasting power. ETPAs address the increasingly crowded frequency spectrum and increasing demand for higher data rates through non-constant RF envelopes and high peak-to-average power ratios (PAPR). Simply put, ET technology allows the operator to use only as much power as necessary to provide the amplified output.
Communication systems using ETPAs require high linearity to minimize signal distortion, reduce bit error rate, improve spectral efficiency, and reduce adjacent channel interference. These requirements gave MaXentric Technologies many parameters to measure and optimize during ETPA development and the company required an automated testbench to test designs for various target applications that use the LTE and 5G wireless bands, GPS signaling, and military RF. The testbench had to be able to test any ETPA for any application, and all on the same day with simply a click to change the RF frequency.
The company selected a variety of PXI instruments from National Instruments (NI) to build the ETPA testbench and uses NI’s LabVIEW System Development Software to program the testbench hardware. An NI PXI-5646R Vector Signal Transceiver (VST) serves as the testbench’s RF signal generator and RF feedback analyzer, which permits the testbench to quickly switch RF frequencies from LTE, to GPS, to military. (NI’s 1st-generation PXI-5646R VST was based on a Xilinx virtex-6 FPGA. Its just-introduced 2nd-generation PXIe-5840 Vector Signal Transceiver is based on a Xilinx Virtex-7 690T FPGA, see “NI launches 2nd-Gen 6.5GHz Vector Signal Transceiver with 5x the instantaneous bandwidth, FPGA programmability.”) An NI PXIe-5451 Arbitrary Waveform Generator initially generated the envelope waveforms for the PA. The NI PXIe-5451 also incorporates an FPGA for digital flatness correction and inline waveform processing.
Here’s a semi-block diagram of the ETPA testbench:
MaXentric Technologies ETPA Testbench
MaXentric used this testbench to tune and optimize an LTE Band 1 (2.14 GHz) ETPA based on the company’s MaXEA 1.0 Integrated Envelope Modulator and a GaN RF transistor used as the power amplifier. The time alignment of the envelope amplitude and RF input signals to the ETPA is critical for optimized, efficient RF performance. Time misalignment between these two signals distorts the output signal, degrades ACPR (adjacent channel power ratio), and reduces efficiency. MaXentric could literally see the improvement or degradation in ETPA linearity and efficiency as the testbench altered the alignment between the RF signal and the envelope supply in real time.
The testbench-enabled tuning greatly improved the LTE ETPA’s linearity and efficiency. Before tuning, the GaN ETPA achieved 6W of output power with 11.5dB of gain and 53% power added efficiency (PAE). After optimization, the ETPA’s output power and gain remained the same with a slight improvement in PAE to 54.6% and with better than -45dBc ACPR—a highly desirable result.
According to MaXentric, using NI’s LabVIEW cut the development time for this ETPA testbench from nearly a year to two months.
Note: This project won in the Electronics and Semiconductor category at this month’s NI Engineering Impact Awards. You can read more details in the full case study here.
Imagine putting a 30cm cube of rock under 70 MPs (MEGAPascals!) of pressure (10,000 psi) at 95°C to induce internal fracturing while imaging the resulting fractures in the rock in 3D and in real time using 60 ultrasonic transducers attached to the rock faces. Sounds challenging, but why do that? To study the geophysical effects of sequestering greenhouse gases underground. That’s the mission of a unique system called the high-channel-count Microseismic Data Acquisition System (MDAS) developed by ICON Technologies for the National Geosequestration Laboratory in Australia.
MDAS is based on custom versions of Altcom Pty Ltd’s and VSProwess’ commercial 3D seismic analysis and display engines, tailored for the National Instruments (NI) LabVIEW System Design Software and LabVIEW FPGA. The system hardware is based on NI’s PXIe chassis with fifteen PXIe-7962R FlexRIO I/O cards, each equipped with NI 5734 16-Bit, 120 MS/s Digitizer Adapter Modules, that provide 60 channels of 16-bit ultrasonic signal acquisition at up to 120 MSamples/sec; a PXIe-6672 timing and synchronization card; and a PXI-5412 Arbitrary Waveform Generator card for generating user-configurable ultrasonic excitation pulses. Here’s a block diagram of the MDAS:
MDAS Ultrasonic Imaging System Block Diagram
Microseismic studies of geologic substrate cracking dynamics helps improve our understanding of many critically important processes that occur on global scales—such as seismic events that precede and follow major earthquakes—and on local scales—such as fluid-transfer mechanisms associated with the geosequestration of greenhouse gases. The mathematics of analyzing microseismic ultrasonic signals derives directly from the mathematics of large-scale seismic signal analysis, but at 1.5KHz to 500KHz instead of 40 Hz.
The MDAS system starts with the 60 ultrasonic transducers uniformly distributed over the six faces of a cubic rock sample measuring as much as 30 cm per side. The rock samples are held at pressures as high as 70MPa (10,000 psi), and temperatures as high as 95°C in a “true triaxial pressured-stress cell” (TTSC). These sensors generate 60 channels of 16-bit ultrasonic data at sample rates as high as 120MSamples/sec (10 to 20MSamples/sec nominal).
Initially, the MDAS’ 60 ultrasonic sensors passively listen for a fracture to start in the rock sample. Acoustic detection of a trigger from any of the 60 transducers initiates a synchronized data-acquisition ballet that captures pre- and post-trigger data from all transducers. The dance starts when the MDAS’ Analysis Engine detects a correlated event pattern indicating the initial formation and propagation of a significant crack. The MDAS immediately converts one of the transducers into an active transmitter while the other 59 transducers continue to act as receivers. Over the next two seconds, each of the 60 transducers becomes a transmitter in turn while the other transducers listen. In this way, the MDAS generates a 3D ultrasonic map of the rock cube’s internal structure and this process continues for as long at the crack continues to propagate.
This process works like an actively “pinging” submarine sonar and generates a map that looks like this:
MDAS 3D Map
Successful application of the time-of-flight technique requires tight synchronization of all data-acquisition channels. By using the NI PXIe-6672 to generate a precise clock that drives all fifteen data-acquisition cards and taking advantage of oversampling (up to 120 MSamples/sec) and the high speed of the FPGA-based PXIe-7962R/NI 5734 FlexRIO I/O cards results in a synchronized acquisition window of better than 10nsec for all 60 channels. The NI FlexRIO cards used in the MDAS are based on Xilinx Virtex-5 SX50T FPGAs.
MDAS represents the cutting edge of microseismic research.
Note: This project won in the Energy category at this month’s NI Engineering Impact Awards. You can read more details in the full case study here.
The world’s first and only commercial supersonic airplanes, the Concorde fleet, was removed from service after a July 2000 disaster caused by runway debris that punctured the Concorde’s tires during takeoff, which created shrapnel that penetrated the plane’s fuel tanks, ignited a fire, and ultimately led to the aircraft crashing into a small hotel on the outskirts of Paris. Since then, runway debris has been a major cause of concern for airports around the world. Detecting such debris has been a research project for Japan’s Electric Navigation Research Institute (ENRI), the national research agency that develops civil technologies for aviation surveillance and communication, air traffic safety, and efficient operation of air traffic routes. ENRI has been developing millimeter-wave radar to detect FODs (foreign object debris) on runways and has developed a working prototype system based on National Instruments’ (NI’s) PXIe platform and FlexRIO FPGA-based processing.
The ENRI-developed millimeter-wave radar system consists of beam-scanning antennas, millimeter-wave transmitting and receiving circuits, signal generation, processing circuits, and synchronization and control circuits. The main areas of research for the FOD-detection system are the 96GHz millimeter-wave front-end circuits, the receiving signal-processing circuits, and the synchronization circuits. The challenge was to create a system what could detect sub-cm objects on an aircraft runway.
Prototype FOD Detection System for Airports
The system had to be tested at a working airport so the team had a very limited amount of time for software development—about one month—because of radio licensing restrictions. As a result, the team chose to use National Instruments’ LabVIEW System Development Software to develop the code—and did it in 10% of the normal development time because of the fast-prototyping abilities of LabVIEW’s graphical programming language. Because they needed to perform real-time 8192-point FFTs (10,000 FFTs/sec), the design used NI’s LabVIEW FPGA software to program the FFTs into the NI PXIe-7975R FlexRIO FPGA module, which is based on a Xilinx Kintex-7 410T FPGA. Here’s a block diagram of the radar-processing system:
FOD-Detection Processing System Block Diagram
The code worked; the prototype design was successful; and the team is now taking advantage of LabVIEW’s easy-to-reuse code develop a helicopter collision-avoidance system based on NI’s Zynq-based sbRIO products.
By Adam Taylor
We have demonstrated how we can quickly and easily get a get an image tracking algorithm up and running on the Zynq-7000 SoC over the last two blogs (parts 1 and 2 are here). We also introduced some embedded vision techniques including thresholding and morphological operations.
However embedded vision covers a very wide area, so I want to take some time in this blog to look at some other embedded-vision applications and how we can implement the relevant algorithms. Embedded-vision algorithms break down into the following high-level categories:
Embedded Vision Categories
These processing techniques split further into tasks that process and extract information from the image and tasks that use the results of these operations for analysis and decision making.
One of the most commonly used embedded-vision techniques is applying filters to the image. Because information within an image resides in the spatial domain and not the frequency domain, image-processing filters typically use convolution filters. That is, we convolve the image with a 2D filter kernel to obtain the desired response.
As with 1D convolution, we must consider the filter’s impulse response. This is called the point-spread function (PSF) in image-processing applications. To control the filter’s function, we define a custom PSF for each function just as we would define different impulse responses for signal-processing filters (e.g. high-pass, low-pass, etc).
Most commonly, we implement the filter kernel as a 2D matrix, which we apply to each pixel in the image. Within the implementation, we need to buffer lines so that we can slide the filter across the image.
Example Image and filter kernel applied on pixel-by-pixel basis on the Lena image
Within these filter kernels, we can define PSF’s for:
These Imaging filters are defined as linear filters. There is another class of filters called non-linear filters that include filters like the median-order filter.
One of the major differences between linear and non-linear filters is their edge-preservation capability. Linear filters can produce blurred edges. Non-linear filters tend to preserve edges.
Example of original Lena image (left) and edge enhancement (Right)
Depending upon the image, we may need to adjust the contrast to extract the most information from in the image. Each pixel has a value within an image. These values will be close together in a low-contrast image, which makes it hard to distinguish between pixels. Contrast enhancement widens the distribution of pixel values to make subsequent image-processing algorithms easier to implement.
The contrast of an image can be determined by its histogram, which shows the distribution of pixel values. A low-contrast image will demonstrate a tight grouping of pixel values in the histogram. A high-contrast image will show a wide spread of values. Commonly used algorithms for contrast enhancement include contrast stretching, which can use linear or non-linear approaches or histogram equalization.
Histogram showing contrast of the Lena image
Many embedded-vision applications require the detection of edges. While edges are easy for the human eye to detect, they can require significant processing in the embedded-vision world. First we must consider the three general types of edges found in images:
Typically, the resultant images from edge detection are binary in nature (e.g. white or black). There are of course a number of different algorithms to detect edges in our images. Three of the most common are:
Edge-Detection Algorithms Clockwise: Original, Laplacian of Gaussian, Canny, and Sobel
Both Canny and LoG operations are often called advanced edge-detection algorithms because they calculate:
Having introduced some even embedded-vision algorithms, we are now going to start looking at how we can use Vivado HLS (high-level synthesis) to implement these algorithms within our embedded-vision application.
Code is available on Github as always.
If you want E book or hardback versions of previous MicroZed chronicle blogs, you can get them below.
All of Adam Taylor’s MicroZed Chronicles are cataloged here.
Here’s a mental jolt. Who says that battery packs based on one battery chemistry are optimally efficient or reliable for automotive systems? That’s the question researchers in the Power and Electronics and Electric Drives Lab at the Center for Sustainable Electrical Energy Systems, located at the University of Wisconsin-Milwaukee. To answer the question, researchers needed to study battery combinations under long-term cycling conditions. For this particular experiment, the research team harnessed seven lithium-ion and two lead-acid cells, effectively creating a hybrid battery pack. They tested these packs over 1000 charge/discharge cycles at a 0.3C rate. After every 50 charge/discharge cycles, they needed to perform a capacity check to see if there’d been any degradation.
Manually rewiring the test setup from a charge to a discharge configuration was taking an hour and had to be performed every three hours, so the team desperately needed to automate the process. They implemented an automated test system using a CompactRIO controller from National Instruments (NI) along with NI’s LabVIEW System Design Software, LabVIEW Real-Time, and LabVIEW FPGA. (NI’s CompactRIO and single-board RIO controller products incorporate a range of Xilinx All Programmable devices including Kintex-7 and Artix-7 FPGAs and Zynq-7000 SoCs.) This data-acquisition system was able to take 20 measurements (9 voltage, 2 current, and 9 temperature measurements) every second with good time correlation among the measurements. The Xilinx 7 series FPGA in the CompactRIO controller allowed the team to meet its acquisition and computational performance requirements.
Here’s a schematic of the test system:
Hybrid Battery Cycle Life Automation System
The manual test system was taking 16 months to complete each series of tests on one battery pack. The automated system operated continuously and completed the task in 9 months.
However, test efficiency was not the sole benefit of the automated system. The batteries are flammable and there’s always a chance of a spark when connecting and disconnecting batteries. The automated system eliminated this hazard and allowed the team to safely lock the batteries being tested in a safety cabinet.
IP-Maker demonstrated a high-speed, FPGA-accelerated application based on the company’s NVMe IP yesterday at the Flash Memory Summit in Santa Clara, California. The demo reached 200,000 TPM (transactions/minute) running an OLTP application using SDQ Server 2016 and HammerDB with a TPC-C profile. The database file is stored on a SATA SSD while the transaction logs are filed in an NVMe-connected RAM to break the application’s log I/O bottleneck. That’s “great performance on such a low-cost server” according to Mickael Guyard, Marketing Director at IP-Maker.
The NVMe RAM is based on IP-Maker’s NVMe IP core instantiated in a Xilinx Virtex-7 FPGA. Here’s a block diagram of IP-Maker’s NVMe RAM Drive used in the demo at the Flash Memory Summit:
Contact IP-Maker directly for more information about the NVMe IP core.
Kazan Networks was demonstrating its Native NVMe over Fabrics (NVMf) SSD at today’s Flash Memory Summit in Santa Clara, California. The NVMf standard is hot off the press—it only appeared two months ago—so Kazan employed a Xilinx Kintex UltraScale FPGA to quickly implement an NVMf SSD controller that accepts two M2-format SSDs. Here’s the photo from yesterday’s press release showing the two M.2 SSD slots on the board:
Kazan Networks Native NVMe over Fabrics SSD controller
And here’s a photo I shot of the board at today’s event showing the FPGA side of the board:
That big silver chip in the board’s center is the Kintex UltraScale FPGA.
The NVMf specification allows you to ship NVMe-protocol communications over Ethernet. Why do that? NVMf allows you to physically separate the server farm from the storage racks for any-to-any connections using standard Ethernet communications between servers and storage while still getting the desired data center performance.
Kazan has realized at least three benefits by using a Kintex UltraScale FPGA to implement its Native NVMf SSD controller. First: short time to market. This board is up and running only two months after the standard was published. Second, the Kintex UltraScale FPGA delivers the high-speed, 1μsec latency through the controller needed to achieve Kazan’s performance goals. A software-based implementation running on an embedded processor would not have delivered anything close to this speed. Third, everything Kazan learns from this initial implementation can be turned around and fed back into the FPGA configuration with essentially zero turnaround time. This is a huge competitive advantage for the company.
Spotted at today’s Flash Memory Summit in the Santa Clara Convention Center: a high-speed NVMe-to-NVMe bridge in the Intelliprop booth.
Intelliprop NVMe-to-NVMe bridge implemented in a Kintex UltraScale FPGA
Intelliprop has combined its NVMe IPC-NV163-DT Device and IPC-NV164-HI Host IP cores running on a Kintex UltraScale KCU105 FPGA Eval Kit to create the bridge. I asked Caleb Whitehead, a digital design engineer at Intelliprop, why would you need that? You could use it as an NVMe aggregator, for example, to connect multiple NVMe devices to one NVMe port, he answered. Another reason: it’s a good demo Intelliprop’s NVMe Device and Host IP cores.
The Netlist HybriDIMM targets today’s biggest bottleneck in server-based computing—storage access latency—by bonding as much as 512Gbytes of Flash memory to 8 or 16Gbytes of DDR4 SDRAM right on the DRAM DIMM. That’s as close as you can get to the CPU without actually climbing onto the silicon and cozying up to the processor.
DRAM+Flash DIMMs have been around for a while. What differentiates the Netlist HybriDIMM is that the memory module incorporates considerable processing power to implement Netlist’s proprietary PreSight Technology, which provides predictive memory management that moves data from Flash memory to the SDRAM before it’s actually needed. According to Netlist, its HybriDIMM provides a 1000x performance boost relative to a PCIe SSD.
Netlist HybriDIMM bonds Flash memory to DDR4 SDRAM in one module
How does Netlist’s PreSight technology do this? Here’s a direct quote from Netlist’s Web site:
“PreSight Technology is the breakthrough that allows for data that lives on a slower media such as NAND to coexist on the memory channel without breaking the deterministic nature of the memory channel. It achieves by prefetching data into DRAM before an application needs it. If for some reason it can’t do this before the applications starts to read the data, it ensures the memory channels integrity is maintained while moving the data on DIMM.”
Whatever’s going on in the Netlist HybriDIMM, you should know that there are two Xilinx Kintex KU040 UltraScale FPGAs running at DDR4 memory speeds inside of the module and working that PreSight magic.
A couple of months ago I wrote about Biostage, a company that employs regenerative biotech to grow implantable human organs on a biocompatible polymer scaffold. The company currently focuses on growing esophagus replacements for dealing with esophageal, bronchial, and tracheal cancers and trachea trauma. The current treatments use most of the patient’s stomach or a section of the small intestine as replacement parts. Neither sound like very good alternatives. The current mortality rate is as high as 30% and there are potential, serious complications in many of the cases so a direct replacement organ grown from the patient’s own stem cells sounds pretty darn good.
The replacement organs are incubated in bioreactors, which are small covered trays containing a nutrient bath. The organ on a polymer scaffold rotates within the bioreactor; half of it is in and half out of the nutrient bath at any one time. A controller built using a National Instruments sbRIO controller running NI’s Linux Real-Time OS and programmed with LabVIEW controls the rotation rate and the pressure, temperature, and pH in the bioreactor. Biostage started development of the bioreactor controller in 2012 starting with an NI CompactRIO chassis and has gone through several generations of NI’s sbRIO platform and they currently use NI’s sbRIO-9627 controller based on the Zynq-7000 SoC. One Biostage controller can operate four bioreactors.
Currently, the company has been implanting these bioreactor-grown organ replacements in large-animal studies and it expects approval to start the first human clinical trials by the end of 2016.
This project won in its category, “Advanced Research” at last week’s NI Engineering Impact Awards and it won the “Humanitarian Award” from NI as well.
Here’s NI’s Vice President of Product Marketing Dave Wilson introducing Biostage and showing the company’s work at NI Week:
The krtkl Snickerdoodle is a low-cost dev board based on the Xilinx Zynq-7000 All Programmable SoC that makes maximum use of the Zynq SoC’s I/O. A company in Montreal named JBLopen has just started publishing a series of blog posts exploring the use of the Snickerdoodle and the latest post discusses software needed to use the on-board SD card for embedded file storage with the Micrium µC/OS RTOS.
For the previous post in this series, see “Zynq-based snickerdoodle says "Hello World" using Micrium’s µC/OS.”
For more information about the krtkl Snickerdoodle board, see “$55 Zynq-based Wireless Snickerdoodle single-board computer with WiFi, Bluetooth launched today on CrowdSupply” and “$55, Zynq-based, WiFi-enabled Snickerdoodle makes MAKE:’s Maker’s Guide to Boards.”
Neutrinos are approximately 500,000x less massive than electrons. Put another way, it’s really, really, really difficult to measure a neutrino’s physical properties. And yet that’s been the mission of Taiwan’s TEXONO (Taiwan EXperiment On NeutrinO) Collaboration for almost two decades. The team’s preliminary goal was to precisely measure neutrino mass. TEXONO was established in 1997 as Taiwan’s first high-energy physics experiment team composed of researchers from universities in Taiwan and mainland China. International partners from India and Turkey have subsequently joined the research program.
How do you measure a neutrino’s physical properties? Well, first you locate your experiment where you know there are neutrinos present—like inside of a nuclear reactor’s core. The core of a nuclear power plant releases huge amounts of neutrinos through beta decay. The intense neutrino flux is a free byproduct of nuclear power generation. TEXONO chose Taiwan’s 1970MW Kuo-Sheng nuclear power station on the northern tip of Taiwan near Taipei as its experimental site and located their experiment 28m from the #1 reactor core.
Reactor-based neutrino experiments before TEXONO were based on measurements of high-energy events. The TEXONO program elected to pioneer the previously unexplored low-energy regime, which had implications for the experimental apparatus including the sensors, sensor-conditioning electronics, and the data-acquisition systems.
A cryo-cooled, high-purity germanium (HPGe) sensor detects low-energy neutrinos in the TEXONO experiments. HPGe detectors have super-low background noise and excellent energy resolution. A sodium iodine scintillation detector surrounds the germanium detector. Scintillators absorb the energy from ionizing radiation and re-emit the energy as photons that can be detected by a photomultiplier tube. In this case, the scintillator is used to detect and absorb non-neutrino ambient radiation that gets past the shielding before it reaches the germanium detector. In this way, events caused by cosmic rays, for example, can be detected and ignored. The germanium sensor and scintillator are surrounded by 1m of copper, lead, and polymer shielding to block gamma radiation and neutrons.
Traditionally, particle researchers have used analog modules to copy signals, generate threshold triggers, and correlate timing between signals. You can use these analog modules for big signals but TEXONO’s neutrino research focuses on extremely low-energy events, which means that only signals near thermal-noise levels are of interest. After dealing with the deficiencies of an analog-based data-acquisition for years, the TEXONO upgraded their system to a digital system based on National Instruments’ (NI’s) PXIe modules and chassis.
The TEXONO Analog Instrument rack, obsoleted by NI PXIe equipment
There are five output signals from the experiment’s main detector, two from sodium iodine detector, and one from each photomultiplier tube. Combined with other auxiliary signals, there are a total of 50 signals, which are captured by PXIe digitizers. The experimental apparatus has nearly 50 detectors, each with different trigger conditions. Very few of these signals have obvious trigger events; some are just slightly higher than threshold. Capturing all of the data from these sensors continuously would generate far too much data, so the experimental teams need triggers to initiate data recording.
The triggers require a real-time algorithm to determine if the small signal fluctuations represent a real physical event or were merely an artifact caused by electrical cable noise. A trigger condition exists if a signal exceed threshold for more than 140 nsec and the experiment uses the Xilinx FPGA in an NI FlexRIO PXIe module to detect a trigger event. When such a trigger event occurs—as many as 40 per second—the FPGA sends TTL trigger signals to activate the digitizers and start the signal recording. NI’s FPGA modules used in this experiment have 20nsec, 14-bit time and voltage resolution to monitor signals continuously and the FPGA can immediately execute complicated trigger algorithms (random trigger, reset inhibit, TDC, etc).
TEXONO PXIe-based instrumentation from National Instruments
TEXONO’s previous signal-discrimination system used older analog electronic modules and hundreds cables for generating triggers, resulting in a twisted mess of cabling and noise. The new experimental data-acquisition system based on NI’s FlexRIO and digitizer PXIe modules allow the experimenters to create software-programmed triggers with NI’s LabVIEW System Design Software and LabVIEW FPGA, resulting in better trigger discrimination. Triggers are easily modified by reprogramming the software to fit different tasks with no need to change cabling between experiments.
The TEXONO Collaboration was a finalist in the Advanced Research category during last week’s NI Engineering Impact Awards. In a case study on NI’s site, the TEXONO researchers note several key advantages that the FPGA-based FlexRIO PXIe module delivered:
RGB Spectrum in Alameda, California has been producing advanced video systems for thirty years and its mission is to create products that deliver “stunning, real-time video presentations in real time.” The company’s latest product, MediaWall V, is the culmination of three decades of video design experience. The product’s custom video processor handles 96M pixels at 60 frames/sec displayed over an array of multiple video panels. The one video processor handles switching, routing, and real-time scaling, taking video streams from any input and presenting the video on any output.
The company uses a broad range of Xilinx FPGAs across multiple device generations from Kintex-7 and Artix-7 through Spartan-6 and Virtex-5 devices. The company has used nearly 50 FGPAs in the MediaWall V design, distributed across five different subsystems. According to the company, FPGAs are absolutely central to the MediaWall V’s ability to smoothly move a video window from here to there on a video wall with user-defined borders, labels, and scaling.
Overall, Xilinx FPGAs perform numerous functions in RGB Spectrum’s MediaWall V Display Processor including:
There’s a new “Powered by Xilinx” video showing you this high-performance video product in action:
For more detailed information about this application of Xilinx All Programmable devices, see “Next-Generation MediaWall V 4K/UHD Display Processor puts any video on any wall-display video mosaic—using three FPGA generations.”
In May, a team of 5G engineers from the Universities of Bristol and Lund set a new world record for wireless spectrum efficiency—145.6 bps/Hz in a narrow 20MHz channel serving 22 clients—using a 5G Massive MIMO SDR system developed and prototyped with FPGA-based SDR equipment from National Instruments (NI). This result breaks the team’s own record set less than two months prior, which had been 79.4bps/Hz serving 12 clients. 5G system goals require this sort of extreme, breakthrough data-carrying efficiency to achieve 1000x more data bandwidth for customers.
128-Antenna 5G Massive MIMO System, with researchers Paul Harris and Siming Zhang
This latest 5G testbed uses a 128-antenna MIMO array (dubbed ARIES) coupled to 64 NI USRP RIO dual-channel SDR radios, which are based on Xilinx Kintex-7 325T FPGAs. The 64 USRP RIO units talk to multiple NI PXIe-7976R FlexRIO FPGA modules (based on Xilinx Kintex-7 410T FPGAs) plugged into NI PXIe chassis for further processing of the Massive MIMO signals. The USRP RIO units connect over high-speed PCIe cabling, through PCIe switch boxes, to NI PXIe-8384 interface modules plugged into the same chassis as the FlexRIO FPGA modules. (By my count, there are at least 68 Kintex-7 FPGAs in this system.) The entire system was developed with NI’s LabVIEW System Design Software. There’s a full description of the 5G Massive MIMO system here, a case study here, and here’s a system block diagram:
Massive MIMO SDR system developed by the Universities of Bristol and Lund using NI hardware and software
This project won big at last week’s NI Engineering Impact Awards. It won in its category, “Wireless and Mobile communications”; it won the “Powered by Xilinx” Award; it won the “Engineering Grand Challenges Award”; it won the “HPE Edgeline Big Analog Data Award”; and it won the “2016 Customer Application of the Year Award” from NI.
No surprise then that NI has introduced the world’s first Application Framework for Massive MIMO to speed innovation in 5G prototyping based on this groundbreaking research. You can scale 5G testbeds from 4 to 128 antennas using this NI 5G framework. More information is available in this 3-minute video:
For more information about NI’s USRP RIO SDR, see “Software-defined radio dev platform for 5G research handles MIMO, massive MIMO using Kintex-7 FPGA.”
To see additional uses for the NI PXIe-7976R FlexRIO FPGA modules, see “NI and Nokia Networks develop 10Gbps, FPGA-based mmWave SDR transceiver for 5G using new NI mmWave Transceiver System,” and “What do you need to build the world’s highest-performance, Real-Time Spectrum Analyzer? RADX says: NI, Xilinx, and Kintex-7 FPGAs.”
By Adam Taylor
As I wrapped up the last blog, I had introduced the concepts that we are going to be using to develop the algorithm.
We can put these concepts together very simply to create the object-tracking algorithm. To do this we will use the following algorithm:
What is very impressive is that regardless of our language of choice for implementation, we can implement this algorithm in very few lines of code.
We did not discuss the Gaussian blur last week. However the performance of many image-processing applications that detect objects or edges can be negatively affected by noise present within the grabbed frame. Blurring the image prior to further processing reduces noise within the frame and this technique is often used for image processing, particularly for edge-detection algorithms (e.g. the Laplacian of Gaussian edge-detection algorithm).
The sequence of frames shown below shows the results for each particular step in the above algorithm for points 2 through 6.
Frame converted to grey scale (2)
Grey scale image following blurring (3)
Absolute difference between reference and current frame (4)
Thresholded image (5)
Dilated image (6)
Once we have the binary image dilated, we run an OpenCV contour-finding algorithm on it. What are contours? They are curves joining all the points of an image with the same intensity or color. When we work with binary images, contours become the boundaries of the image. Within OpenCV, contour finding falls within the structure analysis and shape descriptors functions.
There are a few things we need to know when using the find contours function. The first is that the function will change the image passed to it, so it’s often good to use a copy of the image and not the original image. Secondly, the find-contours function will only detect the contours of a white object in a binary image, so we must be sure to threshold appropriately.
The contour operation provides a list of vectors for each of the detected objects in the binary image passed to it. As the scene changes and as objects enter the frame, the number of objects within the dilated image and hence the number of returned contours could be large. We will need to eliminate contours with a small area. If we fail to do this, the performance of our algorithm will needlessly degrade and we will detect irrelevant things like small wind-induced motion, as can be seen in the image below where the shrubbery is moving in the breeze. This movement unnecessarily clutters the final image.
Image with no discrimination on contours
To ensure that we only track the objects of interest such as moving cars and people and to blot out fluctuations due to the breeze, we can use an OpenCV function that calculates the area of the contour. If the contour is below a certain threshold, we discard it and waste no more processing resource on it.
If the contour does have an area that’s above our threshold, then we want to calculate and draw a box around the area of interest. Again we can do this using an OpenCV function that calculates the bounded rectangle for each of the remaining contour points in the frame. We can then use this information to draw the number of rectangles we require on the original image to identify the objects that have changed since we took the reference frame.
This is a very simple image-detection algorithm, but it is one we can implement with relative ease.
Next week we will look at a few more image-processing algorithms before we venture into how we can use OpenCV with the Xilinx Vivado HLS, the High-Level Synthesis environment. (Note: To watch a 15-minute video about using OpenCV libraries with Xilinx Vivado HLx to create Zynq-based implementations, click here.)
Code is available on Github as always.
If you want E book or hardback versions of previous MicroZed chronicle blogs, you can get them below.