01-20-2014 01:52 AM
Hi all, I have a video system which run on our custom board based on xc7z010-clg225 soc. The fpga logci path of the
video system mainly consists of a cmos image sensor which provide the video source, xsvi interface pcore, vidin2axi4s
pcore, AXI VDMA which send image data to DDR through the axi high performance slave port t. Then we transfer the
image data back to the pc and display it. It seems that at most of time, it woks well. But there are some shadow just
like the following screenshot show in it occasionally. I suspect there must be some noise on our design, but i am a
newer of the video system, and i still can not locate which place will be cause this problem up to now. Hope someone
can give me some suggestions. Any suggestion and comment would be very appreciate!
01-20-2014 11:20 AM
Since most of us have no clue what your picture should look like, could you post a picture without the "shadow" so we might have an idea what to look for?
01-20-2014 06:30 PM
Hi gabor, thanks for your reply! Ok, here is the normal state screenshot at most of time. The big distinguish of this two
screenshot is whether the multiple layer shadow occurs(just my expression, maybe not exactly). Why the abnormal
case occurs occasionally and how to fix it, which are the main issue troubled me. There are still some slightly
distinguish which may be affected by the light of my environment. Any suggestion and comment would be greate
01-20-2014 07:11 PM
What your first image looks like to me is not something I would associate with noise. Noise usually causes "salt and pepper" (high-frequency shot noise) or wavy horizontal bands (low frequency noise like power supply switching). What you have seems like the data bits have been systematically changed. It looks partly like "posterization" which is what you get when you reduce the number of grey-scale levels from 256 down to just a handful, and partly like a stuck bit, say the next to the high-order bit. I would suggest saving raw images (no filtering, no JPEG encoding) and using a program that lets you view the pixel values at the cursor. Irfanview can do this (you can download it free, just Google for it). Perhaps you'll see something systematic about the bad image. Those ragged lines around sections of the image could give a clue. See if they occur where the greyscale of the image is near a change in a high order bit, like multiples of 0x40.
01-22-2014 01:42 AM
Hi gabor, i am sorry that my response is late and thanks for your detailed answer. I was so dull that i think i can only
understand your words partly. You suppose should be right and the pixel value at several places may be changed
unexpectlly. I have saved the image data captured by the cmos and routed to DDR through logic path as a bmp format
file(just the raw image data). Through the IrfanView, i can see the component of the abnormal pixels value are close to
0xff, and please see my attach for a detail description.
In my consideration , the software code do not have the permission to modify the image data in ddr region, and I would
like to think this may be caused by the data path(to math the axi4-stream or axi bus, some bits has padded) in the fpga
logic path in our design. Do you think so? But it is strange that at most of time it works well. I am looking forward to
your reply. Very thanks!
01-22-2014 06:18 AM
It looks like the light or dark lines come where the image has a transition in the upper two bits, for example going from 0x3f to 0x40. This looks like a synchronization issue. If you have some point in the design where the bits are not coherent when they come through, you can have the effect of some bits coming from one pixel and other bits coming from the one next to it. Now suppose you took a normal image and shifted just bits 6 and 7 of the 8-bit data right by one pixel. Then if you had a nice grey ramp going 0x3D, 0x3E, 0x3F, 0x40, 0x41 it would translate to 0x3D, 0x3E, 0x3F, 0x00, 0x41, ...
This looks very much like your bad image.
You need to look where there can be some sort of data synchronization problem in the design. It could be right at the input from the sensor if this is a setup/hold issue. Or it could be internal to the design at a clock domain crossing.
01-22-2014 07:24 PM
Hi gabor, thanks for your detailed analysis. I ignore that there are still some pixels value has changed to 0x00 (black
points). But i still do not understand your words about "in the upper two bits" or "high-order bits". Do you mean that
the upper bits of a color compoent of a pixel ? Frankly speaking, i am not quite understand this words "where the image has a transition in the upper two bits, for example going from 0x3f to 0x40".
About "Now suppose you took a normal image and shifted just bits 6 and 7 of the 8-bit data right by one pixel. Then if you had a nice grey ramp going 0x3D, 0x3E, 0x3F, 0x40, 0x41 it would translate to 0x3D, 0x3E, 0x3F, 0x00, 0x41, ...
This looks very much like your bad image.", I try to understand a method to change the pixel value (bit[6:8] right shift to reduce the number of gray level, right?) to have the same visual effect with our bad image.
Yeah, i should point out that we use the lvds interface to receive data from Aptina mt9v032 image sensor, and we have
struggled to slove the data lock issue for a long time ever, and maybe this is one place we should verify again. Another
point is that the status registers of VDMA pcore complains some error stutas in our design. Here is the dump
Lvds monitor pcore reset ... LVDSMON Status Reg = 0x000C0C03 Lvds monitor pcore reset OK Vdma write frames to mem ... Vdma write frames to mem Over VDMA WRITE Channel Status: PARKPTR = 0x00000000 ------------------ S2MM_DMACR = 0x00010043 S2MM_DMASR = 0x00014810 S2MM_STRD_FRMDLY = 0x00000A00 S2MM_START_ADDR0 = 0x01000000 S2MM_START_ADDR1 = 0x0112C000 S2MM_START_ADDR2 = 0x01258000 S2MM_HSIZE = 0x00000A00 S2MM_VSIZE = 0x000001E0 ---------------- S2MM_HSIZE_STATUS= 0x00000000 S2MM_VSIZE_STATUS= 0x00000000 ----------------
From the pg020 datasheet, we can see the S2MM channel of VDAM complains that there are DMAIntErr, SOFLateErr,
Err_Irq (maybe the logic design of the upstream is not suitable to the requirement of VDMA, so he gives these error
I should have associated these error status with the bad image together. Thank you very much!
01-23-2014 07:35 AM - edited 01-23-2014 07:37 AM
I was talking about shifting the upper two bits of the pixel value (color component if you will) but not within the same pixel. Rather shifted in time to the next pixel in the image. Note that image data is presented serially starting from the left top pixel in the image, then moving to the right until the right top pixel. Next it goes to the left pixel of the second from the top row and so on. If you think of the spatial image as a time sequence, delaying a signal has the effect of moving it to the right in the image you see on the screen. A "ghost" in TV parlance is actually an echo of the original image that adds a right-shifted copy of the image (at reduced intensity) to the original image. What you have looks more like just one or two bits moved (not copied and added) to the right in the image. I'm attaching a simple Verilog test fixture you can simulate to see what I mean. The pixel data just counts up and down crossing the value 0x40 on each excursion. Then there is a one clock cycle delayed version of the pixels which gives the same values but one cycle later. Finally the mixed-up value pixel_upper2_delayed consists of the low 6 bits of the original pixel data and the upper two bits of the delayed pixel data. You will notice that as the pixel value crosses 0x40 counting down you get a bright pixel (0x7F) and as it crosses 0x40 counting up you get a dark pixel (0x00). This looks very much like your bad image where you have bright lines in areas that have a shade that darkens from left to right, and black lines where the image brightens from left to right.
01-24-2014 01:11 AM
Hi gabor, that's a greate help for me to understand the issue and many thanks for your kind help! I haved done some
tests to locate the data synchronize problem in our design. Now i would like to provide more test results to you, and if
you can give me some comments or instructions, that would be greate!
The MT9V032 image sensor we use provide a gray shaded test pattern to used as digital test data (when enable the
use of gray shade test pattern in the signal chain, the data is inserted instead of data from the ADCs). There are three
kinds of gray shaded test pattern: Vertical Shades, Horizontal Shades, Diagonal Shade. I found that when i configure
the image sensor to use the Horizontal Shades test data, it looks the image is ok. But when the image sensor is
configured to output the Vertical Shades test data, it looks the image is abnormal at some of time. Here is some posts.
Horizontal Shades: (it looks it is ok)
The Vertical Shades (some that looks ok):
The Vertical Shades (some that looks abnormal, just some extra black lines in different density):
It looks that several columns occurs at the wrong places. Even though the VDMA complains that there is a SOFLateErr
error status, but i prefer to think this may be caused by a clock domain crossing (since if the vsync is not correct, the
640x480 picture will be cut partly at the bottom several rows). Do you think so?
I attach my ten tests of the vertical shade and horizontal shade tests, and you can have a detailed description!
Although I devote a lot of time and efferts to solve this problem, a little progress i have got ! Here i just want to express
my greate appreciate for your patient and careful answer!
01-24-2014 06:24 AM
The pictures you posted seem to match with the idea of mis-registered bits as we discussed in the previous posts. The "horizontal shades" would not show this issue because an entire row of pixels all have the same value. So if some bits were registered on the wrong clock it wouldn't matter becase the bits aren't changing anyway.
In the "vertical shades" you see very consistently spaced dark bars as predicted by the code I posted. In fact it looks like only bits 7:6 are wrong on the dark columns, i.e. 0x00 should be 0x40, 0x40 should be 0x80, etc.
When I have problems like this I like to narrow the issue down to a particular module in the design. An easy way to do that is to allow any module in the data flow to replace the live data with a simple pattern created by a counter. Then start from the first unit (the one that captures data from the imager) and one by one turn on the data replacement until the problem goes away. If it goes away when the imager interface replaces its data with the simple count, then you know that the problem is right at the input interface, and so on.