UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Adventurer
Adventurer
6,826 Views
Registered: ‎12-28-2012

A ghost image issue

Hi all,  I have a video system which run on our custom board based on xc7z010-clg225 soc.  The fpga logci path of the

 

video system mainly consists of a cmos image sensor which provide the video source, xsvi interface pcore, vidin2axi4s

 

pcore,  AXI VDMA which send image data to DDR through the axi high performance slave port t.  Then we transfer the

 

image data back to the pc and display it.  It seems that at most of time, it woks well. But there are some shadow just

 

like the following screenshot show in it occasionally.  I suspect there must be some noise on our design, but i am a

 

newer of the video system, and i still can not locate which place will be cause this problem up to now. Hope someone

 

can give me some suggestions. Any suggestion and comment would be very appreciate!

 

 

ghost_image.png

 

Best Regards,

Dec

0 Kudos
10 Replies
Instructor
Instructor
6,808 Views
Registered: ‎08-14-2007

Re: A ghost image issue

Since most of us have no clue what your picture should look like, could you post a picture without the "shadow" so we might have an idea what to look for?

-- Gabor
0 Kudos
Adventurer
Adventurer
6,799 Views
Registered: ‎12-28-2012

Re: A ghost image issue

Hi gabor, thanks for your reply! Ok, here is the normal state screenshot at most of time.   The big distinguish of this two

 

screenshot is whether the multiple layer shadow occurs(just my expression, maybe not exactly). Why the abnormal

 

case occurs occasionally and how to fix it, which are the main issue troubled me.   There are still some slightly

 

distinguish which may be affected by the light of my environment.  Any suggestion and comment would be greate

 

appreciate!

 

normal.png

 

 

Best Regards,

Dec

 

0 Kudos
Instructor
Instructor
6,793 Views
Registered: ‎08-14-2007

Re: A ghost image issue

What your first image looks like to me is not something I would associate with noise.  Noise usually causes "salt and pepper" (high-frequency shot noise) or wavy horizontal bands (low frequency noise like power supply switching).  What you have seems like the data bits have been systematically changed.  It looks partly like "posterization" which is what you get when you reduce the number of grey-scale levels from 256 down to just a handful, and partly like a stuck bit, say the next to the high-order bit.  I would suggest saving raw images (no filtering, no JPEG encoding) and using a program that lets you view the pixel values at the cursor.  Irfanview can do this (you can download it free, just Google for it).  Perhaps you'll see something systematic about the bad image.  Those ragged lines around sections of the image could give a clue.  See if they occur where the greyscale of the image is near a change in a high order bit, like multiples of 0x40.

-- Gabor
0 Kudos
Adventurer
Adventurer
6,777 Views
Registered: ‎12-28-2012

Re: A ghost image issue

Hi gabor, i am sorry that my response is late and thanks for your detailed answer.  I was so dull that i think i can only

 

understand your words partly. You suppose should be right and the pixel value at several places may be changed

 

unexpectlly. I have saved the image data captured by the cmos and routed to DDR through logic path as a bmp format

 

file(just the raw image data).   Through the IrfanView, i can see the component of the abnormal pixels value are close to

 

0xff, and please see my attach for a detail description.  

 

imgdata1.png

 

 

In my consideration , the software code do not have the permission to modify the image data in ddr region, and I would

 

like to think this may be caused by the data path(to math the axi4-stream or axi bus, some bits has padded) in the fpga

 

logic path in our design.  Do you think so?   But it is strange that at most of time it works well.  I am looking forward to 

 

your reply. Very thanks!

 

Best Regards,

Dec

 

 

 

0 Kudos
Instructor
Instructor
6,769 Views
Registered: ‎08-14-2007

Re: A ghost image issue

It looks like the light or dark lines come where the image has a transition in the upper two bits, for example going from 0x3f to 0x40.  This looks like a synchronization issue.  If you have some point in the design where the bits are not coherent when they come through, you can have the effect of some bits coming from one pixel and other bits coming from the one next to it.  Now suppose you took a normal image and shifted just bits 6 and 7 of the 8-bit data right by one pixel.  Then if you had a nice grey ramp going 0x3D, 0x3E, 0x3F, 0x40, 0x41 it would translate to 0x3D, 0x3E, 0x3F, 0x00, 0x41, ...

This looks very much like your bad image.

 

You need to look where there can be some sort of data synchronization problem in the design.  It could be right at the input from the sensor if this is a setup/hold issue.  Or it could be internal to the design at a clock domain crossing.

-- Gabor
0 Kudos
Adventurer
Adventurer
6,764 Views
Registered: ‎12-28-2012

Re: A ghost image issue

Hi gabor, thanks for your detailed analysis.  I ignore that there are still some pixels value has changed to 0x00 (black 

 

points). But i still  do not understand your words about "in the upper two bits" or "high-order bits". Do you mean that

 

the upper bits of  a color compoent of a pixel ?   Frankly speaking, i am not quite understand this words "where the image has a transition in the upper two bits, for example going from 0x3f to 0x40". 

 

About "Now suppose you took a normal image and shifted just bits 6 and 7 of the 8-bit data right by one pixel.  Then if you had a nice grey ramp going 0x3D, 0x3E, 0x3F, 0x40, 0x41 it would translate to 0x3D, 0x3E, 0x3F, 0x00, 0x41, ...

This looks very much like your bad image.",  I try to understand a method to change the pixel value (bit[6:8] right shift to reduce the number of gray level, right?)  to have the same visual effect with our bad image.

 

 

Yeah, i should point out that we use the lvds interface to receive data from Aptina mt9v032 image sensor, and we have

 

struggled to slove the data lock issue for a long time ever, and maybe this is one place we should verify again.  Another

 

point is that the status registers of VDMA pcore complains some error stutas in our design.  Here is the dump

 

Lvds monitor pcore reset ...
 LVDSMON Status Reg = 0x000C0C03
Lvds monitor pcore reset OK
Vdma write frames to mem ...
Vdma write frames to mem Over

VDMA WRITE Channel Status:
         PARKPTR        = 0x00000000
        ------------------
         S2MM_DMACR       = 0x00010043
         S2MM_DMASR       = 0x00014810
         S2MM_STRD_FRMDLY = 0x00000A00
         S2MM_START_ADDR0 = 0x01000000
         S2MM_START_ADDR1 = 0x0112C000
         S2MM_START_ADDR2 = 0x01258000
         S2MM_HSIZE       = 0x00000A00
         S2MM_VSIZE       = 0x000001E0
         ----------------
         S2MM_HSIZE_STATUS= 0x00000000
         S2MM_VSIZE_STATUS= 0x00000000
         ----------------

 

From the pg020 datasheet,  we can see the S2MM channel of VDAM complains that there are DMAIntErr, SOFLateErr,

 

Err_Irq  (maybe the logic design of the upstream is not  suitable to the requirement of VDMA, so he gives these error

 

status, right?).   

 

vdam.png

 

 

I should have associated these error status with the bad image together.  Thank you very much! 

 

Best Regards,

Dec

 

 

0 Kudos
Instructor
Instructor
6,738 Views
Registered: ‎08-14-2007

Re: A ghost image issue

I was talking about shifting the upper two bits of the pixel value (color component if you will) but not within the same pixel.  Rather shifted in time to the next pixel in the image.  Note that image data is presented serially starting from the left top pixel in the image, then moving to the right until the right top pixel.  Next it goes to the left pixel of the second from the top row and so on.  If you think of the spatial image as a time sequence, delaying a signal has the effect of moving it to the right in the image you see on the screen.  A "ghost" in TV parlance is actually an echo of the original image that adds a right-shifted copy of the image (at reduced intensity) to the original image.  What you have looks more like just one or two bits moved (not copied and added) to the right in the image.  I'm attaching a simple Verilog test fixture you can simulate to see what I mean.  The pixel data just counts up and down crossing the value 0x40 on each excursion.  Then there is a one clock cycle delayed version of the pixels which gives the same values but one cycle later.  Finally the mixed-up value pixel_upper2_delayed consists of the low 6 bits of the original pixel data and the upper two bits of the delayed pixel data.  You will notice that as the pixel value crosses 0x40 counting down you get a bright pixel (0x7F) and as it crosses 0x40 counting up you get a dark pixel (0x00).  This looks very much like your bad image where you have bright lines in areas that have a shade that darkens from left to right, and black lines where the image brightens from left to right.

-- Gabor
0 Kudos
Instructor
Instructor
6,735 Views
Registered: ‎08-14-2007

Re: A ghost image issue

Here's a picture from simulation showing the effects in both directions:

-- Gabor
Upper_2_delayed.PNG
0 Kudos
Adventurer
Adventurer
6,727 Views
Registered: ‎12-28-2012

Re: A ghost image issue

Hi gabor, that's a greate help for me to understand the issue and many thanks for your kind help!   I haved done some

 

tests to locate the data synchronize problem in our design. Now i would like to provide more test results to you,  and if

 

you can give me some comments or instructions, that would be greate!

 

The MT9V032 image sensor we use provide a gray shaded test pattern to used as digital test data (when enable the

 

use of gray shade test pattern in the signal chain, the data is inserted instead of  data from the ADCs).  There are three

 

kinds of gray shaded test pattern: Vertical Shades, Horizontal Shades, Diagonal Shade.  I found that when i configure

 

the image sensor to use the Horizontal  Shades test data, it looks the image is ok. But when the image sensor is

 

configured to output the Vertical Shades test data, it looks the image is abnormal at some of time.  Here is some posts.

 

 

Horizontal Shades: (it looks it is ok)

tpg_hori.png

 

 

The Vertical Shades (some that looks ok):

tpg_vert.png

 

 

The Vertical Shades (some that looks abnormal, just some extra black lines in different density):

tpg_vert_abnormal.png

 

 

It looks that several columns occurs at the wrong places.  Even though the VDMA complains that  there is a SOFLateErr 

 

error status, but i prefer to think this may be caused by a  clock domain crossing (since if the vsync is not correct, the

 

640x480 picture will be cut partly at the bottom several rows).  Do you think so?

 

I attach my ten tests of the vertical shade and horizontal shade tests, and you can have a detailed description!

 

Although I devote a lot of time and efferts to solve this problem,  a little progress i have got ! Here i just want to express

 

my greate appreciate for your patient and careful answer! 

 

 

Best Regards,

Dec

0 Kudos
Highlighted
Instructor
Instructor
1,895 Views
Registered: ‎08-14-2007

Re: A ghost image issue

The pictures you posted seem to match with the idea of mis-registered bits as we discussed in the previous posts.  The "horizontal shades" would not show this issue because an entire row of pixels all have the same value.  So if some bits were registered on the wrong clock it wouldn't matter becase the bits aren't changing anyway.

 

In the "vertical shades" you see very consistently spaced dark bars as predicted by the code I posted.  In fact it looks like only bits 7:6 are wrong on the dark columns, i.e. 0x00 should be 0x40, 0x40 should be 0x80, etc.

 

When I have problems like this I like to narrow the issue down to a particular module in the design.  An easy way to do that is to allow any module in the data flow to replace the live data with a simple pattern created by a counter.  Then start from the first unit (the one that captures data from the imager) and one by one turn on the data replacement until the problem goes away.  If it goes away when the imager interface replaces its data with the simple count, then you know that the problem is right at the input interface, and so on.

-- Gabor
0 Kudos