cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
493 Views
Registered: ‎07-12-2011

Sensor Demosaic outputs wrong color data

Jump to solution

Hello everybody,

 

I'm currently working at project that implements image processing pipeline to CMOS color sensor. First step is to receive image data from sensor and send them to monitor. Sensor outputs RAW8 1080p60 over MIPI CSI-2 4 lanes. MIPI CSI-2 Rx Subsystem IP Core from Xilinx works perfectly, image is stored into frame buffer (VDMA and DDR3, 2 pixels per clock) and then send to monitor over HDMI. That part is working fine and I can see unprocessed black-and-white image.

Next step is to recover color information by implementing demosaic algorithm. As Xilinx offers Sensor Demosaic IP Core I simply put it between CSI-2 RX Subsystem and AXI VDMA, set parameters (2 samples per clock, 8 bit data width, 1920x1080 resolution and highest quality), enabled it through registers in SDK and... well, I got color image but that was not what I was expecting. Image was greenish and color matching was definitely wrong.

So I attempted to debug this case. CMOS sensor has possibility to send test pattern, so I enabled solid red color to easily see incoming pattern. Next, I connected AXI-Stream output from CSI-2 Rx core and from Demosaic core to ILA to see actual data. Quick investigation shows that CSI-2 Rx core outputs correct image data but demosaic one not.

Sensor's Bayer phase is [LINE #1] Green(Red), Red... [LINE #2] Blue, Green(Blue)... so raw data should be [LINE #1] 0xFF 00 [LINE #2] 0x00 00. 2 pixels per clock and first pixel Green(Red) is at TDATA's LSB. Expected output from Demosaic core should be [LINE #1] 0xFF0000 FF0000 [LINE #2] 0x000000 000000. Yet I get [LINE #1] 0xFE0101 FE0101 (which is fine) and [LINE #2] 0xFE0000 FE0000 - why there is 0xFE on Red component when raw data for whole line is 0x0000?

This repeat for each pair, i.e. [LINE #3] is fine but [LINE #4] has Red=0xFE instead of 0x00. When I switch test pattern to another color, i.e. solid blue, situation looks very similar, with minor change: [LINE #1] is correctly processed (i.e. raw 0x00 00 ---> RGB 0x000000 000000), [LINE #2] also is correctly processed (i.e. raw 0x00 FF ---> RGB 0x01FE01 01FE01) but [LINE #3] is corrupted (raw 0x00 00 ---> RGB 0x00FE00 00FE00 instead of 0x000000 000000).

The same behaviour remains between all four selectable Bayer phases (register 0x0028). It looks like the Sensor Demosaic core incorrectly flush line buffer or other registers and pass some data from first line to another.

demosaic_color_problem_01.png

 

Additional information:

  • Vivado 2019.1, Windows 10 x64 1903
  • MIPI CSI-2 Rx Subsystem IP Core v4.0, RAW8, 4 lanes, 800Mbps per lane, 2 pixels per clock
  • Sensor Demosaic IP Core v1.0, 2 samples per clock, 8bit data width, max resolution 1920x1080, High Resolution Interpolation, Horizontal Zipper Artifact Removal enabled (problem remains when I disable Horizontal... or change to Fringe Tolerant Interpolation)
  • input side, i.e. CSI-2 Rx, demosaic and VDMA S2MM works at 100MHz, unprocessed RAW8 (grayscale) is received and displayed correctly without any artifacts

demosaic_color_problem_02.png

 

Is there a problem with IP Core or it is me doing something wrong?

0 Kudos
1 Solution

Accepted Solutions
Highlighted
444 Views
Registered: ‎07-12-2011

Re: Sensor Demosaic outputs wrong color data

Jump to solution

Hi Leo,

while post that you mentioned describes another kind of greenish image, it pointed me to the right direction. Sensor Demosaic core is working correctly. I feel silly but somehow I forgot that Red pixel from 1st line defines also - after interpolation - red value for 2nd line, not only for first one. In the end that's the idea of Bayer filter...

So core is working as expected and I just wrongly interpreted output stream. Thank you for your help.

 

Best regards,
Przemyslaw

View solution in original post

2 Replies
Highlighted
Xilinx Employee
Xilinx Employee
463 Views
Registered: ‎03-30-2016

Re: Sensor Demosaic outputs wrong color data

Jump to solution

Hello @przemyslaw.rutkowski 

I think your post is very similar to this post.
Could you please check if this helps your issue ?
https://forums.xilinx.com/t5/Video/MIPI-interface-Technology-Development-issues-green-display-on/td-p/874829

Thanks & regard
Leo

0 Kudos
Highlighted
445 Views
Registered: ‎07-12-2011

Re: Sensor Demosaic outputs wrong color data

Jump to solution

Hi Leo,

while post that you mentioned describes another kind of greenish image, it pointed me to the right direction. Sensor Demosaic core is working correctly. I feel silly but somehow I forgot that Red pixel from 1st line defines also - after interpolation - red value for 2nd line, not only for first one. In the end that's the idea of Bayer filter...

So core is working as expected and I just wrongly interpreted output stream. Thank you for your help.

 

Best regards,
Przemyslaw

View solution in original post