cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
cozzmy13
Visitor
Visitor
2,508 Views
Registered: ‎07-15-2019

MIPI CSI-2 RX stream line buffer full error

Jump to solution

In Vivado 2019.1, when using the MIPI CSI-2 RX IP to capture images from a  OV5640 camera (Digilent Pcam), the following error is printed in dmesg on the second capture.

xilinx-csi2rxss 43c70000.mipi_csi2_rx_subsystem: Stream Line Buffer Full!

The Product Guide says that in such occasions, we should hold the video_aresetn signal low for a period of time (under Table 2-12),

I added the following commit to the kernel to support the use of the video_aresetn GPIO from the xcsi2rxss driver.

https://github.com/Xilinx/linux-xlnx/commit/c3e21481103661600d4398e63860a6195af742cc

I used ILA to see that the signal is being flipped down and then back up, and also added a GPIO which I control manually to see if that makes the situation any better, but nothing helped.

What could be the problem?

Tags (3)
0 Kudos
1 Solution

Accepted Solutions
cozzmy13
Visitor
Visitor
2,162 Views
Registered: ‎07-15-2019

I figured it up.

My original scope was to upgrade a prior project of my company from 2017.4 to 2019.1.

The issue was that the reset signals of a lot of IPs were weirdly configured.

I moved all of the resets to peripheral_aresetn, except the memory interconnect which I kept on interconnect_aresetn.

I moved the external hardware reset (specified inside reset-gpios field of the Frame Buffer Write IP in device tree) from the subset converter to the Frame Buffer Write IP itself, where it actually made sense to be.

After that, everything worked smoothly on 640x480 at 15hz.

But on 1920x1080 at 15hz, the picture comes up like below.

If you can provide some pointers on this one, please do.

frame-000013.png

View solution in original post

18 Replies
florentw
Moderator
Moderator
2,429 Views
Registered: ‎11-09-2015

HI @cozzmy13 

This is a screenshot of the debugging section of PG232:

MIPI.JPG

 


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
karnanl
Xilinx Employee
Xilinx Employee
2,417 Views
Registered: ‎03-30-2016

Hello

I agree with Florent.

Just adding my two cents. It is probably your video_aclk frequency is to slow.
Please read PG232 Chapter 3 "Clocking" section.
Please share your calculation result here, we can double check on that.

Regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
cozzmy13
Visitor
Visitor
2,405 Views
Registered: ‎07-15-2019

My video_aclk had the same source as lite_aclk, which is FCLK_CLK0, configured to 100MHz.

The calculations gave a minimum of 200MHz required.

Since I didn't want to up the clock speed past the 175MHz limit suggested by the same document, I changed the pixels per clock to 2 and re-did the whole subsystem to handle 2 pixels.

The pictures came out with wrong colors, but that must be because I remapped the bits wrongly in the subset converter.

A clocking / data rate problem would mean that the image capture wouldn't work the first time, right? But the first image capture works. I can get 14 consecutive frames at 640x480x15hz, on the second capture, the same error message is printed as before.

florentw
Moderator
Moderator
2,399 Views
Registered: ‎11-09-2015

Hi @cozzmy13 

A clocking / data rate problem would mean that the image capture wouldn't work the first time, right? But the first image capture works. I can get 14 consecutive frames at 640x480x15hz, on the second capture, the same error message is printed as before

[Florent] - No, it might work for few frames (depending on your resolution). On the first frame, your FIFO will be able to handle (filling up slowly). But after few frames the fifo will no be able to keep up with the input rate


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
cozzmy13
Visitor
Visitor
2,396 Views
Registered: ‎07-15-2019
But the FIFO is supposed to be reset by the video_aresetn on each line buffer full condition.

With the old pixel mode,
video_aclk2 = 800 * 2 / 1 / 16 = 100

With the new pixel mode,
video_aclk2 = 800 * 2 / 2 / 16 = 50

And I use a 100MHz clock. This should work.
cozzmy13
Visitor
Visitor
2,338 Views
Registered: ‎07-15-2019

Just to clarify, even after reconfiguring the number of pixels per clock, I still run into the same issue.

From the calculations, it is clear that both old and new configurations are supposed to work, but neither do.

0 Kudos
karnanl
Xilinx Employee
Xilinx Employee
2,333 Views
Registered: ‎03-30-2016

Hello @cozzmy13 

What is the data type you are using ?

Thanks
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
cozzmy13
Visitor
Visitor
2,327 Views
Registered: ‎07-15-2019

MIPI CSI RX:

YUV 422 8bit, 2 serial lanes

 

AXIS SUBSET CONVERTER:

TDATA Remap String: 8'b00000000,tdata[15:0]

 

Video Framebuffer Write:

Samples per clock: 1

8bit Video Formats: YUYV8, UYVY8, Y_UV8

 

Device tree is configured to use XVIP_VF_YUV_422.

0 Kudos
karnanl
Xilinx Employee
Xilinx Employee
2,288 Views
Registered: ‎03-30-2016

Hello @cozzmy13 

Thank you for sharing the clock calculation, Your video_aclk frequency should be sufficient.

I think your MIPI CSI-2 IP does not have any issue.
The buffer overflow occurs because of "video_out_tready" is not asserted. (Fixed at 0)
I am suspecting that the back-end module causing this buffer overflow issue.

Could you please confirm using ILA ?

Thanks & regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
cozzmy13
Visitor
Visitor
2,287 Views
Registered: ‎07-15-2019

Indeed, I used ILA to verify the TREADY signals between MIPI and the AXIS Substream Converter and between the AXIS Substream Converter and the Video Frame Buffer Write and after the first capture, the signals were still on 1, but after starting the second capture, the signals were stuck on 0 forever.

karnanl
Xilinx Employee
Xilinx Employee
2,234 Views
Registered: ‎03-30-2016

Hello @cozzmy13 

Thanks for checking the HW behavior.

See also the following thread:
https://forums.xilinx.com/t5/Video/CSI2-RX-embedded-non-image-interface-how-to-synchronize-metadata/td-p/1023164

-- 2019.1 MIPI CSI-2 RX asserted TUSER[0] signal for multiple clocks, (when TUSER width set as 1bit )
while some designs may still work with this kind of unexpected TUSER[0] behavior,
Could you confirm if your system design will work with this TUSER behavior ?
-- If this behavior could cause some issue, available workaround options are :
(a) masked unnecessary TUSER[0] pulse. ( Only the first beat is necessary )
(b) set TUSER width to 96 bit

Thanks & regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
cozzmy13
Visitor
Visitor
2,210 Views
Registered: ‎07-15-2019

I made a small RTL module to make the TUSER signal only take 1 period.

 

library IEEE;
use IEEE.STD_LOGIC_1164.ALL;
entity tuser_fixup is
port(
    clock : in STD_LOGIC;
    tuser : in STD_LOGIC;
    tuser_fixed : out STD_LOGIC
);
end tuser_fixup;

architecture main of tuser_fixup is
signal last_tuser : STD_LOGIC := '0';
begin
    process(clock)
    begin
        if rising_edge(clock) then
            last_tuser <= tuser;
        end if;
    end process;

    tuser_fixed <=
            '1' when last_tuser = '0' and tuser = '1' else
            '0';
end main;

Wired it up, and even though TUSER takes only a signle pulse now, TREADY still becomes 0 at the beggining of the second capture.

Screenshot_20190926_164222.pngScreenshot_20190926_164843.png

0 Kudos
karnanl
Xilinx Employee
Xilinx Employee
2,182 Views
Registered: ‎03-30-2016

Hello @cozzmy13 

If input-ready of your "Video Frame Buffer Write" module (s_axis_tready) suddenly changed to a fixed "0",
You probably needs to debug from this module first. ( Check the clock sources, Video format of input stream and Frame-buffer IP setting )

Thanks & regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
cozzmy13
Visitor
Visitor
2,163 Views
Registered: ‎07-15-2019

I figured it up.

My original scope was to upgrade a prior project of my company from 2017.4 to 2019.1.

The issue was that the reset signals of a lot of IPs were weirdly configured.

I moved all of the resets to peripheral_aresetn, except the memory interconnect which I kept on interconnect_aresetn.

I moved the external hardware reset (specified inside reset-gpios field of the Frame Buffer Write IP in device tree) from the subset converter to the Frame Buffer Write IP itself, where it actually made sense to be.

After that, everything worked smoothly on 640x480 at 15hz.

But on 1920x1080 at 15hz, the picture comes up like below.

If you can provide some pointers on this one, please do.

frame-000013.png

View solution in original post

florentw
Moderator
Moderator
2,116 Views
Registered: ‎11-09-2015

Hi @cozzmy13 

Are you reconfiguring your application to change the resolution? You might want to recheck that every IP is configured for this resolution


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
cozzmy13
Visitor
Visitor
2,103 Views
Registered: ‎07-15-2019

I'm doing the configuration through Linux.

on both:
width=1920 && height=1080 && rate=15
on board: media-ctl -d /dev/media0 -V '"ov5640 0-003c":0 [fmt:UYVY/'"$width"x"$height"'@1/'"$rate"' field:none]' media-ctl -d /dev/media0 -V '"43c60000.mipi_csi2_rx_subsystem":0 [fmt:UYVY/'"$width"x"$height"' field:none]' v4l2-ctl -d /dev/video0 --set-fmt-video=width="$width",height="$height",pixelformat='YUYV' yavta -c14 -f YUYV -s "$width"x"$height" -F /dev/video0

on host:
scp root@addr:~/*.bin .
for f in *.bin; do mv -- "$f" "${f%.bin}.yuv"; done
for f in *.yuv; do ffmpeg -s "$width"x"$height" -pix_fmt yuyv422 -i "$f" -y "${f%.yuv}.png"; done

These are the commands I'm using to produce the bad images.

I checked the registers and the values seem correct, width is 1920, height is 1080, stride is 3840.

0 Kudos
florentw
Moderator
Moderator
2,094 Views
Registered: ‎11-09-2015

HI @cozzmy13 

It seems that in the pipe there is no way to check if the size is correct. So I am wondering if the camera is not sending an incorrect size.

You might want to add some logic to count how many pixels/lines are sent to make sure the size is correct


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
cozzmy13
Visitor
Visitor
2,089 Views
Registered: ‎07-15-2019

I just checked out the old kernel driver for my ov5640 camera and now the images come out fine. I can figure this out on my own. Thanks for all the help!