UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Visitor cozzmy13
Visitor
659 Views
Registered: ‎07-15-2019

MIPI CSI-2 RX stream line buffer full error

Jump to solution

In Vivado 2019.1, when using the MIPI CSI-2 RX IP to capture images from a  OV5640 camera (Digilent Pcam), the following error is printed in dmesg on the second capture.

xilinx-csi2rxss 43c70000.mipi_csi2_rx_subsystem: Stream Line Buffer Full!

The Product Guide says that in such occasions, we should hold the video_aresetn signal low for a period of time (under Table 2-12),

I added the following commit to the kernel to support the use of the video_aresetn GPIO from the xcsi2rxss driver.

https://github.com/Xilinx/linux-xlnx/commit/c3e21481103661600d4398e63860a6195af742cc

I used ILA to see that the signal is being flipped down and then back up, and also added a GPIO which I control manually to see if that makes the situation any better, but nothing helped.

What could be the problem?

Tags (3)
0 Kudos
1 Solution

Accepted Solutions
Visitor cozzmy13
Visitor
313 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

I figured it up.

My original scope was to upgrade a prior project of my company from 2017.4 to 2019.1.

The issue was that the reset signals of a lot of IPs were weirdly configured.

I moved all of the resets to peripheral_aresetn, except the memory interconnect which I kept on interconnect_aresetn.

I moved the external hardware reset (specified inside reset-gpios field of the Frame Buffer Write IP in device tree) from the subset converter to the Frame Buffer Write IP itself, where it actually made sense to be.

After that, everything worked smoothly on 640x480 at 15hz.

But on 1920x1080 at 15hz, the picture comes up like below.

If you can provide some pointers on this one, please do.

frame-000013.png

18 Replies
Moderator
Moderator
580 Views
Registered: ‎11-09-2015

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

HI @cozzmy13 

This is a screenshot of the debugging section of PG232:

MIPI.JPG

 


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
Xilinx Employee
Xilinx Employee
568 Views
Registered: ‎03-30-2016

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Hello

I agree with Florent.

Just adding my two cents. It is probably your video_aclk frequency is to slow.
Please read PG232 Chapter 3 "Clocking" section.
Please share your calculation result here, we can double check on that.

Regards
Leo

Visitor cozzmy13
Visitor
556 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

My video_aclk had the same source as lite_aclk, which is FCLK_CLK0, configured to 100MHz.

The calculations gave a minimum of 200MHz required.

Since I didn't want to up the clock speed past the 175MHz limit suggested by the same document, I changed the pixels per clock to 2 and re-did the whole subsystem to handle 2 pixels.

The pictures came out with wrong colors, but that must be because I remapped the bits wrongly in the subset converter.

A clocking / data rate problem would mean that the image capture wouldn't work the first time, right? But the first image capture works. I can get 14 consecutive frames at 640x480x15hz, on the second capture, the same error message is printed as before.

Moderator
Moderator
550 Views
Registered: ‎11-09-2015

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Hi @cozzmy13 

A clocking / data rate problem would mean that the image capture wouldn't work the first time, right? But the first image capture works. I can get 14 consecutive frames at 640x480x15hz, on the second capture, the same error message is printed as before

[Florent] - No, it might work for few frames (depending on your resolution). On the first frame, your FIFO will be able to handle (filling up slowly). But after few frames the fifo will no be able to keep up with the input rate


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
Visitor cozzmy13
Visitor
547 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution
But the FIFO is supposed to be reset by the video_aresetn on each line buffer full condition.

With the old pixel mode,
video_aclk2 = 800 * 2 / 1 / 16 = 100

With the new pixel mode,
video_aclk2 = 800 * 2 / 2 / 16 = 50

And I use a 100MHz clock. This should work.
Visitor cozzmy13
Visitor
489 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Just to clarify, even after reconfiguring the number of pixels per clock, I still run into the same issue.

From the calculations, it is clear that both old and new configurations are supposed to work, but neither do.

0 Kudos
Xilinx Employee
Xilinx Employee
484 Views
Registered: ‎03-30-2016

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Hello @cozzmy13 

What is the data type you are using ?

Thanks
Leo

0 Kudos
Visitor cozzmy13
Visitor
478 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

MIPI CSI RX:

YUV 422 8bit, 2 serial lanes

 

AXIS SUBSET CONVERTER:

TDATA Remap String: 8'b00000000,tdata[15:0]

 

Video Framebuffer Write:

Samples per clock: 1

8bit Video Formats: YUYV8, UYVY8, Y_UV8

 

Device tree is configured to use XVIP_VF_YUV_422.

0 Kudos
Xilinx Employee
Xilinx Employee
439 Views
Registered: ‎03-30-2016

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Hello @cozzmy13 

Thank you for sharing the clock calculation, Your video_aclk frequency should be sufficient.

I think your MIPI CSI-2 IP does not have any issue.
The buffer overflow occurs because of "video_out_tready" is not asserted. (Fixed at 0)
I am suspecting that the back-end module causing this buffer overflow issue.

Could you please confirm using ILA ?

Thanks & regards
Leo

0 Kudos
Visitor cozzmy13
Visitor
438 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Indeed, I used ILA to verify the TREADY signals between MIPI and the AXIS Substream Converter and between the AXIS Substream Converter and the Video Frame Buffer Write and after the first capture, the signals were still on 1, but after starting the second capture, the signals were stuck on 0 forever.

Xilinx Employee
Xilinx Employee
385 Views
Registered: ‎03-30-2016

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Hello @cozzmy13 

Thanks for checking the HW behavior.

See also the following thread:
https://forums.xilinx.com/t5/Video/CSI2-RX-embedded-non-image-interface-how-to-synchronize-metadata/td-p/1023164

-- 2019.1 MIPI CSI-2 RX asserted TUSER[0] signal for multiple clocks, (when TUSER width set as 1bit )
while some designs may still work with this kind of unexpected TUSER[0] behavior,
Could you confirm if your system design will work with this TUSER behavior ?
-- If this behavior could cause some issue, available workaround options are :
(a) masked unnecessary TUSER[0] pulse. ( Only the first beat is necessary )
(b) set TUSER width to 96 bit

Thanks & regards
Leo

0 Kudos
Visitor cozzmy13
Visitor
361 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

I made a small RTL module to make the TUSER signal only take 1 period.

 

library IEEE;
use IEEE.STD_LOGIC_1164.ALL;
entity tuser_fixup is
port(
    clock : in STD_LOGIC;
    tuser : in STD_LOGIC;
    tuser_fixed : out STD_LOGIC
);
end tuser_fixup;

architecture main of tuser_fixup is
signal last_tuser : STD_LOGIC := '0';
begin
    process(clock)
    begin
        if rising_edge(clock) then
            last_tuser <= tuser;
        end if;
    end process;

    tuser_fixed <=
            '1' when last_tuser = '0' and tuser = '1' else
            '0';
end main;

Wired it up, and even though TUSER takes only a signle pulse now, TREADY still becomes 0 at the beggining of the second capture.

Screenshot_20190926_164222.pngScreenshot_20190926_164843.png

0 Kudos
Xilinx Employee
Xilinx Employee
333 Views
Registered: ‎03-30-2016

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Hello @cozzmy13 

If input-ready of your "Video Frame Buffer Write" module (s_axis_tready) suddenly changed to a fixed "0",
You probably needs to debug from this module first. ( Check the clock sources, Video format of input stream and Frame-buffer IP setting )

Thanks & regards
Leo

0 Kudos
Visitor cozzmy13
Visitor
314 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

I figured it up.

My original scope was to upgrade a prior project of my company from 2017.4 to 2019.1.

The issue was that the reset signals of a lot of IPs were weirdly configured.

I moved all of the resets to peripheral_aresetn, except the memory interconnect which I kept on interconnect_aresetn.

I moved the external hardware reset (specified inside reset-gpios field of the Frame Buffer Write IP in device tree) from the subset converter to the Frame Buffer Write IP itself, where it actually made sense to be.

After that, everything worked smoothly on 640x480 at 15hz.

But on 1920x1080 at 15hz, the picture comes up like below.

If you can provide some pointers on this one, please do.

frame-000013.png

Moderator
Moderator
267 Views
Registered: ‎11-09-2015

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

Hi @cozzmy13 

Are you reconfiguring your application to change the resolution? You might want to recheck that every IP is configured for this resolution


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
Visitor cozzmy13
Visitor
254 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

I'm doing the configuration through Linux.

on both:
width=1920 && height=1080 && rate=15
on board: media-ctl -d /dev/media0 -V '"ov5640 0-003c":0 [fmt:UYVY/'"$width"x"$height"'@1/'"$rate"' field:none]' media-ctl -d /dev/media0 -V '"43c60000.mipi_csi2_rx_subsystem":0 [fmt:UYVY/'"$width"x"$height"' field:none]' v4l2-ctl -d /dev/video0 --set-fmt-video=width="$width",height="$height",pixelformat='YUYV' yavta -c14 -f YUYV -s "$width"x"$height" -F /dev/video0

on host:
scp root@addr:~/*.bin .
for f in *.bin; do mv -- "$f" "${f%.bin}.yuv"; done
for f in *.yuv; do ffmpeg -s "$width"x"$height" -pix_fmt yuyv422 -i "$f" -y "${f%.yuv}.png"; done

These are the commands I'm using to produce the bad images.

I checked the registers and the values seem correct, width is 1920, height is 1080, stride is 3840.

0 Kudos
Moderator
Moderator
245 Views
Registered: ‎11-09-2015

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

HI @cozzmy13 

It seems that in the pipe there is no way to check if the size is correct. So I am wondering if the camera is not sending an incorrect size.

You might want to add some logic to count how many pixels/lines are sent to make sure the size is correct


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
Visitor cozzmy13
Visitor
240 Views
Registered: ‎07-15-2019

Re: MIPI CSI-2 RX stream line buffer full error

Jump to solution

I just checked out the old kernel driver for my ov5640 camera and now the images come out fine. I can figure this out on my own. Thanks for all the help!