cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Adventurer
Adventurer
1,065 Views
Registered: ‎10-20-2019

mipi csi-2 tx problem

I have a image sensor, it outputs mipi signals to ECU as shown below. I can observe the image from the ECU. Now the ECU did not  recognize the image signal that I transmitted using FPGA. 

2020-05-12_162042.png

I want to confirm the meaning of 'LineCount' parameter of the function 'XCsi2TxSs_SetLineCountForVC'. Is it just the number of active rows? Is the Vertical Blanking or embedded data rows included?

0 Kudos
59 Replies
Highlighted
Adventurer
Adventurer
688 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Thank you for the further analysis.

I am using Vivado 2019.1.  I checked with you that the patch was not needed in 2019.1 in another question before.

The ILA files including the TX signals  are attached. The find results tables of  some signals are also attached. Because I did not find them in TX module, I added the m_axis signals of axis_subset_converter module to debug.

It looks like  the line_cntr_vc0_r_reg  signal is influenced by the tready signal. 

2020-06-17_110254.png

061701: the first time that the mipi signals were captured after the ECU was powered on. It worked fine.

061702/061703: They are similar.Time of SOF.

Best Regards!

 

Highlighted
Adventurer
Adventurer
688 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Thank you for the further analysis.

I am using Vivado 2019.1.  I checked with you that the patch was not needed in 2019.1 in another question before.

The ILA files including the TX signals  are attached. The find results tables of  some signals are also attached. Because I did not find them in TX module, I added the m_axis signals of axis_subset_converter module to debug.

It looks like  the line_cntr_vc0_r_reg  signal is influenced by the tready signal. 

2020-06-17_110254.png

061701: the first time that the mipi signals were captured after the ECU was powered on. It worked fine.

061702/061703: They are similar.Time of SOF.

Best Regards!

 

Tags (1)
Highlighted
Adventurer
Adventurer
685 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I am sorry. The two replys above are same.  It maybe the network reason.

Best Regards!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
649 Views
Registered: ‎03-30-2016

Hello @xxwang 

Thank you for sharing your ILA waveform.

1. I can see that MIPI CSI-2 RX output has no issue. TUSER[0] is asserted once.
2. But MIPI CSI-2 TX input has TUSER[0] asserted twice.

TX_input_has_2_FS.png
Please fix MIPI CSI-2 TX input to avoid this issue.

Thanks & regards
Leo

Highlighted
Adventurer
Adventurer
640 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Thank you.

The AXI-S subset converter IP is customized as shown below. I thought  TUSER[0] of MIPI CSI-2 TX input  was the same as TUSER[0] of MIPI CSI-2 RX output. I don't understand why they are different. Could you please tell me how to fix that?

2020-06-17_151757.png

2020-06-17_152646.png

Best Regards!

0 Kudos
Highlighted
Adventurer
Adventurer
612 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I probed some tuser[0] signals as shown below. They are all different. I do not understand. And I do not know why   TUSER[0]  of  was  asserted twice.

2020-06-17_162352.png

Best Regards!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
594 Views
Registered: ‎03-30-2016

Hello @xxwang 

Hmm, I believe this is the issue we saw in 2019.2 MIPI CSI-2 RX, when TUSER[0] pulse is asserted for multiple beats.
We already have a patch for 2019.2 (AR#73100) , but unfortunately there is no patch available for 2019.1.

Question :
Would you be able to upgrade your Vivado to 2020.1 ? ( or 2019.2 + patch AR#73100)

Regards
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
570 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Thank you for the reply.

I installed vivado version 2019.2 and the AR73100 patch. I upgrated the project and tested. It looks like the bug problem has been resolved. The ILA waveform is shown below. But the ECU still did not recogonize the mipi signal.  Is the signal transmitted by mipi ip cores  still different from the real sensor's mipi signal?

  2020-06-18_150634.png

2020-06-18_150719.png

mipi tx register dump:

2020-06-18_151214.png

Best Regards!

Best Regards!

Highlighted
Xilinx Employee
Xilinx Employee
555 Views
Registered: ‎03-30-2016

Hello Wang.

>I installed vivado version 2019.2 and the AR73100 patch. I upgrated the project and tested. It looks like the bug problem has been resolved

Thank you for upgrading your design.

1. I can see that your MIPI CSI-2 register dump looks good, there is no "unexpected value".
2. Looking at the waveform screenshot, I do not see any issue either.

> But the ECU still did not recogonize the mipi signal. Is the signal transmitted by mipi ip cores still different from the real sensor's mipi signal?

Noted.
Please understand that MIPI CSI-2 RX cannot time-stamped every sync packet its received and forward it to MIPI CSI-2 TX IP.
So, MIPI CSI-2 TX will send the same RAW12 image/video data, but with a different timing.

I saw a customer system with serializer/deserilizer device that requires the following timing spec :
     A. The time interval between Frame-Start and HS Packet-Header must be more than 1us
     B. The time interval between Packet Footer and Packet-Header must be more than 1us
     C. The time interval between Frame-End and Frame start must be more than 2us
All of these timing requirements are not defined by MIPI spec, but this device requires those timing constraints to process video data correctly.
I think you are using the same device.

So,
1. By enabling "Register Based Frame End Generation", I think you already cleared requirement (C), you can also check your waveform to double check on this.
2. Looking at ILA waveform you have shared before, I can see that requirement (B) is already adhered.
    I estimated that Packet Footer to Packet Header in your data stream is more than 6.4us.
       Packet_Footer_to_packet_header_is_more_than_1us.png
3. The problem is requirement (A).
   AXI4S_or_Native.png
    If you are using AXI4-stream interface for MIPI CSI-2 TX subsystem,
    When TUSER[0]=1 and TVALID=1 observed in the input port, MIPI CSI-2 will send HS packet-header , immediately after sending Frame-start short packet.  There is no way you can follow requirement (A).

4. One possible solution is to create your own MIPI CSI-2 TX IP.
    You can control timing gap that fit your ECU requirement.

5. Another possible workaround is by using MIPI CSI-2 TX Subsystem with Native interface. (see also PG260 appendix B)
    Using a Native interface you can control the gap between Frame-start and HS packet Header, by increasing the gap between "vid_vsync" and "vid_hsync".
    Native_interface_how_to_increase_the_gap.png

          To use MIPI CSI-2 TX native interface , you need to clear these two obstacles.
          (1) You need to convert MIPI CSI-2 RX output (which is AXI4-stream format) to Video native interface format.
               # I would recommend to write your own module to do this format conversion.
               #  Or you can also try to use "AXI4-Stream to Video Out" IP. ( but you still need to do an extra work to control the Gap between
                   https://www.xilinx.com/support/documentation/ip_documentation/v_axi4s_vid_out/v4_0/pg044_v_axis_vid_out.pdf
          (2) MIPI CSI-2 TX native interface is hard to use, because there is no TREADY signal that shows MIPI CSI-2 core is ready (or not ready) to receive data.
               If the input data rate is not optimal you may see Buffer under-run or Buffer full flag asserted.
               You may need to implement a line buffer to control the data-rate fed into MIPI CSI-2 TX Subsystem.

Kind regards
Leo

Highlighted
Adventurer
Adventurer
542 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Thank you for your  experience and your detailed explanation. It is awesome.

It seems No.5 solution is easier than No.4 solution. I will try to implement No.5 solution.


 If the input data rate is not optimal you may see Buffer under-run or Buffer full flag asserted.

Is the BUffer here the  line buffer inside mipi tx ip core ? Can the mipi tx ip core output the  Buffer under-run or Buffer full flag signals? I see the 'Line Buffer Full' signal in 'Interrupt Status Register'. But how to get and use it in vivado design? 
You may need to implement a line buffer to control the data-rate fed into MIPI CSI-2 TX Subsystem.

Is the ' a line buffer' here also the  line buffer inside mipi tx ip core or another buffer that I need to add  by myself before the mipi tx ip core?  Is the 'Generic Short Packet (GSP) FIFO Full' needed to consider?

Best Regards!

 

 

 

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
482 Views
Registered: ‎03-30-2016

Hello @xxwang 

>> If the input data rate is not optimal you may see Buffer under-run or Buffer full flag asserted.
>Is the BUffer here the line buffer inside mipi tx ip core ?

The buffer I mentioned above is the Line buffer inside TX controller core.
tx_line_buffer.png

>Can the mipi tx ip core output the Buffer under-run or Buffer full flag signals?
>I see the 'Line Buffer Full' signal in 'Interrupt Status Register'. But how to get and use it in vivado design?

Interrupt_enable.png

Please enable register address 0x28 bit[2] and bit[0] only.
If Buffer full or under-run occurs , you can check using interrupt=1 output signal from the IP.

out_int_port.png

>> You may need to implement a line buffer to control the data-rate fed into MIPI CSI-2 TX Subsystem.
>Is the ' a line buffer' here also the line buffer inside mipi tx ip core or
>another buffer that I need to add by myself before the mipi tx ip core? Is the 'Generic Short Packet (GSP) FIFO Full' needed to consider?

No, this buffer is not the buffer inside the MIPI TX core.
Please add this buffer yourself.

Kind regards
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
442 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I have tried to use MIPI CSI-2 TX Subsystem with Native interface.  I added a FIFO between RX and TX as a line buffer. I think it has worked, but the ECU still reported errors.  The signals of TX native interface is shown below and I labeld the signal names. Could you help me with it?  Is the gap between Frame-start and HS packet Header ok? Thank you!

2020-06-23_103513.png

2020-06-23_110847.png

Best Regards!

Highlighted
Xilinx Employee
Xilinx Employee
423 Views
Registered: ‎03-30-2016

Hello @xxwang 

From your ILA waveform, I can see that:

1. Your MIPI CSI-2 TX input signal is following PG260 Figure B-3 requirement. No issue.

2. You vid_enable are asserted for 912 clock sample.
    So ( 912 x 2 x 12/8 = 2736 words ) is expected.

3. The gap between Frame-start and HS packet Header is about
           912 sample x 10.2ns = 9.3us
    If your ECU requirement is 1us or more , this waveform has no issue.
     Wang_native_if_input.png

4. I do not see any issue in MIPI CSI-2 TX register dump.

5. Is your sensor sending continuous HS clock ? or non-continous clock ?
    Perhaps you need to mimic sensor clock mode setting.
         clock_mode_cont_non_cont.png

6. Is your ECU still reporting the same error messages ? Or is it displaying different messages ?
   # Since I do not see any issue with MIPI CSI-2 RX/TX IPs, I believe it is time to contact your ECU vendor to check for the details timing requirement.


Kind regards
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
415 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Thank you for your help.

I am told it is continuous  clock.

The ECU is still reporting the same error messages.

Is the timing requirement required by ECU or serializer device?

Best Regards!

0 Kudos
Highlighted
Adventurer
Adventurer
405 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I tested with the  non-continous clock mode, the ECU is still reporting the same error messages.

2020-06-23_161704.png

Best Regards!

0 Kudos
Highlighted
Adventurer
Adventurer
391 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Does the line rate in MIPI CSI-2 TX IP core have to be configured to the right rate? Or would a higher rate be ok?

Best Regards!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
372 Views
Registered: ‎03-30-2016

Hello @xxwang 

 

> I tested with the  non-continous clock mode, the ECU is still reporting the same error messages.

 

Okay, so continuous and non-continuous clock mode has the same result. Noted.

 

> Is the timing requirement required by ECU or serializer device?

 

The timing requirement I have mentioned above is something I have seen from deserializer IC(such as CDX4960/CDX4963).
I do not know anything about ECU.

 

>Does the line rate in MIPI CSI-2 TX IP core have to be configured to the right rate? Or would a higher rate be ok?


I don’t understand this question.

I believe you have set line-rate of MIPI CSI-2 RX and MIPI CSI-2 TX subsystem as 784Mbps @4 lanes, to match the line-rate of your image sensor.

Why do you want to increase the MIPI CSI-2 TX line-rate ? If you don’t see any error in both MIPI IP ?

 

Thanks
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
371 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I read some information from the  serializer datasheet. Can the clock mode ( continuous HS clock   or non-continous clock ) be infered from this?

I will give you a private message.

2020-06-23_181014.png

Best Regards!

0 Kudos
Highlighted
Adventurer
Adventurer
362 Views
Registered: ‎10-20-2019

Hello, @karnanl 

The image sensor with 784Mbps line rate has worked fine with xilinx project. It is another image sensor now. I am told its line rate is 516Mbps. I calculated the line rate maybe 512Mbps from the signals output by mipi rx ip core. So I am not sure the accurate line rate. I do not know if it does matter.

Best Regards!

0 Kudos
Highlighted
Adventurer
Adventurer
358 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I think I found the timing requirement in deserializer IC datasheet.

2020-06-23_185152.png

Best Regards!

0 Kudos
Adventurer
Adventurer
354 Views
Registered: ‎10-20-2019

Hello, @karnanl 

And I think this requirement has been met.

2020-06-23_191040.png

Best Regards!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
299 Views
Registered: ‎03-30-2016

Hello @xxwang 

 

> The image sensor with 784Mbps line rate has worked fine with xilinx project.

> It is another image sensor now. I am told its line rate is 516Mbps. I calculated the line rate maybe 512Mbps from the signals output by mipi rx ip core. So I am not sure the accurate line rate. I do not know if it does matter.

 

I see.

 

  1. PG232 suggested configuring MIPI CSI-2 RX with the same line-rate as Image sensor line-rate.
  2. In this case, you set MIPI CSI-2 RX line-rate as 784 Mbps, but your sensor is sending  MIPI signal with lower line-rate 512/516Mbps.
    It probably works in most cases. ( although we cannot recommend this  case )
  3. You also set MIPI CSI-2 TX with 784Mbps.
    From the MIPI CSI-2 TX perspective, I do not see any issue. MIPI CSI-2 TX is able to send all the data it received from MIPI CSI-2 RX. No error flags asserted.

 

But I do not know if your system will work with 784Mbps.

How do you know if your ECU/deserializer IC is working correctly using 784Mbps ?

How can you use an image sensor without knowing its line-rate or other specs?

 

Thanks
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
257 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I am sorry. I did not express myself clearly.

There are two image sensors and corresponding two ECUs. The image sensor with 784Mbps line rate has worked fine with xilinx project before. The image sensor I worked with is another sensor, and the vivado project is another project. I set MIPI CSI-2 RX line-rate and MIPI CSI-2 TX both as 516Mbps in this project that I debug now. 

 I am told its line rate is 516Mbps. I calculated the line rate maybe 512Mbps from the signals output by mipi rx ip core. I do not know why they are different. So I am not sure the accurate line rate. 

Best Regards!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
223 Views
Registered: ‎03-30-2016

Hello @xxwang 

I believe you have a multiple use-cases for your system.
Wang_usecases.png

On (C) case, I do not see any error on both MIPI CSI-2 RX and MIPI CSI-2 TX register dump and input signal , I think you will some need help from ECU vendor to find out the root-cause.


Regards
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
173 Views
Registered: ‎10-20-2019

Hello, @karnanl 

Thank you very much.

I think there are too many uncertain factors in my use-case. I will  seek for support from ECU vendor. 

And  on the other hand  it is hard to get support from the ECU vendor, So I will change my debug scenario. I connect the image sensor to a Tool which can configure the image sensor and can  also collect mipi signals and display the image. In this senario, I can configure the image sensor by myself and I can  ensure the required information.

But I  met some problems and I need your help. I posted it in another question to avoid confusion.

https://forums.xilinx.com/t5/Video-and-Audio/mipi-csi-2-rx-to-receive-mipi-signals-from-image-sensor/td-p/1121735

Best Regards!

0 Kudos
Highlighted
Adventurer
Adventurer
114 Views
Registered: ‎10-20-2019

Hello, @karnanl 

According to your explanation before, I think the primary reason is the mipi csi-2 rx ip core lost the detailed timing sequence. Is it possible that I create my own MIPI CSI-2 RX IP based on xilinx's MIPI D-PHY IP to receive and reserve the  detailed timing sequence?

Best Regards!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
92 Views
Registered: ‎03-30-2016

Hello @xxwang 

> Is it possible that I create my own MIPI CSI-2 RX IP based on xilinx's MIPI D-PHY IP to receive and reserve the detailed timing sequence?

Yes you can create your own MIPI CSI-2 RX IP using Xilinx MIPI D-PHY IP, but you are on your own.
I see a few customers are using their own MIPI CSI-2 RX IP.

Xilinx MIPI D-PHY RX IP has a standard PPI interface as defined by MIPI D-PHY specification. The IP will output any short/long packet immediately.
So, if you can preserve the short/long packet timing and send it to Xilinx MIPI D-PHY TX, MIPI D-PHY RX input and MIPI D-PHY TX output will mostly has the same timing.
Please note that Xilinx MIPI CSI-2 TX IP does not support this feature, so you will need to build both MIPI CSI-2 RX and MIPI CSI-2 TX by yourself.

Thanks
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
87 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I understand. Is it possible that xilinx provide a paid customized  project ?

Best Regards!

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
76 Views
Registered: ‎03-30-2016

Hello @xxwang 

I do not have any knowledge to answer this kind of questions.
Please talk with your FAE.

Thanks
Leo

0 Kudos
Highlighted
Adventurer
Adventurer
71 Views
Registered: ‎10-20-2019

Hello, @karnanl 

I understand. Thank you!

Best Regards!

0 Kudos