cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Adventurer
Adventurer
946 Views
Registered: ‎06-05-2019

D-PHY module can output signal but no video_out in mipi csi example design

Jump to solution

I used the mipi csi example design on my ZCU102 board and replaced the LI-IMX274MIPI with my own sensor ar0144. The sensor has been configured and I can get the image from this sensor on the other image development boards.

When i connected the sensor to the ZCU102, I can monitor the output from D-PHY module following the guidance https://forums.xilinx.com/t5/Video/monitor-mipi-data-regardless-mipi-clk-with-ILA/m-p/1032960#M28376 . But the rx subsytem had output according to the vedio_out monitored by ila

video_aclk and dphy_clk input are connected as same as the original example design.

微信截图_20191019214728.png

 

微信截图_20191019223851.png

 

Tags (2)
0 Kudos
1 Solution

Accepted Solutions
Highlighted
Observer
Observer
712 Views
Registered: ‎04-26-2019

Hi @ivanfan ,

I've been meaning to respond sooner, but have been occupied with other items.

In my case, there were two primary things that I had to do to successfully stream data out of the CSI2-RX video_out port:

  • Preventing buffer overflow inside the CSI2-RX block by making sure my video_aclk frequency was sufficiently larger than my MIPI interface clock
    • (solved by provisioning the external camera sensor to use a div-by-2 version of the sensor MCLK I was providing it)
  • Filtering the video_out_tuser output of the CSI2-RX block to constrain it to a single clock cycle wide (instead of 4 clock cycles wide, in my case)
    • (solved by making a tiny RTL block that "shaped" video_out_tuser to a single clock cycle wide)

A significant "clue" in my project was that I was only getting 580 pixels per horizontal line out of the CSI2-RX subsystem, instead of the 640 pixels per horizontal line that I was provisioning the camera sensor with, which implied that the CSI2-RX subsystem was not able to keep up with the MIPI interface data rate (buffer overflow).

I'm not sure I made any specific changes to the CSI2-RX block to solve my issue; all of my settings seem to mesh intuitively with my specific application.

My CSI2-RX settings are:

  • Pixel Format = RAW8
  • Serial Data Lanes = 2
  • Include Video Format Bridge (VFB) = checked
  • Line Rate (Mbps) = 840
  • Filter User Defined data types = checked
  • Line Buffer Depth = 4096
  • Allowed VC  =  All (default/auto)
  • Pixels Per Clock = 1
  • TUSER Width = 1
  • Enable CRC = checked
  • Enable Active Lanes = checked

I've highlighted a few settings above in red that would be good to scrutinize for your particular project, just to double check that you have the desired configuration in place.

For the SDK piece, I believe it is important to provision each of your blocks from an inside-first to outside-last sequence.  In my application this meant that I provisioned my devices in this order:

  1. Internal video processing IP registers/interrupts
  2. CSI-2 RX Subsystem registers/interrupts
  3. External camera sensor registers

The last provisioning step basically tells the camera sensor to "go", at which point it starts transfering video packets into the Zynq MPSoC.

Expanding on the 3 steps above, here's a more detailed "main()" subroutine sequence that seems to be working for me (though still in summary form); the color-coded scheme above (pink, green, blue) is re-used below to explain which subroutines go with which steps (subroutines in black are IIC or interrupt-related):

  1. Configure_Custom_IP()
  2. Intialize_IIC()
  3. Initialize_IIC_Intr()
  4. Initialize_XScuGic_Stuff()  (more interrupt stuff; includes  XScuGic_Connect(), XScuGic_Enable(), XScuGic_SetPriorityTriggerType() function calls)
  5. Initialize_CsrRxSs()
  6. Enable_CSI()
  7. Initialize_Camera_Sensor()
  8. Start_Camera_Sensor()

 

I hope this info helps.  Good luck with your project!

 

Kind Regards,

-8bitmode

 

 

 

 

View solution in original post

10 Replies
Highlighted
Xilinx Employee
Xilinx Employee
892 Views
Registered: ‎03-30-2016

Hello @ivanfan 
I hope you are doing fine.

Please let me clarify.
    You see that MIPI D-PHY RX PPI interface output is toggling but
    your MIPI CSI-2 RX Subsystem video_out_tvalid is never asserted (fixed to zero).

If this is the case, could you please share MIPI CSI-2 RX Subsystem register dump ?
Please share , so we can double check for you.
CSI2_RX.png

Regards
Leo

Highlighted
Adventurer
Adventurer
852 Views
Registered: ‎06-05-2019

I used Xil_In32() to read the value of registers. According the value of Core Status Register, it seems that the mipi core did not receive any packet. 

Any help regarding the issue will be appreciated.

微信截图_20191022170229.png

0 Kudos
Highlighted
Adventurer
Adventurer
834 Views
Registered: ‎06-05-2019

The status of D-PHY register are shown in the following picture:

微信截图_20191022220308.png

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
811 Views
Registered: ‎03-30-2016

Hello Ivan @ivanfan 

Thank you for sharing your register dump.

Reading your MIPI CSI-2 RX register :

0x00Core is enabled
0x042 lane setting all enabled
0x10No Packet received. Need to confirm D-PHY data lanes status 
0x24Data lanes are in stop-states


Yes, your understanding is correct that MIPI CSI-2 RX core does not receive any packet.

Reading your MIPI D-PHY RX register :

0x00D-PHY is enabled
0x18INIT_DONE=0, MODE=HS_MODE ( clock lane)
0x1CINIT_DONE=0, MODE=LP_MODE (Data lane-0)
0x20INIT_DONE=0, MODE=LP_MODE (Data lane-1)
0x24INIT_DONE=0, MODE=LP_MODE (Data lane-2)
0x28INIT_DONE=0, MODE=LP_MODE (Data lane-3)


It seems that your clock lane is okay,
but your data lanes is in LP mode.

Could you please sample the MIPI D-PHY register to find out whether the data lanes is fixed in LP mode ?
or is it toggling between LP/HS mode ?
# I am suspecting that your sensor is sending a fixed LP-11.

Thanks
Leo

Highlighted
Adventurer
Adventurer
802 Views
Registered: ‎06-05-2019

Thank you for your reply.

The pictures I posted above are created by sampling the the MIPI D-PHY register every 100ms.

The value of DL0_STATUS register(0x1C) is 0x992F0009. It should be that only the data lane-0 works in HS_MODE and the other 3 data lane is fixed in LP_MODE. 

0 Kudos
Highlighted
Xilinx Employee
Xilinx Employee
792 Views
Registered: ‎03-30-2016

Hello @ivanfan 

>The value of DL0_STATUS register(0x1C) is 0x992F0009.
>It should be that only the data lane-0 works in HS_MODE and the other 3 data lane is fixed in LP_MODE.

Opps Sorry, Reading your MIPI D-PHY RX register :

0x00D-PHY is enabled
0x18INIT_DONE=0, MODE=HS_MODE ( clock lane)
0x1CINIT_DONE=0, MODE=HS_MODE (Data lane-0)
0x20INIT_DONE=0, MODE=LP_MODE (Data lane-1)
0x24INIT_DONE=0, MODE=LP_MODE (Data lane-2)
0x28INIT_DONE=0, MODE=LP_MODE (Data lane-3)



Question:
Your MIPI CSI-2 RX setting is using 2 lanes setting.
Why Data Lane-1 is fixed at LP-mode ?


I do not understand.
Thanks & regards
Leo

Highlighted
Adventurer
Adventurer
777 Views
Registered: ‎06-05-2019

I also want to find out the reason.

I can see that the data0 and data1 pins of the sensor have data passing through with the external oscilloscope.

0 Kudos
Highlighted
Observer
Observer
713 Views
Registered: ‎04-26-2019

Hi @ivanfan ,

I've been meaning to respond sooner, but have been occupied with other items.

In my case, there were two primary things that I had to do to successfully stream data out of the CSI2-RX video_out port:

  • Preventing buffer overflow inside the CSI2-RX block by making sure my video_aclk frequency was sufficiently larger than my MIPI interface clock
    • (solved by provisioning the external camera sensor to use a div-by-2 version of the sensor MCLK I was providing it)
  • Filtering the video_out_tuser output of the CSI2-RX block to constrain it to a single clock cycle wide (instead of 4 clock cycles wide, in my case)
    • (solved by making a tiny RTL block that "shaped" video_out_tuser to a single clock cycle wide)

A significant "clue" in my project was that I was only getting 580 pixels per horizontal line out of the CSI2-RX subsystem, instead of the 640 pixels per horizontal line that I was provisioning the camera sensor with, which implied that the CSI2-RX subsystem was not able to keep up with the MIPI interface data rate (buffer overflow).

I'm not sure I made any specific changes to the CSI2-RX block to solve my issue; all of my settings seem to mesh intuitively with my specific application.

My CSI2-RX settings are:

  • Pixel Format = RAW8
  • Serial Data Lanes = 2
  • Include Video Format Bridge (VFB) = checked
  • Line Rate (Mbps) = 840
  • Filter User Defined data types = checked
  • Line Buffer Depth = 4096
  • Allowed VC  =  All (default/auto)
  • Pixels Per Clock = 1
  • TUSER Width = 1
  • Enable CRC = checked
  • Enable Active Lanes = checked

I've highlighted a few settings above in red that would be good to scrutinize for your particular project, just to double check that you have the desired configuration in place.

For the SDK piece, I believe it is important to provision each of your blocks from an inside-first to outside-last sequence.  In my application this meant that I provisioned my devices in this order:

  1. Internal video processing IP registers/interrupts
  2. CSI-2 RX Subsystem registers/interrupts
  3. External camera sensor registers

The last provisioning step basically tells the camera sensor to "go", at which point it starts transfering video packets into the Zynq MPSoC.

Expanding on the 3 steps above, here's a more detailed "main()" subroutine sequence that seems to be working for me (though still in summary form); the color-coded scheme above (pink, green, blue) is re-used below to explain which subroutines go with which steps (subroutines in black are IIC or interrupt-related):

  1. Configure_Custom_IP()
  2. Intialize_IIC()
  3. Initialize_IIC_Intr()
  4. Initialize_XScuGic_Stuff()  (more interrupt stuff; includes  XScuGic_Connect(), XScuGic_Enable(), XScuGic_SetPriorityTriggerType() function calls)
  5. Initialize_CsrRxSs()
  6. Enable_CSI()
  7. Initialize_Camera_Sensor()
  8. Start_Camera_Sensor()

 

I hope this info helps.  Good luck with your project!

 

Kind Regards,

-8bitmode

 

 

 

 

View solution in original post

Highlighted
Adventurer
Adventurer
679 Views
Registered: ‎06-05-2019

Thank you very much for your detailed answer.

What surprised me is that after the sensor register configuration is completed, I pull out the line of mipi clk from the fmc socket and reconnect it, the DL1 can receive packet and work in HS_MODE. And I can get the image output from HDMI. I can not find out the reason. Anyway, the project can finally work and output the image.

Thank you again for your help.

Highlighted
Moderator
Moderator
587 Views
Registered: ‎11-21-2018

Hi @ivanfan 

It sounds like you found a solution to your problem. 

If this is the case, can you mark the response which helped as a solution (click on "Accept as solution" button below the reply).

If this is not solved/answered, please reply in the topic giving more information on your current status.

Thanks and Regards,

Aoife
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos