cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
parthshah76
Observer
Observer
1,197 Views
Registered: ‎10-24-2018

How to transmit Monochrome sensor 14 bit raw data using MIPI CSI-2 TX Controller IP core

Jump to solution

Objective:

Transmission of Non-MIPI camera sensor data (i.e. any monochrome sensor) using MIPI CSI-2 TX Controller Ip core from on one Arty7 Board to another Arty7 board as MIPI CSI-Rx system or any MIPI Receiver based Video processor.

IP core Application as per document:

The Xilinx MIPI CSI-2 TX Controller implements camera sensor transmitter interface over MIPI D-PHY Interface. It can be used to bridge between non-MIPI camera sensors to MIPI based video processor.  This Core provides combination of MIPI CSI-2 transmit controller and D-PHY interface.

System Overview as per our understanding:

  •  After power on, Clock generator IP core generates different clock required for FPGA subsystems. (Status : Working)
  • FPGA configures camera sensor by providing appropriate commands via I2C Interface.(Status : Working)
  • After successful configuration, Sensor generates PSYNC (Pixel Clock), HSYNC (Line Clock), VSYNC (Frame clock) and 14-bit video data which are input to FPGA.  (Status : Working)
  • Since sensor provides Non-MIPI output signals and Xilinx MIPI CSI-2 Tx doesn’t have monochrome RAW data input, one complete control system required in FPGA which convert these sensors output signals in to MIPI CSI-2 TX Core compatible data and control signals. (Status :  How to give above signals to this IP core, After implementing required logic we are not getting any output on this IP core)
  • MIPI CSI-2 Tx controller core has In-built physical layer which generated D-PHY outputs. (Status : Not outputs on D-PHY)
  • D-PHY output can be accessible to another Arty-7 board as MIPI CSI-2 Rx or any MIPI based video processor to perform further post-processing.
  • Above description is given in block diagram and sensor waveform
  • Sensor specification: image resolution 384 x 288 @ 30 FPS with 14 bit RAW video data.

Problem faced sensor provides Non-MIPI output signals and Xilinx MIPI CSI-2 Tx doesn’t have monochrome RAW data input, So we are not able generate any D-PHY output after configuration. No examples related to RAW data of 8 or 14 bit.

References:

1.      MIPI CSI Controller Subsystems –

https://www.xilinx.com/products/intellectual-property/ef-di-mipi-csi-rx.html#overview

2.      MIPI CSI-2  Transmitter Subsystem  V2.1 - 

https://www.xilinx.com/support/documentation/ip_documentation/mipi_csi2_tx_subsystem/v2_1/pg260-mipi-csi2-tx.pdf

Tags (2)
Block_Diagram.jpg
Sensor_Output.png
0 Kudos
1 Solution

Accepted Solutions
karnanl
Xilinx Employee
Xilinx Employee
1,106 Views
Registered: ‎03-30-2016

Hello @parthshah76 

Sorry, I got confused. Please let me clarify my previous answer.

If you want to feed RAW8 data,
    assign your data into s_axis_tdata[7:0]
    set s_axis_tdata[23:8] to a fixed zeros.

If you want to feed RAW14 data,
    assign your data into s_axis_tdata[13:0]
    set s_axis_tdata[47:14] to a fixed zeros.

Hope this helps.

Thanks,
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------

View solution in original post

0 Kudos
11 Replies
karnanl
Xilinx Employee
Xilinx Employee
1,178 Views
Registered: ‎03-30-2016

Hello @parthshah76 

> Since sensor provides Non-MIPI output signals and Xilinx MIPI CSI-2 Tx doesn’t have monochrome RAW data input, one complete control system required in FPGA which convert these sensors output signals in to MIPI CSI-2 TX Core compatible data and control signals. (Status : How to give above signals to this IP core, After implementing required logic we are not getting any output on this IP core)
> MIPI CSI-2 Tx controller core has In-built physical layer which generated D-PHY outputs. (Status : Not outputs on D-PHY)
>Problem faced: sensor provides Non-MIPI output signals and Xilinx MIPI CSI-2 Tx doesn’t have monochrome RAW data input, So we are not able generate any D-PHY output after configuration. No examples related to RAW data of 8 or 14 bit.

MIPI CSI-2 TX Subsystem supports RAW8/10/12/14 data input.
If you could change your system requirement to send RAW8/RAW10 data , please set correct value in s_axis_tuser[6:1].
     RAW8 : 0x2A
     RAW10 : 0x2B
     RAW12 : 0x2C    
     RAW14 : 0x2D
MIPI_CSI2_TX_Data_Type.png

Kind regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
parthshah76
Observer
Observer
1,172 Views
Registered: ‎10-24-2018

Yes, We tried to changed the same but How to adjust Data Width?

Since our data size is 14 bit per pixel clock, but Pixel Encoding Shows 42 bit as minimum width.

Now we tried with 8 bit Raw , which results in 24 bit as minimum width. ( 14 bit data + 10 bit Padded by zero)

Is there any way to get it done ?

How many registers we need to configure using AXI lite interface ?

Pixel_Encoding.png
0 Kudos
karnanl
Xilinx Employee
Xilinx Employee
1,148 Views
Registered: ‎03-30-2016

Hello @parthshah76 

If you set "Maximum Bits per Component"=8 bits, s_axis_tdata will be 24bit. See the following rule :
       Data Width = Byte aligned of ("Maximum Bits per Component" x 3 x "Pixel Per Clock")

If you want to feed RAW8 data,
      assign your data into s_axis_tdata[7:0]
      set s_axis_tdata[23:8] to a fixed zeros.

BTW, Input pixel interface of MIPI CSI-2 TX is following data format guidance mentioned in UG934.
      https://www.xilinx.com/support/documentation/ip_documentation/axi_videoip/v1_0/ug934_axi_videoIP.pdf

Kind regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
karnanl
Xilinx Employee
Xilinx Employee
1,107 Views
Registered: ‎03-30-2016

Hello @parthshah76 

Sorry, I got confused. Please let me clarify my previous answer.

If you want to feed RAW8 data,
    assign your data into s_axis_tdata[7:0]
    set s_axis_tdata[23:8] to a fixed zeros.

If you want to feed RAW14 data,
    assign your data into s_axis_tdata[13:0]
    set s_axis_tdata[47:14] to a fixed zeros.

Hope this helps.

Thanks,
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------

View solution in original post

0 Kudos
Arpita_06
Newbie
Newbie
1,016 Views
Registered: ‎12-08-2020

We have created complete block design for resolution of 384 x 288 with 14 bit data coming from sensor and here are we are facing certain issues given below :

1.After sending video data from Video in AXI to S-Axis of MIPI there is no output and S axis-t-ready is zero.      

Given values in t-user [95:0]

  • 95-64 Reserved 63-48
  • Word count = 384
  • 47-32 Line number = 288 packets ( At every positive edge of HSYNC)
  •  31-16 Frame number = every number incremented at every positive edge of VSYNC
  • 6-1 Data type = 2D
  • 0- Frame start = m-axis-video-t-user

2.While I tried using AXI traffic controller with MIPI CSI 2 for interfacing with S – axi lite as per given in example design, but only MIPI-PHY-HS clocks are generated, no changes in MIPI PHY data

3.While studying example design given for MIPI CSI 2, we are not able to identify data of MIPI-PHY and how this interface is working.

We would like share our design with Functional simulation , Is there any way to share you project or with Email?

karnanl
Xilinx Employee
Xilinx Employee
1,007 Views
Registered: ‎03-30-2016

Hello @Arpita_06 

Please do not add your question to other people's post.
Please post a new question instead.
BTW, if s_axis_tready stucked to 0, MIPI CSI-2 TX core cannot send/receive any data.

So,
1. Did you apply the reset sequence correctly ?
TX_reset_seq.png

2. Please check if s_axis_aclk and dphy_clk_200M are free-run and stable when you initialize the IP.
   tready_is_stucked_low.png

3. Please share your XCI file
4. Please share your MIPI CSI-2 TX register dump

Kind regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
parthshah76
Observer
Observer
889 Views
Registered: ‎10-24-2018

Thank you for your response. Previous question was asked by my team member since i was facing issue for posting from my username.

we are able to get the output on D-PHY. We are able to simulate design using behavioral simulation. but we are facing issue for implementation which is described below :

 

  • Currently we are using Arty-7 Development board by Digilent which has Artix-7 FPGA and This board has PMOD connectors high speed and low-speed both. Currently we are using  output of 1 lane only. So overall outputs are as given below :

                               mipi_phy_if_0_clk_hs_n,
                               mipi_phy_if_0_clk_hs_p,
                               mipi_phy_if_0_clk_lp_n,
                               mipi_phy_if_0_clk_lp_p, 
                               mipi_phy_if_0_data_hs_n,
                               mipi_phy_if_0_data_hs_p,
                               mipi_phy_if_0_data_lp_n,
                               mipi_phy_if_0_data_lp_p,

  • How to define XDC file for MIPI outputs in this Artix-7 FPGA? What should be I/O standard for above outputs.  We are getting implementation error related to differential and single ended.
  • On page number -6 of user guide , it is mentioned that external D-Phy chip or resistive circuit is needed for I/O implementation. We do not have idea about such circuit. Can you share the details on the same . Can't we directly drive the outputs of D-PHY by defining I/O standards?
  • MIPI alliance has CCI interface which is I2C for communication with Transmitter. In this design, we do not know how Receiver system like processor communicate with Transmitter? 
  • Can you provide us link for MIPI CSI-2 Specification document PDF.?
0 Kudos
karnanl
Xilinx Employee
Xilinx Employee
879 Views
Registered: ‎03-30-2016

Hello @parthshah76 

>How to define XDC file for MIPI outputs in this Artix-7 FPGA? What should be I/O standard for above outputs. We are getting implementation error related to differential and single ended.
>On page number -6 of user guide , it is mentioned that external D-Phy chip or resistive circuit is needed for I/O implementation. We do not have idea about such circuit. Can you share the details on the same . Can't we directly drive the outputs of D-PHY by defining I/O standards?

As you already aware Xilinx 7-series device IO does not natively support MIPI D-PHY spec, so you need to add a converter on your board.
Please check XAPP894 (https://www.xilinx.com/support/documentation/application_notes/xapp894-d-phy-solutions.pdf) that provides a basic guidance on implementing resistor network for 7-series devices.

This XAPP suggest to use LVCMOS18 for LP pins and DIFF_HSTL_I_18 for HS pins.

XAPP894_Fig10.png

[DIFF_HSTL_I_18]
    mipi_phy_if_0_clk_hs_n,
    mipi_phy_if_0_clk_hs_p,
    mipi_phy_if_0_data_hs_n,
    mipi_phy_if_0_data_hs_p,
[LVCMOS18]
    mipi_phy_if_0_clk_lp_n,
    mipi_phy_if_0_clk_lp_p,
    mipi_phy_if_0_data_lp_n,
    mipi_phy_if_0_data_lp_p,

>MIPI alliance has CCI interface which is I2C for communication with Transmitter. In this design, we do not know how Receiver system like processor communicate with Transmitter?

CCI is I2C lines to control Sensor/Camera.
Our MIPI CSI-2 TX Subsystem does not require control from I2C interface, all register control for MIPI CSI-2 TX can be done via AXI4-lite interface.

> Can you provide us link for MIPI CSI-2 Specification document PDF.?

You can get MIPI CSI-2 spec from mipi.org. (Membership required)
This is the link : https://members.mipi.org/wg/All-Members/document/folder/8325

Thanks & regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
parthshah76
Observer
Observer
797 Views
Registered: ‎10-24-2018

Thanks for sharing I/O standard.

After using LVCMOS18 for LP pins and DIFF_HSTL_I_18 for HS pins in XDC file we are getting output as shown in Image - A for behavioural simulation only but in post implementation timing simulation shown in Image-B, mipi_phy clocks are generating but we are not getting output of mipi_phy_data.

Values of mipi_phy_data are:

mipi_phy_if_0_data_hs_n = 0,

mipi_phy_if_0_data_hs_p = x,

mipi_phy_if_0_data_lp_n = x,

mipi_phy_if_0_data_lp_p = 0

And after checking internal signals of MIPI CSI transmitter, they are showing not showing values like txdatahs, txreadyhs etc. shown in Image-C

 

You can see these results in below image- A, Image-B and Image - C. How can we Solve this implementation results and Can you please guide us?

Image-A.png
Image-B.png
Image-C.png
0 Kudos
karnanl
Xilinx Employee
Xilinx Employee
689 Views
Registered: ‎03-30-2016

Hello @parthshah76 

I tried to run MIPI CSI-2 TX Subsystem Example Design simulation just now.
I found that both Post-Synthesis Functional and Timing simulation are running just fine. (see below)
MIPI_TX_ExDes_2020.2_sim_are_running_just_fine_for_func_and_tmg.png

So, I believe there is nothing wrong with the MIPI IP itself.
Perhaps "Simulation board" can give you a better hint to solve your issue.

Regards
Leo


------------------------------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
If starting with Versal take a look at our Versal Design Process Hub and our Versal Blogs
Versal Example Designs : LINK
>>------------------------------------------------------------------------------------------------
0 Kudos
parthshah76
Observer
Observer
605 Views
Registered: ‎10-24-2018

Hello Karnani,

We tried implementing example design again in which are getting behavioral simulation correct while post-implementation timing results not showing anything.

Kindly suggest us how we can proceed. is it related to VIVADO tool related any dependency or settings?

 

Here by we are attaching snapshot for the same.

mipi_behave_sim.png
mipi_behave_sim1.png
mipi_ex_timing_sim.png
mipi_ex_timing_sim1.png
0 Kudos