cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Observer
Observer
411 Views
Registered: ‎12-05-2009

ZynqMP PS DisplayPort baremetal live video and non-live graphics overly

Hi,

I am trying to get an OSD-like overlay over a live video from a mipi-camera. The HW and SW parts are mainly from Greg Taylor's example project:

https://github.com/gtaylormb/ultra96v2_imx219_to_displayport

I adapted it to my camera (OV9281) and added the xdpdma_video_example which I pulled in from the dpdma driver.

Now I can see the live video or the graphics overlay, but not both together. sonycman reported the same issue about a year ago and claimed he has overcome it ("...vsync and hsync have to be strictly aligned to each other..."), but I do not understand what that means exactly.

To switch from live to graphics I am changing from this line in the DP init function:

XAVBuf_InputVideoSelect(AVBufPtr, XAVBUF_VIDSTREAM1_LIVE, XAVBUF_VIDSTREAM2_NONLIVE_GFX);

to this:

XAVBuf_InputVideoSelect(AVBufPtr, XAVBUF_VIDSTREAM1_NONE, XAVBUF_VIDSTREAM2_NONLIVE_GFX);

I did expect the overplayed output for the first case, but do see a black upper half of the screen and a green lower half. (In case I enable global Alpha I get a red upper and green lower half.)

Changing the clock source from XAVBUF_PL_CLK to XAVBUF_PS_CLK in:

XAVBuf_SetAudioVideoClkSrc(AVBufPtr, XAVBUF_PL_CLK, XAVBUF_PS_CLK);

does not have any effect.

 

What am I missing here? Any help is greatly appreciated!

 

HW: Ultra96 (Avnet)

SW: Vivado/Vistis 19.2

0 Kudos
Reply
2 Replies
Observer
Observer
317 Views
Registered: ‎12-05-2009

Hi,

With a trick I m now able to show the none-live graphis blended over the live-video! The trick is set the avbuf input selection to none-live graphis only first. Then, after the clock sources are selected (the trick doesn't work before that) the avbuf input selection is changed to both, none-live graphis and live-video. The issue now is the two frames are not in sync.

Here is the live-video (PL-TPG with the mipi-camera feed partially passed-through in the center):

IMG_7075.JPG

 

 

 

 

This is my graphics overlay (50% alpha for the RGB segments, 100% transparent for the bottom segment):

IMG_7074.JPG

 

 

 

 

And here is the result (graphic is corrcet, but live-video is stable at the of sync position).

IMG_7073.JPG

 

 

 

 

And here is the main portion of the code responsible for the DP setup:

int InitDpDmaSubsystem(Run_Config *RunCfgPtr)
{
	u32 Status;
	XDpPsu		*DpPsuPtr = RunCfgPtr->DpPsuPtr;
	XDpPsu_Config	*DpPsuCfgPtr;
	XAVBuf		*AVBufPtr = RunCfgPtr->AVBufPtr;
	XDpDma_Config *DpDmaCfgPtr;
	XDpDma		*DpDmaPtr = RunCfgPtr->DpDmaPtr;


	/* Initialize DisplayPort driver. */
	DpPsuCfgPtr = XDpPsu_LookupConfig(DPPSU_DEVICE_ID);
	XDpPsu_CfgInitialize(DpPsuPtr, DpPsuCfgPtr, DpPsuCfgPtr->BaseAddr);
	/* Initialize Video Pipeline driver */
	XAVBuf_CfgInitialize(AVBufPtr, DpPsuPtr->Config.BaseAddr, AVBUF_DEVICE_ID);

	/* Initialize the DPDMA driver */
	DpDmaCfgPtr = XDpDma_LookupConfig(DPDMA_DEVICE_ID);
	XDpDma_CfgInitialize(DpDmaPtr,DpDmaCfgPtr);

	/* Initialize the DisplayPort TX core. */
	Status = XDpPsu_InitializeTx(DpPsuPtr);
	if (Status != XST_SUCCESS) {
		return XST_FAILURE;
	}
	/* Set the format graphics frame for DPDMA*/
	Status = XDpDma_SetGraphicsFormat(DpDmaPtr, RGBA8888);
	if (Status != XST_SUCCESS) {
			return XST_FAILURE;
	}
	/* Set the format graphics frame for Video Pipeline*/
	Status = XAVBuf_SetInputNonLiveGraphicsFormat(AVBufPtr, RGBA8888);
	if (Status != XST_SUCCESS) {
			return XST_FAILURE;
	}
	/* Set the QOS for Video */
	XDpDma_SetQOS(RunCfgPtr->DpDmaPtr, 11);
	/* Enable the Buffers required by Graphics Channel */
	XAVBuf_EnableGraphicsBuffers(RunCfgPtr->AVBufPtr, 1);
	/* Set the output Video Format */
	XAVBuf_SetOutputVideoFormat(AVBufPtr, RGB_8BPC);

	//NEW
	XAVBuf_SetInputLiveVideoFormat(AVBufPtr, RGB_8BPC);

	/* Enable the non-live graphics only at first
	 */
	XAVBuf_InputVideoSelect(AVBufPtr, XAVBUF_VIDSTREAM1_NONE, XAVBUF_VIDSTREAM2_NONLIVE_GFX);

	/* Configure Video pipeline for graphics channel */
	XAVBuf_ConfigureGraphicsPipeline(AVBufPtr);

	/* Configure the output video pipeline */
	XAVBuf_ConfigureOutputVideo(AVBufPtr);

	/* Disable the global alpha, since we are using the pixel based alpha */
	XAVBuf_SetBlenderAlpha(AVBufPtr, 0x0, 0);

	/* Set the clock mode */
	XDpPsu_CfgMsaEnSynchClkMode(DpPsuPtr, RunCfgPtr->EnSynchClkMode);

	XDpPsu_CfgMsaEnSynchClkMode(DpPsuPtr, 1);
	/* Set the clock source depending on the use case.*/
	XAVBuf_SetAudioVideoClkSrc(AVBufPtr, XAVBUF_PL_CLK, XAVBUF_PS_CLK);

    /* IF live-video is enabled only now the overlay works, thought it is not in sync */
	XAVBuf_InputVideoSelect(AVBufPtr, XAVBUF_VIDSTREAM1_LIVE, XAVBUF_VIDSTREAM2_NONLIVE_GFX);

	/* Issue a soft reset after selecting the input clock sources */
	XAVBuf_SoftReset(AVBufPtr);

	return XST_SUCCESS;
}

And here the block diagram:

Bildschirmfoto vom 2020-11-18 16_01_57.png

 

One more observation: I was checking the DPDMA_CH3_CNTL Register after I applied the trick and it shows the HALT bit not set:

xsct% mrd 0xFD4C0618
FD4C0618: 00002EEC

When I check it without the trick the HALT bit is set:

FD4C0618: 00002EEF

 

Best Regards,

Thomas

Observer
Observer
268 Views
Registered: ‎12-05-2009

Hi,

After some more debug I decided to give up on the live-video input in combination with nonlive-graphics. I changed the design to both nonlive-video and nonlive-gaphics. It actually saves all the frmbufrd, vtc and axis-to-video converter logic. I need the grapcs triple buffered but I am sure I can handle this with the dpdma controller and the available interrupts.

In case somebody comes to the same point and ultimately needs the live-video input plus the nonlive graphics, here some more hints I found without getting it completely working:

Setting break points in xdpdma_intr.c:XDpDma_VSyncHandler(XDpDma *InstancePtr), than the various if(...) statements and stepping slowly (give the monitor time to settle) from breakpoint to break point, you can actually see how first the video comes active and then, for s single frame only, the graphics overly is visible all correct and in sync. After this single flash of hope DPDMA_CH3_CNTL is indicating the graphics channels is disabled:

xsct% mrd 0xFD4C0518
FD4C0518: 00002EEF

and DPDMA_ISR is indicating some errors:

xsct% mrd 0xFD4C0004
FD4C0004: 08240240

both registers showed good values just before.

Good luck in finding the last missing bits...

Thomas