cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
gcsimmonsjr
Adventurer
Adventurer
3,568 Views
Registered: ‎05-28-2018

VPSS pipeline and Deinterlacer causing problems

Jump to solution

 

We have the VPSS Full-flegged system working to a point, but we're seeing some interesting issues.

First, if we put a progressive 1080P@60/RGB video input, and configure the VPSS for 1080P@60/RGB Video In, and 1080P@60/RGB Video Out. The picture is stable, the colors look good, but we get a column banding artifact. The video will run long term without the pipeline stalling or crashing. In this case, the Deinterlacer is bypassed.

Leaf.png

Second, if we put in a 1080i@60/RGB Video Input, and configure the VPSS for a 1080P@60/RBG Video Output, we get:

Leaf2.png

You can see that there is data corruption in the frame and the frame is partially duplicated. Also, in only seconds, the VPSS locks up, and sometimes a full FPGA re-load is required, not just restarting the Microblaze code.

I am postulating that the buffers allocated to the Deinterlacer are interfereing with the buffers allocated to the VDMA. We've looked at the code, and it hard to tell how the VPSS mananges a large 256Mx32bit (0x8000_0000-8FFF_FFFF) memory resource. It's plenty BIG enough for the 5 1080P frames + 3 1080i frames that the datasheet calls out (10-bit RGB color.)

Any clues as to what might be happening here?   Where's the banding coming from?  Why is the Deinterlacer SO unhappy? We think it may be the Deinterlacer walking all over the VDMA buffers, or vise versa.

Then, we changed the Video Input to 720P@60/RBG and upscaled to 1080P@60/RBG, and LOOK what it did!

Leaf3.png

What in the WORLD is going on?

 

 

0 Kudos
44 Replies
florentw
Moderator
Moderator
1,171 Views
Registered: ‎11-09-2015

Hi @reaiken and @gcsimmonsjr ,

I will need some time to investigate. I was planing to do an example as part as my video series soon anyway ;)

The only thing I can point you to is the xapp1291 which is using the deintelacer feature of the VPSS.

Also some observations which can help:

  • In 2018.3 version in which the interlaced support was introduced for the TPG...the TPG does not have the correct behaviour as it is not following the UG934 (we noticed that the fid is changed every line not every field). So do not believe this should be the correct reference. I need to check if it is fixed in 2019.1
  • Then, as @reaiken mentioned, in the PG231, the deinterlacer is expecting the SOF signal only once every 2 field. However this is not coherent with the figure 2-11 of UG934

I am currently quite busy, so I do not think I will be able to do a design next week


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
phil@pswitchers.com
Participant
Participant
1,151 Views
Registered: ‎01-21-2019

@florentw,

We are at a bit of a stand-still here.

Can you guys at Xilinx Dev shed some light on why the VPSS Deint is not keeping up with a 1080i@60 video stream?

Regards,

Phil

(on behalf of @reaiken @gcsimmonsjr )

0 Kudos
gcsimmonsjr
Adventurer
Adventurer
1,140 Views
Registered: ‎05-28-2018


We improved several things in our Aurora-to-Video IP block, and it is now much more efficient at feeding data into the Deinterlacer. However, we still see some extreme behavior where the Deinterlacer is pausing the stream for nearly an entire line time of 1080i60, causing the loss of a significant amount of video pixel data and messing up line counts and frame sizes. As you probably know, the Aurora RX block does not provide a TREADY and does not allow throttling of the data stream, so the Deinterlacer really can only pause the stream for the horizontal blanking period of time before bad things start happening.

Considering that the TPG with supposed interlaced support was relased in a legitimate S/W release, and the fact that this is an HLS block (like the VPSS,) it feels like someone is just hacking code for these blocks and not really, thoroughly, testing these before release, realistically causing these video IP blocks to fall into question, if we cannot trust that they've been completely vetted by Xilinx.

Not trying to bash on you guys, but we're basing real products on this code!  If we end up writing our own IP, then please explain to me the value here?  We've been constantly wrestling with issues trying to support interlaced video with Xilinx Video IP for these past 6 months.

0 Kudos
reaiken
Explorer
Explorer
1,103 Views
Registered: ‎07-18-2011

Here's a thought...

Have you tried modifying XAPP1291 to fit your system?   It alledgely works at 1080i60.

Unfortunately, it uses a KC705 eval board which has a Kintex and HDMI transceivers on it, so the design will have to be heavily modified to fit the Artix and whatever video I/O you are using, but it may shed some light on our dilemma if you can get the software and basic hardware to work on the Artix.

Perhaps the Kintex 200MHz AXI bus at 2 pixels per clock makes it work on the eval board, and the Artix 100MHz AXI bus at 2 pixels per clock is too slow to handle the pipeline?  Perhaps switching to 4 pixels per clock on the video path might help?

 

0 Kudos
gcsimmonsjr
Adventurer
Adventurer
1,092 Views
Registered: ‎05-28-2018


Yes, interesting thought. We actually had access to a KC705 board yarns ago, but you have to have a subscription to Xilinx Full-Up tools in order to even use it!  It comes with a 1-year subscription to Vivato Full Tools, but of course, it expired 2 years ago.

Some NEW news. We decided on a new experiment, which has yeilded some interesting results. No, it DID not fix the problem, but I think confirms that the Deinterlacer is SLOW and introduces FRAMES of latency.

We added an ADDITIONAL VDMA to the video stream BEFORE the Deinterlacer, so VDMA in front and VMDA behind. This will allow us to have a 3-frame latency delay to the Deinterlacer, and allow it to take the data pretty much as slow as it wants. All those line artifacts dissappeared, but now, with motion video, we have this REALLY interesting "ghosting" effect!!  We have 3 test videos that are mostly static frames with one object moving. We see the Deinterlacer create "good" frames, but then we get this ghost of the moving object, that will freeze in place for seconds, then move forward and freeze again! The "solid" verision of the object appears to be moving and looks somewhat OK.

So ... what do we know by this?  Probably nothing for sure, BUT, it seems to confirm our suspicions that the Deinterlacer CANNOT keep up with line-rate video - 1080i60 --> 1080p60.

0 Kudos
phil@pswitchers.com
Participant
Participant
1,070 Views
Registered: ‎01-21-2019

With the VDMA to Deint to VDMA configuration @gcsimmonsjr was just referencing, here is a video showing a 1080i clip going into a 5 frame VDMA on the input side and a 3 frame VDMA on the output side.

0 Kudos
florentw
Moderator
Moderator
1,058 Views
Registered: ‎11-09-2015

Hi phil@pswitchers.com and @gcsimmonsjr 

The good point here is that you have the VPSS working even if there is a limitation. I need to reproduce your issue to investigate. Does any of you have a design on a Xilinx board (with HDMI FMC or on board HDMI RX (ZCU10x board)) that you could share? I would help to speed up things.

@reaiken Just to give some updates on what I found about the VPSS and TPG

  • The TPG has interlaced support from 2018.3 however in 2018.3 the fid signal is asserted every line not every field. This is an issue which is fixed in 2019.1. I wrote an Answer Record about this (AR#72428)
  • In interlaced mode, you need to devide the VSIZE by 2 to output the correct output. Ex. to send 1920*1080i60 you would need to set VSIZE (or heigh) to 540. This does not makes sense to me but I need to convince development. But for the current versions, this is the way it is. I will write an AR about this
  • In the VPSS PG, we can see that the tuser at the input of the VPSS needs to be asserted only for odd fields. This a mistake in the PG and not following the UG934. The VPSS expects tuser to be asserted every field.

I will continue my investigation on this


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos
gcsimmonsjr
Adventurer
Adventurer
1,049 Views
Registered: ‎05-28-2018


Hmmm.  I wouldn't say "working" per se. We really have NO idea where these "ghosting" images are coming from.  With the extensive experience we had with the VDMAs, I can't say the issue is coming from them - which, again, leaves the Deinterlacer.

We had access to a KC705 board with an Invrevium HMDI FMC about 2 years ago, but we've since given that back to the FAE who loaned it to us. I could see about postentially getting it back.

0 Kudos
reaiken
Explorer
Explorer
1,043 Views
Registered: ‎07-18-2011

@florentw

The only eval board I have is a Digilent Nexys Video board, all my current designs are on custom boards for avionics displays.  If you have access to a Nexys Video card, I can make a design for it and send it to you.

@gcsimmonsjr   phil@pswitchers.com 

The "ghosting" is likely caused by the input VDMA frame sync drop/repeat of fields when the deinterlacer can't keep up with the 1080p60 output stream.  Monitor your field IDs to see what is going on (I believe they are in Gray code, so take that into account), and check how you have the frame sync and genlock set up on the VDMAs.

Unfortunately, even if an input VDMA fixes the deinterlacing issue, that is a steep price to pay, as it takes a lot of DDR bandwidth to run in and out, especially when you have multiple inputs. 

I'm working on another issue right now, but I'll have to get back on the deinterlacer very shortly for a product I have that is overdue in getting released because of this issue, so I'll try the 4 pixels per clock to see if it helps, unless you have a chance to try it first.

 

 

0 Kudos
gcsimmonsjr
Adventurer
Adventurer
1,035 Views
Registered: ‎05-28-2018


Couldn't agree with you more on the cost!

The 4-pixels per clock is an interesting thought, we're currently doing 2-pixels per clock. The interesting thing is that if you look at the input side to the Deinterlacer, it's really only taking, net, 1-pixel per clock. It digests 2 pixels, and then drops tready for a clock cycle, then digests another 2 pixels, and drops tready again.

It would be interesting to see if the Deinterlacer digests 4 pixels, and then drops tready for 3 clock cycles before moving on to the next set!

0 Kudos
phil@pswitchers.com
Participant
Participant
1,017 Views
Registered: ‎01-21-2019

@reaiken 

RE: Ghosting

I see your logic in thinking the VDMA is the cause and you very well may be correct.

Here is what puzzles me:  In order for the ghosted frame to remain visible for 1-2 seconds like this, the system would need to output an old frame 60-120 times consecutively before the frame is updated. It is almost like a single frame buffer is not being updated by the incoming stream. For example, frame 1 and frame 2 are storing incoming video data but frame 3 is being ignored. During the output from the VDMA to the Deint, all 3 are being transmitted with the 3rd frame displaying the ghosted image.

 

0 Kudos
reaiken
Explorer
Explorer
1,015 Views
Registered: ‎07-18-2011

phil@pswitchers.com 

Yes, I suspect that if you watch the field ID sequence, you will see it repeating the same field and then jumping ahead when it finally gets a good read/write to that field.

If the deinterlacer is throttling the data beyond a field boundary, it could mess up the field sequence.  Depending on how you have your genlock mode set up, it may stay parked awhile, then jump ahead a few fields, or replay old data after a good field write/read.

 

0 Kudos
gcsimmonsjr
Adventurer
Adventurer
2,736 Views
Registered: ‎05-28-2018


So, I think we have determined a couple of things may be causing the ghosting. 1) we had an output Video Clock that was twice as fast as it should have been. 1080i60 input, and our output clock was 148.50MHz and it should have been 74.25MHz.  2) Also, I had the VDMA Buffering our input stream in Dynamic Master --> Dynamic Slave, and the VDMA after the Deinterlacer was configured as Dynamic-Slave --> Dynamic-Master. Both of these issues added up to that weird "ghosting" issue.

We've fixed both of these issues and our Deinterlaced Video is working well now.

We've also determined that if you try putting the Deinterlacer in bypass mode with an Interlaced signal coming in, it works. If you try to put a progressive signal into the Deinterlacer (and you fix that stupid Deinterlacer configuration checker routine so that it doesn't fail with a progressive input to the Deinterlacer Only,) it will crash the Deinterlacer.    

View solution in original post

reaiken
Explorer
Explorer
868 Views
Registered: ‎07-18-2011

@gcsimmonsjr 

That all makes sense, glad you got it sorted out.  The dynamic master/slave configuration controls which frames get "stepped over" when reading or writing, so you were getting a lot of old data mixed in with new data as the two VDMAs were fighting each other.

That still leaves the big question:   Is there any way to make the deinterlacer work with an Artix 7 100MHz AXI bus without having to add an additional VDMA on every input channel?

Unfortunately, I'm still busy trying to fix another issue on my project, so I haven't had a chance to try the 4 pixels per clock idea.  Hopefully, I'll get to it this weekend. 

As for your deinterlacer crash, be sure you are resetting the entire path from back to front when a change in the input source is detected.   These video IP blocks seem to be very sensitive to the order in which things are stopped and started.   Be sure to use the XAxiVdma_IsBusy API call to wait after you stop a VDMA channel, to make sure it is finished doing its business before you try to reconfigure and start it again.

 

 

 

0 Kudos
florentw
Moderator
Moderator
786 Views
Registered: ‎11-09-2015

@gcsimmonsjr 

Again, for the by-pass mode for the VPSS as deinterlacer only, make sure you are using 2019.1. You might have it working in 2018.3 updating the drivers but it is not expected to work in previous version as it was no implmented.

 

@reaiken 

On Artix-7 it will really depends on your resolution. But if you are only doing 1080i, it should be fine if you are in 4 pixels/clock


Florent
Product Application Engineer - Xilinx Technical Support EMEA
**~ Don't forget to reply, give kudos, and accept as solution.~**
0 Kudos