12-11-2019 02:21 AM - edited 12-11-2019 02:24 AM
I would like to use the Xilinx UltraScale+ VCU (Video Codec Unit), on my custom board.
I have created a simplified Vivado design of the Xilinx Targeted Reference Design. I hope to use the Test Pattern Generator, create an RTSP stream and view the test pattern on a media player on my PC.
Just to check the Vivado design, I have exported the HDF from Vivado and used PetaLinux tools to create a BOOT.BIN and image.ub. This boots to the Linux login on my custom board.
But what do I actually have to do with the PetaLinux tools to get everything the VCU needs into the BOOT.BIN and image.ub?
03-19-2020 05:26 AM - edited 03-19-2020 06:06 AM
Question: "But what do I actually have to do with the PetaLinux tools to get everything the VCU needs into the BOOT.BIN and image.ub?"
Answer: The Xilinx tools do not provide everything the VCU needs to be a useful IP block
To use the VCU, the user needs to use the PetaLinux tools to create a Linux operating system and then manually create software to run on the Linux operating system to control the VCU.
If you have software skills there may be some useful ideas in the TRD if you poke around long enough. Trying to target the TRD to a custom board for me was a complete waste of time. UG1250 (v2018.3) describes the TRD, but does not give any instructions on creating a custom design.
PG252 (Dec 2019) gives instructions on creating a Vivado design, but no instructions on progressing from the Vivado HDF through to creating the files on an SD card to boot from. But does provide "out of box examples" to get the VCU to do some things, although not necessarily sufficient information to actually use the VCU. This may be of some use https://forums.xilinx.com/t5/Video/Out-of-box-examples-in-PG252-VCU-Video-Codec-Unit/m-p/1064770#M30145
Note if using the Xilinx Test Pattern Generator (TPG) in the Vivado design, to ensure that the <PetaLinux project>/meta-user/recipes-bsp/device-tree/files/system-user.dtsi file is edited as required.
I found these GStreamer commands helpful, to get the VCU to do something
I can see a video test pattern in a Media Player if I use command:
gst-launch-1.0 videotestsrc ! videoconvert ! omxh265enc num-slices=8 gop-length=60 periodicity-idr=60 control-rate=low-latency target-bitrate=5000 cpb-size=3000 prefetch-buffer=true ! video/x-h265, profile=main,level=\(string\)6.2,tier=main ! queue ! rtph265pay ! udpsink host=192.168.1.6 port=50000 max-lateness=-1 qos-dscp=60 async=false buffer-size=60000000 max-bitrate=120000000
Or see a video file with command:
gst-launch-1.0 filesrc location=/home/root/bbb_sunflower_2160p_30fps_normal_avc.mp4 ! qtdemux name=demux demux.video_0 ! h264parse ! omxh264dec ! queue ! omxh265enc num-slices=8 gop-length=60 periodicity-idr=60 control-rate=low-latency target-bitrate=5000 cpb-size=3000 prefetch-buffer=true ! video/x-h265, profile=main,level=\(string\)6.2,tier=main ! queue ! rtph265pay ! udpsink host=192.168.1.6 port=50000 max-lateness=-1 qos-dscp=60 async=false buffer-size=60000000 max-bitrate=120000000
So perhaps the way to be able to use the VCU is learning GStreamer and using the TPG V4L2 driver https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18841638/Xilinx+V4L2+TPG+driver , and then creating a V4L2 driver for a custom design...
12-11-2019 06:42 AM
Hi @watari ,
I've added GStreamer by:
What else should I add? And how do I add it?
The folders in rdf0428-zcu106-vcu-trd-2018-3/apu/apps/ (vcu_apm_lib etc) are "include and "src" and also there's .cproject and .project files...but vcu_apm_lib (etc) does not seem to appear in the the PetaLinux project when it is created from xilinx-vcu-trd-zcu106-v2018.3-final.bsp?? (which I'm using as a reference)
12-11-2019 12:56 PM
12-12-2019 12:26 AM
You might want to go through PG252 - Chapter 13 - Application Software Development.
This contains the information on how to set a petalinux project.
This is slightly different from the VCU TRD, because the VCU has some extra application layers.
But this shoud bring you in the same state has the BSPs (ZCU104, ZCU106 or the BSP inside the VCU TRD) which should contains the basics to run a vcu application
12-12-2019 02:28 AM
Thanks for the link to the web page that lists the source code repos (for the VCU and GStreamer) and gives instructions for setting the size of the CMA (Continuous Memory Allocation).
Thanks for the link to the PG252. I might have missed them but I can't see any instructions on setting the PetaLinux Config...
I also now know about this useful page Zynq UltraScale+ MPSoC VCU TRD 2018.3 - Run and Build Flow
However, for the PetaLinux project this page assumes it is created from the supplied BSP that targets a Xilinx board.
I was hoping there would be documentation that indicates the PetaLinux config settings that should be made to use the VCU, example attached. Is there anything like that?
12-12-2019 02:49 AM
Most of the petalinux configuration is done when you import the HDF/XSA file into the petalinux project. I will see what is enabled in your PL (as the VCU for example) and should enable most of the drivers. I am not expecting much more that should be done in the config
The rest should be documented in PG252
12-12-2019 01:34 PM
At least, you add the followings, too. (especially I guess it requires "gstreamer1.0-rtsp-server")
$ petalinux-config -c rootfs
12-16-2019 08:20 AM - edited 12-16-2019 08:21 AM
12-16-2019 08:23 AM - edited 12-16-2019 08:24 AM
Everything should be in the PG252 for the VCU drivers and even for the gstreamer requirement.
Remember that the PG252 mentions Gstreamer receipe files.
Could you clarify what you think you need?
12-16-2019 08:32 AM
Hi @florentw, there isn't anything in the PG252, as far as I can see, on what config needs to be set
12-16-2019 08:42 AM
You clearly saw in the VCU TRD that nothing was special was set in the VCU TRD. I believe most of the settings required for the VCU are automatically set when you import the HDF into the petalinux project.
But note that Gstreamer is more at the application level, so it is more up to the users to include what is required for their application. But for the TRD, the receipe mentioned in pg252 should be used to include gstreamer.
12-18-2019 12:29 AM
Hi @florentw , thanks for your reply, but I have not found it easy to see what was set for the VCU TRD. There are a lot of PetaLinux config menus and I'm not keen to look through them all, to compare the VCU TRD to my PetaLinux project...hence my original question.
Thanks for the link to https://github.com/Xilinx/meta-petalinux/tree/rel-v2019.2/recipes-multimedia/gstreamer
Are you saying I should be downloading it and adding it as a recipe under <PetaLinux Project>/project-spec/meta-user ?
12-18-2019 01:19 AM
I agree the VCU TRD is now fully clear on what is included. This is why from the beggining I have pointed you to the PG252 which has this information.
For adding the recipe to your petalinux project, refer to the section Integrating the VCU and GStreamer Patches in PG252 p179.
12-18-2019 01:40 AM
Hi @florentw Thanks for the link...I had originally skipped that section as it starts "1. Boot the board using a PetaLinux pre-built image.", and I had not got that far, but I can see that it later then refers to adding GStreamer patches prior to the PetaLinux build. So I'll try that as well as the PetaLinux config settings. Cheers.
12-18-2019 02:00 AM
Hi @florentw ,
I've got as far as point 3b on page 179 (PG252 Dec 10 2019), then I'm not sure what to do at that step?
Then the example at step 3c speaks of "test1.patch", as I don't have this file, is there nothing for me to do at this step?
I'm not a software engineer, so you may have to spell it out please!
12-18-2019 02:04 AM
This steps is if you need to add patches for Gstreamer. If you do not have any patches then you can skip it.
Thus patch1 and patch2 are generic names
12-18-2019 02:17 AM
12-18-2019 02:43 AM
To run the TRD application, yes you would require patches. You might want to look inside the BSP for which patches are included.
But to run gstreamer on your custom board, you might not requires the patches. It will really be dependant on the application
12-18-2019 03:19 AM
Ok, thanks @florentw , will do. I would have preferred there to be Xilinx documentation that describes how get a PetaLinux project to the same state as the Xilinx VCU TRD PetaLinux BSP, but will just do comparisions and hopefuly (eventually) update this forum post when I'm successful.
12-18-2019 03:44 AM
Keep in mind that the VCU TRD is a very big application with not only the VCU.
The PG252 gives the steps to be in the same state as the ZCU106 BSP which can run basic VCU applications (the VCU out of the box examples mentioned in PG252)
12-18-2019 04:10 AM
@florentw I have the same inquiry on how to make VCU work from scratch, e.g. how can I build a simplest VCU example starting from hdf rather than using your TRD bsp. It seems to me a huge job to port your TRD to a customer board. I just need a simplest example to verify the VCU is working and there is no hardware issue at this stage, unfortunately, porting the design seems massively complicated to me. Please help, thanks.
12-18-2019 05:08 AM
You have the same query, thus my answer is the same. Everything is in the Application Software Development section of PG252
12-18-2019 05:20 AM
Hi @florentw ,
I didn't get anywhere with the out of box examples because, I just get the response "command not found" if I run say vcu-demo-transcode-to-file.sh
PG252 (Dec 10 2019) on page 165 states “All of the software prerequisites for using the VCU are included in Xilinx® PetaLinux included in Vitis software development platform release.”, but then lists a table of Application software which includes links to recipes-multimedia
So to use the Out of the Box examples, do I need to manually add these before the petalinux-build or should the PetaLinux tools have added them automatically?
But when I look at the PetaLinux project created from the TRD BSP there is no folder “recipe-multimedia”, so I’m not sure that makes for a useful comparison for getting the Out of Box examples to work??
Nor does the TRD BSP seem to help me know what patches I need to add for VCU or GStreamer? Is there a list of patches somewhere?
12-18-2019 06:39 AM
I can see if I go here https://github.com/Xilinx/meta-petalinux/tree/rel-v2018.3 I am then able to download files from github that include the folder recipes-multimedia. That contains gstreamer/gstreamer-vcu-examples/<all the Out of Box .sh files>
I feel that the section in PG252 you mention is meant to be telling me how I get these files into the PetaLinux project, but I am unable to follow them.
Are you able to explain what step 3b on page 179 (PG252 Dec 10 2019) is asking me to do?
12-18-2019 08:14 AM
12-18-2019 01:20 PM
You must add recipe files in Table 13-1 on page 165 of PG252 (Dec 10 2019) in proper directory.
Also, refer chapter 8 (from page 68) on the following URL.
12-19-2019 01:22 AM