cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
1,504 Views
Registered: ‎01-07-2013

RFSoC ADC -> DAC Design

I am using a ZCU111 to try loops a signal received on the ADC to the DAC. Initially I used the 8x8-ADC-R2C-4GSPS-DAC-C2R preset to get the design started. My data rate is 2.21184 Gsps for both the ADC and the DAC. I left the default ADC setting of 8 samples per cycle at 276.480 MHz. The DACs are also using their default of 16 samples per cycle at 138.24 MHz. I am using the internal PLL and the reference clock is 245.76 MHz to match the TRD. The "Clock Out" values for each of the DAC and ADC tiles is 138.24 MHz. I connected the clk_adc0 output to a Clocking wizard which generates the 276.38 MHz clock for all the ADCs. I also connected the clk_dac0 to a Clocking wizard which also outputs 138.24 MHz. You can see the clocking portion of my design in the included PNG.

To loop the data from the ADC to the DAC I connected the real data (Even numbered Master ports on the Data Converter) through an AXI-4 Stream Interconnect with 4096 sample FIFOs on the Master and Slave to the Slave DAC ports on the RF Data Converter. You can also see one of these connections in the attached screenshot.

 

On the processor I am running a modified version of the rfdc-data-write-example. I still run rfdc_inst_init to register with libmetal and initialize the driver. Then I run an unmodified initRFclock command. That call looks like this:

 

ret = initRFclock(ZCU111, LMK04208_12M8_3072M_122M88_REVAB, DAC_245_76_MHZ,DAC_245_76_MHZ, ADC_245_76_MHZ);
	if(ret != SUCCESS){
		printf("Unable to configure RF clocks\n");
		return ret;
	}

 

That returns successfully and all four clock status LEDs on the ZCU111 are lit.

I have my own initialization function which attempts to bring up the RF Data Converters. First I do a dynamic configuration of the PLL. This seemed to be necessary because without this call GetPLLConfig would report an incorrect DAC sample rate. The Configuration call looks like this:

 

Status = XRFdc_DynamicPLLConfig(&RFdcInst, RFDC_DAC, Tile, XRFDC_INTERNAL_PLL_CLK, 245.76, 2211.840);
			if(Status) {
				printf("FAILED to configure DAC PLL %d\n", Tile);
				return Status;
			}

 

And is repeated for all the ADCs and DACs.

Then I  Reset all the ADC and DAC tiles like this:

 

	Status = XRFdc_Reset(&RFdcInst, XRFDC_ADC_TILE, -1);
	if (Status != XRFDC_SUCCESS) {
		return XRFDC_FAILURE;
	}

 

I disable the Data FIFOs for the ADCs and DACS. I set the fabric clock division factor to 1 for the DAC to produce a 138.24 MHz dac0_clk output. Then I set the Interpolation factor to 2.

I enable all the ADCs and DACs in the order they are connected. So they are enabled in this order ADC0:0, DAC0:0, ADC0:1, DAC0:1 (Tile:Block).

At this point I believe that the design should be running so I read a bunch of status information from the device. I run XRFdc_GetPLLConfig and XRFdc_GetPLLLockStatus. The ADC outputs from all these calls are identical. They look like this:

 

ADC Tile 0 PLL Configurations:: PLL Enable: 1
ADC Tile 0 PLL Configurations:: PLL Refclk Rate: 245.760000
ADC Tile 0 PLL Configurations:: PLL Sample Rate: 2.211840
ADC Tile 0 PLL Configurations:: PLL FB Divider: 36
ADC Tile 0 PLL Configurations:: PLL Output Divider: 4
ADC Tile 0 PLL Configurations:: PLL RefClkDivider: 1
ADC Tile 0 PLL Lock Status:: PLL Locked: 2

 

The DACs are all identical also and look like this:

 

DAC Tile 0 PLL Configurations:: PLL Enable: 1
DAC Tile 0 PLL Configurations:: PLL Refclk Rate: 245.760000
DAC Tile 0 PLL Configurations:: PLL Sample Rate: 2.211840
DAC Tile 0 PLL Configurations:: PLL FB Divider: 36
DAC Tile 0 PLL Configurations:: PLL Output Divider: 4
DAC Tile 0 PLL Configurations:: PLL RefClkDivider: 1
DAC Tile 0 PLL Lock Status:: PLL Locked: 2

 

I run XRFdc_GetIPStatus, XRFdc_GetFIFOStatus, XRFdc_GetBlockStatus, and XRFdc_GetIntrStatus. I get values that look like the DACs are both over running and underrunning. The output from the DACs look like this:

 

DAC Tile 0 Block 0 enabled
DAC Tile 0 Block 0 Tile State: 15
DAC Tile 0 Block 0 Power-up State: 1
DAC Tile 0 Block 0 PLL Status: 1
DAC Tile 0 Block 0 FIFO Status: 1
DAC Tile 0 Block 0 Sampling Frequency: 2.211840
DAC Tile 0 Block 0 Analog Datapath Status: 0x00000010
DAC Tile 0 Block 0 Digital Datapath Status: 0x00002021
DAC Tile 0 Block 0 Digital Clock Status: 0x01
DAC Tile 0 Block 0 FIFO Flags: 0x03
DAC Tile 0 Block 0 FIFO Flags Asserted: 0x03
DAC Tile 0 Block 0 Interrupt Status: 0x0000000f

 

I do try to clear the interrupt status and read it again but the interrupt status remains 0x0000000f.

The end result is that the output on the scope is flat lined. I don't see any DAC output. How can I get this datapath working?

rfsoc_Clocking.png
rfsoc_datapath.png
0 Kudos
11 Replies
klumsde
Moderator
Moderator
1,436 Views
Registered: ‎04-18-2011

First step I would do is make sure that the DAC path is working on its own. 

Use the Evaluation design or RF analyzer to look at the input here. 

Also look at this ADC path, are you receiving good data here. is the signal power Ok at the output of the ADC? 

If you get sensible data out this way then you know the problem is somewhere in the internal loopback you have made. 

You said

"I left the default ADC setting of 8 samples per cycle at 276.480 MHz. The DACs are also using their default of 16 samples per cycle at 138.24 MHz"

how are you taking care of the mismatch of the bus width here??

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
1,404 Views
Registered: ‎01-07-2013

First, sincerely, thank you for responding and trying to help. 

I took your suggestions and worked on getting the TRD working on my board. Specifically I worked from the instructions here:

https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/57770205/RFSoC+RFdc+Build+and+Run+Flow+Tutorial?preview=%2F57770205%2F133955641%2FPL_settings.PNG

My experience with Xilinx devices lately is that the documentation has gotten to be very poor. It's much more spread out than it used to be. It's not keeping up with the tool releases. This has been true for everything from Petainux to the RFSoC to Vitis/AI.

I'll use this as an example. Hopefully you can put some pressure on people on your end to improve the documentation. I'll walk through what I found.

First, for a "Build and Run Flow Tutorial" it is odd to start by introducing the GUI. You can't use the GUI until the last step. Why start here? Also, my experience with Petalinux is that it only works with Linux. The last time I checked it wasn't even supported in a Linux VM or in WSL. So philosophically it drives me crazy that the GUI tool is not portable. It's Windows only. So I either have to use two machines or run a Windows VM on my Linux box.

Next, the documentation isn't up to date for 2019.2 even though the TRD release is for 2019.2. The documentation contains references to non-existent things like system-user_8x8.dtsi. I built the Vivado design, exported the XSA with the bitstream, I got Petalinux to config with the new XSA file, I did a build, build SDK, package, and copied everything over to the SD Card. The first problem is that nothing runs because there are no instructions about needing to copy over the mts, nonmts, and ssr folders. I went to the code and figured out that I needed to do that but the next problem is that these folders contain .bit.bin files used by fpgautil. There are no instructions on creating a .bit.bin or the .dtbo files. So I went and found instructions on creating a .bit.bin file. It requires the creation of a .bif file. I had one from a Vitis project I am working on. Everything seemed to apply so I used bootgen to create the .bit.bin file that fpgautil needs. I copied this into the nonmts directory. I booted the board and tried to run rftool. That didn't work because the provided BSP didn't include FPGA Manager support. No fpgautil so the PL never gets programmed. I went back through the tools, added FPGA Manager, and rebuilt everything. I thought for sure this would work. Nope. I get an error:

metal: info: Registered shmem provider linux_shm.
metal: info: Registered shmem provider ion.reserved.
metal: info: Registered shmem provider ion.ion_system_contig_heap.
metal: info: Registered shmem provider ion.ion_system_heap.
metal: error:
Invalid device id 0

So I say, forget it. Let's see if I just copy the stock images to the SD Card if I can get this working. That looked better at the beginning. The board boots. I can see the PL get programmed. The LMXs are all configured. The LMX Mux out LEDs are all lit. The console says

starting server...
starting data server...
Server Init Done

Next I tried first using JTAG to talk to the board through the RF Data Converter Eval User Interface Software. This failed with Err1172. I configured that the serial console is working. I also confirmed that Hardware Manager can see the board. The Eval Tool, not so much.

So I caved and reconfigured my networking. I left the ZCU111 at 192.168.1.3 (The default). I configured my computer for 192.168.1.200. My PC can ping the ZCU111:

Pinging 192.168.1.3 with 32 bytes of data:
Reply from 192.168.1.3: bytes=32 time=6ms TTL=64
Reply from 192.168.1.3: bytes=32 time=6ms TTL=64

I confirmed that the ZCU111 can ping the host machine. I configured the Communications in the Eval Tool to use Ethernet and match these settings. I restarted the tool and I get a big banner that says "No device communication." The GUI window can't be maximized so it runs off the screen. It doesn't matter if I change my display resolution to 4k or 1080. But if I move things around I can see that along the bottom it says "Unable to connect to rftrd service."

You also mentioned using RF Analyzer. I will get there. I would be extremely useful if you could help address some of the issues with the TRD first. We'll get to RF Analyzer next. It would be useful for a lot of designs to have access to RF Analyzer. I haven't found documentation on how to add it to an existing design. The only thing I have found is instructions to Open Example Design and then it puts the Microblaze in to do what is needed in the PL to communicate with the RF Analyzer tool. It would be way better if this happened behind the scenes when a user selects the "RF Analyzer" drop down when configuring the RF Converter IP. Just roll the Microblaze and supporting infrastructure into the IP block itself. 

CommConfig.PNG
0 Kudos
klumsde
Moderator
Moderator
1,366 Views
Registered: ‎04-18-2011

Hi dustin.nicholes@gmail.com 

I appreciate that the getting started guide has gone out of synch since the scheme for loading the PL image has changed and the advent of Vitis has made the flow slightly different. we can certainly look at ways to improve this.  

Next I tried first using JTAG to talk to the board through the RF Data Converter Eval User Interface Software. This failed with Err1172. I configured that the serial console is working. I also confirmed that Hardware Manager can see the board. The Eval Tool, not so much.

JTAG was never really supported for this combination of eval design and GUI. 

I think this has gone a little bit off track. 

I didn't really intend for you to build the whole design and linux image.

I made the suggestion to use the evaluation design or the RF Analyzer so that we could debug this specific problem in your design. 

This will work over JTAG with no issues. It even comes with a pre-canned bitstream. why not use that to figure out that both ADC and DAC RF paths are working as you intend. then return to debugging your design?

 

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
1,356 Views
Registered: ‎01-07-2013

This morning I reformatted a new SD Card on a different computer with the canned TRD images. I booted the ZCU111 with this design and used the same old computer with the RF Data Converter GUI installed. I was able to get the GUI to talk to the ZCU111. I do not understand why this would make any difference. I definitely confirmed that the ZCU111 was booting. The ZCU111 console indicated that the server and data server were started. I could ping the board. The failure of the GUI to connect and the absence of a meaningful error message is terrible. If JTAG isn't supported by the GUI it needs to be removed as an option. You said "we can certainly look at ways to improve this" about a number of issues. That sounds like a way to dismiss the big picture. I'm sure we can solve the individual issue but it's worth taking this feedback, making the documentation more complete and up to date so that future users don't run into the same things. Honestly, it will save you time in the long run. Otherwise people will just keep posting the same things in the forums.

Sorry. Rant over (For now).

My test setup is a little different than the strict loopback from DAC->ADC. I have a signal generator generating a 100 MHz sine wave connected to the ADC channels. I have a scope connected to the DAC channels. I did it this way because it can use the same setup to confirm the functionality of the TRD and my own design. This test did reveal that the breakout board was shipped with loose SMA connectors. I tightened the SMA connectors and now I can see the 100 MHz sine wave being generated by the DACs on the scope. I can also see the signal generator's 100 MHz sine wave on the ADCs in the Eval GUI. My remaining concern is that as I increase the amplitude out of the SIGGEN I don't see the amplitude of the time domain signal increase. The power in the FFT spectrum increases but the Time Domain does not. The time domain series maxes out around 150. The ADC is 12-bits so I should be able to get +/-2048 and I don't see anything close. Is there some kind of AGC in place? Why is this happening?

I also noted that if you are in loop acquire and you change the number of samples the GUI and rftool start throwing unrecoverable errors and have to be killed. Please disable controls that cannot that will cause the program to crash.

In any case, having confirmed that the RF design works I went back to my original design and confirmed that the ADC->DAC loopback I put together still does not work. I will keep trying different approaches on my end

0 Kudos
klumsde
Moderator
Moderator
1,346 Views
Registered: ‎04-18-2011

It is good that the ADC and DAC analog sides appear to work. 

So your next step should be I think is to hook a system ILA to this internal loopback to see what is going on. 

Keith 

 

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
1,293 Views
Registered: ‎01-07-2013

I have the ADC and DAC ports of interest (And some not of interest but just for reference) pulled out to ILAs. The naming convention on the waveform is ADC<TILE><BLOCK> and then Sample0 refers to [15:0] and Sample1 refers to [31:16]. I have i/q both pulled out to the ILA but the q channel is always zero. You can see that the streams are all active. I set the radix to signed decimal and the style to analog. This way it should be easy to see if I'm getting anything but noise. It doesn't look like I am. There is nothing connected to the ADC10 SMA. There is a 100 MHz sine wave connected to the ADC11 SMA. In the captured samples the two channels look identical. I can turn off the function generator and it still looks the same.

What next?

rfdcDebug.png
0 Kudos
klumsde
Moderator
Moderator
1,289 Views
Registered: ‎04-18-2011

this looks like just noise to me. 

when you do a getIPStatus in the software application (i presume you have one running) what does it say here? The ADC tile should have reached step 15 of the startup FSM. 

 

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
1,283 Views
Registered: ‎01-07-2013

Here's the IP Status after initialization:

ADC Tile 1 Block 1 Enabled
ADC Tile 1 Block 1 Tile State: 15
ADC Tile 1 Block 1  Power-up State: 1
ADC Tile 1 Block 1  PLL Status: 1

0 Kudos
klumsde
Moderator
Moderator
1,228 Views
Registered: ‎04-18-2011

Maybe pass me the script to make your block design so I can try this on my end... 

I'm not sure what can be the issue here. 

Taking a step back the low amplitude on the adc output when we use the trd is something to look at. It could be something to do with the gui given the FFT power level changes properly

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
0 Kudos
1,210 Views
Registered: ‎01-07-2013

I have a new development that I think will help diagnose this problem. I drove the RF converter's fabric clock off board so I could measure its frequency. The clock is totally the wrong frequency. Based on my RF Converter settings that clock should be running at 138.24 MHz. On my scope I am seeing a really ugly ~30.43 MHz clock. I am using the code Xilinx sends with their rfdc-data-write example to set the clocks. Specifically:
 
ret = initRFclock(ZCU111, LMK04208_12M8_3072M_122M88_REVAB, DAC_245_76_MHZ,
DAC_245_76_MHZ, ADC_245_76_MHZ);
 
That call returns without error. I then confirmed that the software reads the correct configuration from the FPGA. It shows:
 
XRFdc_Config :: DAC 0 Tile Config : Enable 1
XRFdc_Config :: DAC 0 Tile Config : PLL Enable 1
XRFdc_Config :: DAC 0 Tile Config : Sample Rate 2.211840
XRFdc_Config :: DAC 0 Tile Config : Refclk Freq 276.480000
XRFdc_Config :: DAC 0 Tile Config : Fabclk Freq 138.240000
XRFdc_Config :: DAC 0 Tile Config : FB Div 32
XRFdc_Config :: DAC 0 Tile Config : Output Div 4
XRFdc_Config :: DAC 0 Tile Config : Refclk Div 1
XRFdc_Config :: DAC 0 Tile Config : Multiband Cfg 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Mixer Type 2
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Mixer Type 2
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Mixer Type 2
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 0 Tile Config : Digital CFG:: Mixer Type 2
XRFdc_Config :: DAC 1 Tile Config : Enable 1
XRFdc_Config :: DAC 1 Tile Config : PLL Enable 1
XRFdc_Config :: DAC 1 Tile Config : Sample Rate 2.211840
XRFdc_Config :: DAC 1 Tile Config : Refclk Freq 276.480000
XRFdc_Config :: DAC 1 Tile Config : Fabclk Freq 138.240000
XRFdc_Config :: DAC 1 Tile Config : FB Div 32
XRFdc_Config :: DAC 1 Tile Config : Output Div 4
XRFdc_Config :: DAC 1 Tile Config : Refclk Div 1
XRFdc_Config :: DAC 1 Tile Config : Multiband Cfg 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Mixer Type 2
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Mixer Type 2
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Mixer Type 2
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Data Width 16
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Interpolation Mode 2
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: FIFO Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Adder Enable 0
XRFdc_Config :: DAC 1 Tile Config : Digital CFG:: Mixer Type 2
 
Everything I see there looks okay.
 
I thought that maybe the clock programming code I was using had a bug. So I tried programming the clocks with the System Controller. I discovered that System Controller doesn't have the ability to read back the frequency of the LMK or LMX parts. System Controller isn't shipped with a clock file that would work to configure the clocks to the example design's default 276.48 MHz.
 
I found that the RFDC Eval GUI is distributed with TICS files for both the LMK and the LMX. Unfortunately these are not compatible with the System Controller GUI (Please fix). I used TICS itself and loaded the LMK04208_ZCU111_revAB_CKin1=12M8_VCO=3072M_Out=122M88_MTS.tcs file. I used this file and exported the HEX values to a text file the System Generator could read. Then I used the LM2594_0245M76.tcs file to generate the HEX values for the LMX2594 device.
 
In System Controller I used these files to try to program the clock parts. I discovered that if you provide System Controller with a clock file name that does not exist it doesn't throw an error. It looks like everything worked. That should be fixed. I discovered that the frequency reported in the GUI is just parsed from the text file name. This is not very useful and potentially misleading. Please fix this. When I program the clocks with my clock files I see the corresponding LEDs go off and then back on when the programming completes. With the clocks programmed using System Controller I commented out the lines in my code that programmed the clocks and reran the code. The clock output is the same wrong ~30 MHz.
 
How can we diagnose how this is going wrong?
0 Kudos
pthakare
Moderator
Moderator
1,198 Views
Registered: ‎08-08-2017

Hi dustin.nicholes@gmail.com 

This is Pramod and i am handling the SR you have filed yesterday  on this issue. As suggested to check the ADC or DAC_CLK  out from tile on J95 connector , you are not observing the intended clock and our first step should be to debug the clocking.  I will send you out additional suggestion on clocks debugging on SR mail.

 

-------------------------------------------------------------------------------------------------------------------------------
Reply if you have any queries, give kudos and accept as solution
-------------------------------------------------------------------------------------------------------------------------------
0 Kudos