cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
dclark
Newbie
Newbie
401 Views
Registered: ‎10-15-2018

DDR PHY Training fails on custom Zynq board

Jump to solution

We are bringing up our custom board which contains a Zynq Ultrascale+ 4EG SOC and we are experiencing an issue during the DDR PHY training process in the A53 FSBL. Here are some stats about our board:

  • Zynq P/N: XCZU4EG-2SFVC784E
  • DDR4 SODIMM P/N: MTA4ATF51264HZ-2G6E1
  • DDR clock speed: 900MHz
  • Our design is based on the ZCU102 dev board (no changes we made to the PS DDR configs in Vivado between our dev board build and our custom board build)
  • Vivado 2019.2
  • Vitis: 2019.2.1

Here are the steps taken which result in the error we are seeing:

  • We have generated PMU FW for our system and run it. All signs indicate this runs successfully. PS_DONE, PS_ERR signals are the expected values and pmu_global registers are at the expected values (no errors, such as pwr_supply_status (0xffd8010c) = 0x7, error_status_1 (0xffd80530) = 0, error_status_2 (0xffd80540) = 0, and global_cntrl (0xffd80000) = 0x00018810.
  • We have generated a FSBL project with the proper parameters to be able to step through it with the debugger.
  • To test the system, first we run the PMU FW through the Vitis debugger, which we have configured to reset the system, program the FPGA, and power up the PL. Once this runs until the Microblaze PMU status shows “Sleeping. No clock”, we launch the FSBL via a second debug configuration. This debug config resets only the APU (not the entire system) and runs the FSBL flow for initialization. This drops us into the debug view where we can step through the code, add breakpoints, inspect memory, etc.
  • If we just press play, let it run for a while, and then press pause, we can see that it gets stuck in the XFsbl_DdrcPhyTraining function polling the DDR_PHY_PGSR0_OFFSET register. It is expecting to see a value of 0x80000CFF, but instead, it reads a value of 0x80c000ff. This indicates a Write leveling adjustment error and DQS gate training error. The FSBL spins in a loop forever waiting for the correct value (but it never gets it).

Things we have tried to fix/get around or troubleshoot this error:

  • Comment out the DDR PHY training function call - this fails to initialize the DDR
  • Comment out the points in the DDR training process that were getting hung up in while loops (because of errors). No difference.
  • Disable the tests that were causing the DDR training to fail by deasserting the appropriate bits in the DDR_PHY_PIR_OFFSET register (as well as the expected values/masks)
  • We have confirmed that the DDR circuitry on our board meets the Xilinx UG583 for DDR board layout
  • Tested power rails, reset signals, SPD interface and everything checks out
  • Probed the DDR clock. We have yet to get a great reading on this to tell the quality of the clock, but there does seem to at least be a clock.
  • We have tried a number of different DDR clock speeds including 600MHz, 900MHz, and 1067MHz so far. The 600MHz clock gave us a value of 0x844004FF which indicates a DQS Gate training err and read eye training error (no write leveling adjustment error). This seems to suggest that we do get a little further along in the DDR training process than with a faster clock, but does not resolve all of our errors.
  • Enabled 2T timing. This did not seem to make a difference.
  • Register to register comparison of the DDR_PHY register set from the ZCU102 dev board and our custom board. There were many registers different between the two, I believe most of these differences are calibration values.
  • Lots of googling and reading of Xilinx forum posts
0 Kudos
Reply
1 Solution

Accepted Solutions
dclark
Newbie
Newbie
301 Views
Registered: ‎10-15-2018

We are now able to successfully initialize DDR (including training). It appears the issue was due to manufacturing related problems. Long story short, there was a bit of a BOM mixup with our board house which resulted in our first batch of boards needing ~300 passivie components to be hand soldered on one side. We have one board from this batch which did not successfully complete DDR PHY training and one which does. The second batch of boards all complete DDR PHY training without issue (all components placed autonomously). We believe that there may have been a mixup with the hand soldered parts on the board that does not work. I will close this issue now.

View solution in original post

1 Reply
dclark
Newbie
Newbie
302 Views
Registered: ‎10-15-2018

We are now able to successfully initialize DDR (including training). It appears the issue was due to manufacturing related problems. Long story short, there was a bit of a BOM mixup with our board house which resulted in our first batch of boards needing ~300 passivie components to be hand soldered on one side. We have one board from this batch which did not successfully complete DDR PHY training and one which does. The second batch of boards all complete DDR PHY training without issue (all components placed autonomously). We believe that there may have been a mixup with the hand soldered parts on the board that does not work. I will close this issue now.

View solution in original post