07-21-2010 11:09 AM
I have been having some issues with DDR memory. Reads to the device return incorrect data, but only on some addresses. We have a custom designed board with a Virtex-4 FX60 part. The pin-out of the FPGA, with respect to the DDR chip, is not MIG compatible so I've been using the static PHY to interface with the memory device. We are using two Micron DDR SDRAM parts: MT46V32M16-6:F for a combined data-width of 32-bits and 128MB of memory.
The plb bus is running at a clock frequency of 100MHz and the processor at 200MHz. I have one PLB master connecting to Port 0 of the MPMC as well as other shared peripherals (MDM, MPMC ctrl register, etc).
I've ran the static phy calibration program, located at %XILINX_EDK%\sw\XilinxProcessorIPLib\drivers\ and have run into an issue I can't seem to reconcile.
Before continuing, it may be worth noting that we don't have a hardware UART on our board, so for troubleshooting I am using an MDM pcore to emulate a UART over JTAG. From what I've gathered, this interface has its host of issues as well.
Back to the issue:
1) When I run the calibration program I notice (through both printf's and XMD mrd's) that reading from some addresses in DDR do not seem to have the correct contents. In addition, the erroneous data follows a pattern where 0x4, 0xC, 0x14, 0x1C, etc, all have bad data. As an example, here's print out after writing all zeros:
XMD% mwr 0 0 20 w # writing all zeros to address 0 - 4F XMD% mrd 0 20 # reading back address 0 - 4F 0: 00000000 4: 21000041 8: 00000000 C: 21000001 10: 00000000 14: 21000041 18: 00000000 1C: 21000001 20: 00000000 24: 21000041 28: 00000000 2C: 21000041 30: 00000000 34: 21000041 38: 00000000 3C: 21000041 40: 00000000 44: 21000041 48: 00000000 4C: 21000041
Further testing indicates that the non-zero values in memory have bits that are stuck hi regardless of what I write into those memory addresses. As an example, if I write 0x42000002 at address 0x4, the output is 0x63000043:
XMD% mwr 0x4 0x42000 XMD% mrd 0x4 4: 63000043
The two values (the value in memory + the value I wrote) are just OR'd!
Anyway, why does this only occur on every 8 addreses (0x4, 0xC, 0x14, etc)? Any insight would be greatly appreciated.
I've also attached my MHS for reference:
# ############################################################################## # Created by Base System Builder Wizard for Xilinx EDK 10.1.03 Build EDK_K_SP3.6 # Target Board: Custom # Family: virtex4 # Device: xc4vfx60 # Package: ff672 # Speed Grade: -10 # Processor: ppc405_0 # Processor clock frequency: 200.00 MHz # Bus clock frequency: 100.00 MHz # On Chip Memory : 64 KB # Total Off Chip Memory : 64 MB # - DDR_SDRAM = 128 MB # ############################################################################## PARAMETER VERSION = 2.1.0 PORT fpga_0_DDR_SDRAM_Clk_pin = fpga_0_DDR_SDRAM_Clk, DIR = O PORT fpga_0_DDR_SDRAM_Clk_n_pin = fpga_0_DDR_SDRAM_Clk_n, DIR = O PORT fpga_0_DDR_SDRAM_Addr_pin = fpga_0_DDR_SDRAM_Addr, DIR = O, VEC = [12:0] PORT fpga_0_DDR_SDRAM_BankAddr_pin = fpga_0_DDR_SDRAM_BankAddr, DIR = O, VEC = [1:0] PORT fpga_0_DDR_SDRAM_CAS_n_pin = fpga_0_DDR_SDRAM_CAS_n, DIR = O PORT fpga_0_DDR_SDRAM_CE_pin = fpga_0_DDR_SDRAM_CE, DIR = O PORT fpga_0_DDR_SDRAM_CS_n_pin = fpga_0_DDR_SDRAM_CS_n, DIR = O PORT fpga_0_DDR_SDRAM_RAS_n_pin = fpga_0_DDR_SDRAM_RAS_n, DIR = O PORT fpga_0_DDR_SDRAM_WE_n_pin = fpga_0_DDR_SDRAM_WE_n, DIR = O PORT fpga_0_DDR_SDRAM_DM_pin = fpga_0_DDR_SDRAM_DM, DIR = O, VEC = [3:0] PORT fpga_0_DDR_SDRAM_DQS = fpga_0_DDR_SDRAM_DQS, DIR = IO, VEC = [3:0] PORT fpga_0_DDR_SDRAM_DQ = fpga_0_DDR_SDRAM_DQ, DIR = IO, VEC = [31:0] PORT sys_clk_200_pin = proc_clk_s, DIR = I, SIGIS = CLK, CLK_FREQ = 200000000 PORT sys_rst_pin = sys_rst_s, DIR = I, RST_POLARITY = 0, SIGIS = RST PORT sys_clk_100_pin = sys_clk_s, DIR = I, SIGIS = CLK, CLK_FREQ = 100000000 PORT sys_clk_100_90_pin = sys_clk_s90, DIR = I, SIGIS = CLK, CLK_FREQ = 100000000 PORT dcm_locked_pin = Dcm_ext_locked, DIR = I PORT DDR_SDRAM_MPMC_Rst_pin = DDR_SDRAM_MPMC_Rst, DIR = I, SIGIS = RST BEGIN ppc405_virtex4 PARAMETER INSTANCE = ppc405_0 PARAMETER HW_VER = 2.01.a # PARAMETER C_FASTEST_PLB_CLOCK = DPLB1 PARAMETER C_IDCR_BASEADDR = 0b0100000000 PARAMETER C_IDCR_HIGHADDR = 0b0111111111 BUS_INTERFACE IPLB0 = plb BUS_INTERFACE DPLB0 = plb BUS_INTERFACE ISOCM = ppc405_0_iocm BUS_INTERFACE DSOCM = ppc405_0_docm BUS_INTERFACE JTAGPPC = jtagppc_cntlr_0_0 BUS_INTERFACE RESETPPC = ppc_reset_bus PORT BRAMISOCMCLK = sys_clk_s PORT BRAMDSOCMCLK = sys_clk_s PORT CPMC405CLOCK = proc_clk_s END BEGIN plb_v46 PARAMETER INSTANCE = plb PARAMETER C_NUM_CLK_PLB2OPB_REARB = 100 PARAMETER HW_VER = 1.03.a PORT PLB_Clk = sys_clk_s PORT SYS_Rst = sys_bus_reset END BEGIN isocm_v10 PARAMETER INSTANCE = ppc405_0_iocm PARAMETER HW_VER = 2.00.b PARAMETER C_ISCNTLVALUE = 0xa3 PORT ISOCM_Clk = sys_clk_s PORT sys_rst = sys_bus_reset END BEGIN isbram_if_cntlr PARAMETER INSTANCE = ppc405_0_iocm_cntlr PARAMETER HW_VER = 3.00.b PARAMETER C_BASEADDR = 0xffff8000 PARAMETER C_HIGHADDR = 0xffffffff BUS_INTERFACE ISOCM = ppc405_0_iocm BUS_INTERFACE DCR_WRITE_PORT = isocm_porta BUS_INTERFACE INSTRN_READ_PORT = isocm_portb END BEGIN bram_block PARAMETER INSTANCE = isocm_bram PARAMETER HW_VER = 1.00.a BUS_INTERFACE PORTA = isocm_porta BUS_INTERFACE PORTB = isocm_portb END BEGIN dsocm_v10 PARAMETER INSTANCE = ppc405_0_docm PARAMETER HW_VER = 2.00.b PARAMETER C_DSCNTLVALUE = 0xa3 PORT DSOCM_Clk = sys_clk_s PORT sys_rst = sys_bus_reset END BEGIN dsbram_if_cntlr PARAMETER INSTANCE = ppc405_0_docm_cntlr PARAMETER HW_VER = 3.00.b PARAMETER C_BASEADDR = 0x82000000 PARAMETER C_HIGHADDR = 0x82007fff BUS_INTERFACE DSOCM = ppc405_0_docm BUS_INTERFACE PORTA = dsocm_porta END BEGIN bram_block PARAMETER INSTANCE = dsocm_bram PARAMETER HW_VER = 1.00.a BUS_INTERFACE PORTA = dsocm_porta END BEGIN jtagppc_cntlr PARAMETER INSTANCE = jtagppc_cntlr_0 PARAMETER HW_VER = 2.01.c BUS_INTERFACE JTAGPPC0 = jtagppc_cntlr_0_0 END BEGIN proc_sys_reset PARAMETER INSTANCE = proc_sys_reset_0 PARAMETER HW_VER = 2.00.a PARAMETER C_EXT_RESET_HIGH = 0 BUS_INTERFACE RESETPPC0 = ppc_reset_bus PORT Slowest_sync_clk = sys_clk_s PORT Dcm_locked = DCM_all_locked PORT Ext_Reset_In = sys_rst_s PORT Bus_Struct_Reset = sys_bus_reset PORT MB_Debug_Sys_Rst = mdm_0_Debug_SYS_Reset END BEGIN mdm PARAMETER INSTANCE = mdm_0 PARAMETER HW_VER = 1.00.d PARAMETER C_USE_UART = 1 PARAMETER C_INTERCONNECT = 1 PARAMETER C_MB_DBG_PORTS = 0 PARAMETER C_UART_WIDTH = 8 PARAMETER C_BASEADDR = 0x84400000 PARAMETER C_HIGHADDR = 0x8440ffff BUS_INTERFACE SPLB = plb PORT Debug_SYS_Rst = mdm_0_Debug_SYS_Reset END BEGIN dcm_module PARAMETER INSTANCE = mpmc_dcm PARAMETER HW_VER = 1.00.d PARAMETER C_CLK0_BUF = TRUE PARAMETER C_CLKIN_PERIOD = 10.000000 PARAMETER C_CLKOUT_PHASE_SHIFT = VARIABLE_POSITIVE # Note: Exact value may vary by device. # # For example, Virtex-4 ES devices require VARIABLE_POSITIVE (AR#20529) # PARAMETER C_EXT_RESET_HIGH = 0 PORT CLKIN = sys_clk_s PORT CLK0 = MPMC_Clk_Mem PORT CLKFB = MPMC_Clk_Mem PORT RST = DCM_ext_locked PORT LOCKED = DCM_all_locked PORT PSCLK = sys_clk_s PORT PSDONE = mpmc_dcm_PSDONE PORT PSEN = mpmc_dcm_PSEN PORT PSINCDEC = mpmc_dcm_PSINCDEC END BEGIN mpmc PARAMETER INSTANCE = DDR_SDRAM PARAMETER HW_VER = 4.03.a PARAMETER C_MPMC_BASEADDR = 0x00000000 PARAMETER C_MPMC_HIGHADDR = 0x07FFFFFF PARAMETER C_MEM_PARTNO = MT46V32M16-6 PARAMETER C_MPMC_CLK0_PERIOD_PS = 10000 PARAMETER C_MEM_TYPE = DDR PARAMETER C_MEM_DATA_WIDTH = 32 PARAMETER C_MPMC_CTRL_BASEADDR = 0x84200000 PARAMETER C_MPMC_CTRL_HIGHADDR = 0x8420FFFF PARAMETER C_USE_STATIC_PHY = 1 PARAMETER C_STATIC_PHY_RDEN_DELAY = 0 BUS_INTERFACE MPMC_CTRL = plb BUS_INTERFACE SPLB0 = plb PORT MPMC_Rst = DDR_SDRAM_MPMC_Rst PORT DDR_DQS = fpga_0_DDR_SDRAM_DQS PORT DDR_DM = fpga_0_DDR_SDRAM_DM PORT DDR_DQ = fpga_0_DDR_SDRAM_DQ PORT DDR_Addr = fpga_0_DDR_SDRAM_Addr PORT DDR_BankAddr = fpga_0_DDR_SDRAM_BankAddr PORT DDR_WE_n = fpga_0_DDR_SDRAM_WE_n PORT DDR_CAS_n = fpga_0_DDR_SDRAM_CAS_n PORT DDR_RAS_n = fpga_0_DDR_SDRAM_RAS_n PORT DDR_CS_n = fpga_0_DDR_SDRAM_CS_n PORT DDR_CE = fpga_0_DDR_SDRAM_CE PORT DDR_Clk_n = fpga_0_DDR_SDRAM_Clk_n PORT DDR_Clk = fpga_0_DDR_SDRAM_Clk PORT MPMC_InitDone = DDR_SDRAM_MPMC_InitDone PORT MPMC_Clk_Mem = MPMC_Clk_Mem PORT MPMC_Clk0 = sys_clk_s PORT MPMC_Clk90 = sys_clk_s90 PORT MPMC_DCM_PSDONE = mpmc_dcm_PSDONE PORT MPMC_DCM_PSEN = mpmc_dcm_PSEN PORT MPMC_DCM_PSINCDEC = mpmc_dcm_PSINCDEC END
07-21-2010 12:30 PM
Is the static phy software example actually finishing? It should not calibrate if the that address behavior is consistent.
You are using variable positive mode on the DCM, and so the calibration example #define needs to be modified to reflect the bounds of the phase shift.
What is strange about this is that it appears that the some of the address locations are not written to at all. If it is true that the writes are not committing, there is some other bigger problem here- the MPMC static PHY only calibrates the read path- the write path is fixed and should always meet timing. What I would check is Section 6 of the map report to see if all of the control signals have an output flip flip (OFF/OFF1/OFF2) packed into the IOB for EVERY control signal. It is represented in one of the columns.
It may also make sense to try a newer version of MPMC.
I hope this helps,
07-26-2010 12:08 PM
>>Is the static phy software example actually finishing?
Actually, no it wasn't. I hacked it while troubleshooting. The times that I did succeed I was writing/reading the test patterns to addresses numbered as multiples of 8 only :smileyindifferent: (0, 0x8, 0x10, etc).
>> the calibration example #define needs to be modified to reflect the bounds of the phase shift.
Yes, and I did. The DCM is set up for VARIABLE_POSITIVE, so the phase shift boundaries are 0 to 179. I've also lowered the edge taps to 15 (from the original 20). Otherwise it would never succeed.
/************************** Constant Definitions *****************************/ /* * The following constants map to the XPAR parameters created in the * xparameters.h file. They are defined here such that a user can easily * change all the needed parameters in one place. */ #define MPMC_DEVICE_ID XPAR_MPMC_0_DEVICE_ID #define MPMC_BASEADDR XPAR_MPMC_0_MPMC_BASEADDR /* * Start Address of the Memory used for Calibration. Normally it is * the starting address of the memory connected to MPMC. */ #define MPMC_CALIBRATON_STARTADDR MPMC_BASEADDR /* * Use a smaller footprint printf so that the program will fit in BRAM */ #define printf xil_printf /* * Defines range of DCM Tap values. These values will differ based on the * FPGA FAMILY type and the frequency of operation. * Refer the FPGA user guide of the specific family for these ranges. * MPMC_NUMBER_TAPS needs to be less than 256. * MPMC_EDGE_TAPS should be set to the minimum acceptable range of valid taps. */ #define MPMC_MIN_TAPS 0 #define MPMC_MAX_TAPS 179 #define MPMC_EDGE_TAPS 15 #define MPMC_NUMBER_TAPS MPMC_MAX_TAPS - MPMC_MIN_TAPS /* * This is the setting the should be used if calibration read/writes will be on * port 0 only. It may be acceptable to have these settings if it is only * cache transfers that will be on a different port since cache transfers are * only executed once the word transfers have a valid setting. */ #define MPMC_RDEN_DELAY_MIN_VAL 0x00000000 #define MPMC_RDEN_DELAY_MAX_VAL 0xF0000000 #define MPMC_RDEN_DELAY_INC 0x10000000 /* * Specify byte lanes to calibrate over */ #define MPMC_CALIB0_MASK 0xFFFFFFFF #define MPMC_CALIB1_MASK 0xFFFFFFFF
>>What is strange about this is that it appears that the some of the address locations are not written to at all.
I disagree, if I write 0xFF's as above, then I would see 0xFF's on all address values:
XMD% mwr 0 0xffffffff 20 w XMD% mrd 0 20 0: FFFFFFFF 4: FFFFFFFF 8: FFFFFFFF C: FFFFFFFF 10: FFFFFFFF 14: FFFFFFFF 18: FFFFFFFF 1C: FFFFFFFF 20: FFFFFFFF 24: FFFFFFFF 28: FFFFFFFF 2C: FFFFFFFF 30: FFFFFFFF 34: FFFFFFFF 38: FFFFFFFF 3C: FFFFFFFF 40: FFFFFFFF 44: FFFFFFFF 48: FFFFFFFF 4C: FFFFFFFF
But writing zeros:
XMD% mrd 0 20 0: 00000000 4: 21000041 8: 00000000 C: 21000041 10: 00000000 14: 21000041 18: 00000000 1C: 21000041 20: 00000000 24: 21000041 28: 00000000 2C: 21000001 30: 00000000 34: 21000001 38: 00000000 3C: 21000041 40: 00000000 44: 21000001 48: 00000000 4C: 21000001
>> What I would check is Section 6 of the map report to see if all of the control signals have an output flip flip (OFF/OFF1/OFF2) packed into the IOB for EVERY control signal. It is represented in one of the columns.
They all do have an output flip flop. I've attached section 6 of the map report.
>>It may also make sense to try a newer version of MPMC.
I'll give this a shot.
08-03-2010 03:04 PM
>>It may also make sense to try a newer version of MPMC.
I got the same result. I'm getting the alternating 0's and 0x21000041 that I mentioned above.
Here's an interesting find. I was digging through the mpmc wrapper .ngc (in implementation/) and traced DQS. It comes down to a series of IOBUF's, as expected. The unexpected thing was that the IOBUF's output pin wasn't connected to anything. It is simply floating. In the synthesis report there is no mention of this signal being optimized out at all.
It was my understanding that the DQS was needed to properly read the data from the DDR memory device. Unless of course I'm missing something about the static PHY.
Attached is a screenshot of the find.
Any thoughts on this?
08-05-2010 03:21 PM
This is intended. The capture of the read data is calibrated by the software algorythm, by trying all possible settings until enough margin is found.
08-05-2010 03:28 PM
08-06-2010 10:55 AM
Thanks Dylan. The results are attached. I modified the MPMC_MAX_TAPS from 255 to 179. I believe this is the max for a 100MHz input clock, phase shift type: VARIABLE_POSITIVE.
These are the last 6 tap values, the rest are in the attachment:
Testing Tap 172,Regfile CC000000 Wr:DEADBEEF,11111111,22222222,33333333,44444444,55555555,66666666,77777777 Rd:27F6797D,21D6697D,27F6797D,21D6697D,27F6797D,21D6697D,27F6797D,21D6697D Testing Tap 173,Regfile CC000000 Wr:17171717,71717171,2D2D2D2D,D2D2D2D2,4B4B4B4B,B4B4B4B4,87878787,78787878 Rd:25D6297D,21D66BDD,25D6297D,21D66BDD,25D6297D,21D66BDD,25D6297D,21D66BDD Testing Tap 174,Regfile CC000000 Wr:DEADBEEF,11111111,22222222,33333333,44444444,55555555,66666666,77777777 Rd:25D6297D,21D669FD,25D6297D,21D669FD,25D6297D,21D669FD,25D6297D,21D669FD Testing Tap 175,Regfile CC000000 Wr:17171717,71717171,2D2D2D2D,D2D2D2D2,4B4B4B4B,B4B4B4B4,87878787,78787878 Rd:25F6397D,21D669FD,25F6397D,21D669FD,25F6397D,21D669FD,25F6397D,21D669FD Testing Tap 176,Regfile CC000000 Wr:DEADBEEF,11111111,22222222,33333333,44444444,55555555,66666666,77777777 Rd:27F6297D,219249E9,27F6297D,219249E9,27F6297D,219249E9,27F6297D,219249E9 Testing Tap 177,Regfile CC000000 Wr:17171717,71717171,2D2D2D2D,D2D2D2D2,4B4B4B4B,B4B4B4B4,87878787,78787878 Rd:21F6297D,21D66BFD,21F6297D,21D66BFD,21F6297D,21D66BFD,21F6297D,21D66BFD Testing Tap 178,Regfile CC000000 ERROR: Could not calibrate.
08-06-2010 03:48 PM
Check your schematic:
-Are there any unused pins that connect to the VRef supply? Consider setting the bitgen UnusedPin setting to None.
This would explain why you could write 1's but not 0's.
-How much trace skew do your datalines and DQS lines have?
Note that you can change the start reg value to less that 0x5000000, as long as it doesn't hang the controller. I just had it to save time.