UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Adventurer
Adventurer
5,502 Views
Registered: ‎10-24-2016

AXI DMA interrupts only once // Max lenght packet for DMA transfer

Good afternoon,

 

I am trying to pass data to DDR3 memory through DMA. I am using an Spartan 6 XSLX45T (custom board) with one Microblaze core. 

 

I need to save data from an image sensor. Data is captured with IDDR2 flip flops and then, I create in the logic 32bits packets to send to DMA at 8.33MHz ( 1 packet of 32 bits every 60nsec). The idea is to increase the speed in the future, but at this moment, up to get it working, this is the speed selected.

 

So, i have created a custom VHDL core which pass data ( the 32bits packets) to AXI Stream bus, which is connected to AXI DMA S2MM channel. I have simulated my core and tested it with ISim and with SDK Debugger and it works fine. Handshaking is well done and the first transfer to dma is carried out.

 

A complete frame from the sensor is 67633152 bits, which is 67.633Mbits. The number of 32 bits packets I have to send to DDR3 memory is 4227072 packets. 

 

My idea was to send all data with a single DMA transfer, activated by an interrupt when the first 32bits packet is ready to be sent. But it seems that there is a lenght limit in a DMA transfer which is 8 MByte -1 (althought I have not achieve to send more than 512 packets). I have configured the buffer Length width with 23 bits (2^23 = 8388608) so I understand that it is not enough for my application because I need to transfer  67633152 bits for a complete frame.

 

DMA config.png

 

So the next option I am thinking to implement is to carry out the transfer of a complete frame in several parts, generating a new interrupt for each group of packets. For example, every 128 packets of 32 bits i would assert TLAST on my VHDL custom core to finish the transfer and the next packet ( Nº129 in my example) would generate a new interrupt and a new transfer would start for the next 128 packets of 32 bits. But the problem is that this dont work. only the first interrupt is carried out and not the next ones.

 

So, what could be happen?  Do I have to do something special to allow DMA engine to process new interrupts?? I make the acknowledge of the interrupt inside the interrupt handler so I understand that DMA should be ready for a new interrupt.

 

This is the code for Microblaze (DMA S2MM Interrupt handler):

 

 

static void DmaS2MMInterruptHandler (void *Callback)
{

    int Status;
    print ("SW has entered in DMA S2MM interrupt\n\r");
    u32 IrqStatus;
    int TimeOut;
    XAxiDma *AxiDmaInst = (XAxiDma *)Callback;


    /* Read pending interrupts */
    IrqStatus = XAxiDma_IntrGetIrq(AxiDmaInst, XAXIDMA_DEVICE_TO_DMA);

    xil_printf ("IrqStatus value is: %X \r\n", IrqStatus);

    /* Acknowledge pending interrupts */
    XAxiDma_IntrAckIrq(AxiDmaInst, IrqStatus, XAXIDMA_DEVICE_TO_DMA);

    print ("Acknowledge for DMA pending interrupts has been sent\n\r");

    /*
    	* If no interrupt is asserted, we do not do anything
    	*
    	* (XAXIDMA_IRQ_ALL_MASK = 0x00000770)
    */
    if (!(IrqStatus & XAXIDMA_IRQ_ALL_MASK)) {
    	print ("No active interrupts detected doing mask with XAXIDMA_IRQ_ALL_MASK\n\r");
    	print ("DMA S2MM Handler interrupt EXIT\n\r");
    	return;
    }

    /*
    	* If error interrupt is asserted, raise error flag, reset the
    	* hardware to recover from the error, and return with no further
    	* processing.
    	*
    	* (XAXIDMA_IRQ_ERROR_MASK = 0x00004000)
    */

    if (!(IrqStatus & XAXIDMA_IRQ_ERROR_MASK)) {

    	print ("Error detected while applying XAXIDMA_IRQ_ERROR_MASK \n\r");
    	Error = 1;

    	/* Reset could fail and hang
    	 * NEED a way to handle this or do not call it??
    	 */
    	XAxiDma_Reset(AxiDmaInst);

    	print ("AXI DMA instance RESET applied\n\r");

    	TimeOut = RESET_TIMEOUT_COUNTER;

    	print ("Waiting for the timeout of DMA RESET\r\n");

    	while (TimeOut) {
    		if(XAxiDma_ResetIsDone(AxiDmaInst)) {
    			print ("DMA engine has been RESET\n\r");
    			break;
    		}

    		TimeOut -= 1;
    	}

    	return;
    }
    else
    {
    	print ("No error detected on XAXIDMA_IRQ_ERROR_MASK\r\n");
    }

    /*
    	* If completion interrupt is asserted, then set RxDone flag
     */
    if ((IrqStatus & XAXIDMA_IRQ_IOC_MASK)) {

    	RxDone = 1;
    	print ("Completion interrupt has finished \n\r");
    	print ("RX Done Flag has been asserted\n\r ");
    }

    TransfersNumber = TransfersNumber -1;

}

 

And this is the part of the main function where i make a "while" loop which is waiting for the interrupts of DMA engine up to the variable TransfersNumber is "0". For this example TransfersNumber starts with value 2, and it is decreased inside DMA S2MM interrupt handler every time an interrupt in DMA engine is generated ( at least, this is what i expect to get, but it never get off of the while loop, so TransfersNumber never reaches value 0). Until this variable is expected to be "0", the code stays in the loop waiting for a new interrupt. Initial address ( Initial memory address (InitialAddress variable in XAxiDma_SimpleTransfer function is increased every time an interrupt occur to avoid overlapping data in memory.)

 

 

while (TransfersNumber !=0)
{
    Status = XAxiDma_SimpleTransfer(&AxiDma, InitialAddress,
			MAX_PKT_LEN, XAXIDMA_DEVICE_TO_DMA);
    if (Status = XST_SUCCESS){
	print ("DMA transfer completed succesfully\r\n");
    }
}

while (!RxDone)
{

}
Status = CheckData (MAX_PKT_LEN);
if (Status != XST_SUCCESS){
     print ("CheckData FAILED\n\r");
     return;
}

 

So my questions are:

 

1. Do I have a way to pass data from sensor to memory in a single DMA transfer?? Is there any other method to pass data directly to DDR3??

2. Why only one DMA interrupt is processed?? Do I have to call some special function to allow a new interrupt??

3. Do I have to carry out some special with AXI4 stream signals to allow generate a new interrupt??

 

If you need additional information, let me know. 

 

Thanks by advance for your answers. 

 

Regards.

 

 

 

 

0 Kudos
7 Replies
Adventurer
Adventurer
5,489 Views
Registered: ‎09-19-2016

Re: AXI DMA interrupts only once // Max lenght packet for DMA transfer

Hi,

 

have you checked how many bytes has been actually transferred after that interrupt (take a look at function XAxiDma_BdGetActualLength())?

 

Packets are delineated by the TLAST signal, so when you say: "For example, every 128 packets of 32 bits i would assert TLAST on my VHDL custom core to finish the transfer..." that means you have 1 packet of size 128 * 32b == 4096b == 1KB. Maybe you didn't set sizes of buffers large enough. 

 

Also, I think that after interrupt is asserted, IP module that sends data should wait for interrupt to be deasserted before starting new transfer (that was at least the case when we were implementing similar module).

 

Regards

 

 

0 Kudos
Adventurer
Adventurer
5,481 Views
Registered: ‎10-24-2016

Re: AXI DMA interrupts only once // Max lenght packet for DMA transfer

Hi @x_irie

 

Firstly many many thanks for your answer. 

 

The maximum number of 32 bits packets I can send now is 508. So 508x32= 16256 bits ( althought I have configured Buffer Lenght Field Widht in XPS with 23 bits, which should allow me to send up to 8388608 bits in a single transfer). In addition, It is strange because every 128 packets I assert TLAST signal in my core, and this should indicate the finish of the transfer, but if i configure MAX_PKT_LEN in SDK project to 508, the 508 packets are received and saved correctly on memory, so I dont know exactly why TLAST is doing anything when it is asserted by my core (maybe I am not understanding well how DMA interrupt works).

 

I dont know where I have to configure length of buffers as you say. Could you please tell me where Can I modify this parameter??

 

But as I have told in my example, I can send 128 packets of 32 bits, but only once. If I try to process a new interrupt, DMA S2MM interrupt handler is no more executed. 

 

Regarding to :

 

Also, I think that after interrupt is asserted, IP module that sends data should wait for interrupt to be deasserted before starting new transfer (that was at least the case when we were implementing similar module).

 

As i have told you, I am capturing data from an image sensor, and I can not stop the process until the last pixel has been received. I understand that in 60nsec (time between 32bits packets reception), DMA should be ready to process a new interrupt, but maybe not... 

 

So according to your experience, what would be the correct process to allow more than one interrupt in the DMA engine??

 

If DMA has this size limit, how is the correct configuration to save data from an image sensor?? Because I understand that i am not the first person trying to capture data from an image sensor.

 

Thanks again for your reply. 

 

Best regards.

0 Kudos
Adventurer
Adventurer
5,459 Views
Registered: ‎09-19-2016

Re: AXI DMA interrupts only once // Max lenght packet for DMA transfer

I should have asked this first. Do you use DMA in Scatter Gather or Simple mode? Is it single channel or multi channel?

 

I guess you are referencing to examples Xilinx provided here. I suggest you to make simple test design, such as this one, and than just experiment with that one, it will be easier for you to understand how DMA works when you know that IP core DMA is communicating with is working properly.

 

MAX_PKT_LEN should define how big your transfer is. It is expressed in bytes. So, if you assert TLAST after 128*32b == 4096b == 1024B, then MAX_PKT_LEN should be 1024.

 

Regards

 

0 Kudos
Adventurer
Adventurer
5,447 Views
Registered: ‎09-19-2016

Re: AXI DMA interrupts only once // Max lenght packet for DMA transfer

Wow, that was really bad math. :) 4096 bits is 512 bytes. Sorry :) MAX_PKT_LEN should be 512
0 Kudos
Adventurer
Adventurer
5,409 Views
Registered: ‎10-24-2016

Re: AXI DMA interrupts only once // Max lenght packet for DMA transfer

Hi again @x_irie.

 

The example you say ( Axi DMA engine from FPGA Developer) is the one I have used to start with the design. My custom core communicates with the Axi stream generator proposed in the example. The problem is that the example is for working in polled mode, and I understand (maybe I am wrong) that i have to work with interrupts because of the maximum lenght of the DMA transfer ( the idea as I said in last mail is to create transfers of , for example, 128 packets of 32 bits, and every time a complete packet has been sent by DMA, generate a new interrupt to process the next 128 packets). What I am scared is if the time to process the DMA interrupt is larger than the time that next data from sensor would be ready to be sent by the DMA. This means that probably i would loose data. 

 

I am working in simple mode (not scatter gather mode). Do you know if there is other option to send data bigger than DMA transfer lenght limit?? If not, do you think it would be possible for me to work with interrupts without loose data ( each packet is sent to DMA every 60nsec, and maybe processing the interrupt takes more time than 60nsec. Microblaze is working at 100MHz).

 

Anyway, I continue having problems to generate a new interrupt with DMA. Only first interrupt is handled, the followings are not taken into account and, the SW stops once the first packet is sent. 

 

Thanks again for your answer. I have now understood how to set MAX_PKT_LENGHT parameter.

 

Regards.

0 Kudos
Adventurer
Adventurer
5,400 Views
Registered: ‎09-19-2016

Re: AXI DMA interrupts only once // Max lenght packet for DMA transfer

Yep, that example indeed is for poll mode, but I thought you can just change the software, and use the one you want.

 

Nevertheless, maybe you can be able to send data that is bigger than DMA limit, at least to seem to you in SW that the things are happening like that (in HW you just can't actually do bigger transfers...). If you use Scatter Gather mode you can set interrupt to occur after some number of packets are transferred . So you can set as many packets as you want to, so that total sum of their sizes is equal to your desired transfer size. You can basically set N packets, each of sizes of DMA limit, and than tell the DMA to generate an interrupt after N packets are processed. Take a look at SG example, it is a bit complicated, but that has a good chance to solve your problem. COALESCING_COUNT is the parameter that you should pay attention to.

 

If you are afraid that you might lose data, have you considered inserting FIFO in which you can save data from your IP core if for some reason processing of DMA interrupt can cause you problems? I am not sure for how long DMA interrupt lasts, maybe you can insert ILA block and look at the signals?

0 Kudos
Adventurer
Adventurer
5,389 Views
Registered: ‎10-24-2016

Re: AXI DMA interrupts only once // Max lenght packet for DMA transfer

Thanks again for all your suppor @x_irie

 

I will try what you suggest with scatter gather mode but my time to finish this project is short, and I dont think I will be enough time to learn Chipscope and Scatter Gather, but as you suggest, this seems to be the only way to carry it out.

 

On the other hand, definitely, Only one interrupt is processed by DMA and I can not understand why. It doesnt mind what I try, only one interruption is carried out. 

 

If someone can help me with a small example whit DMA  serving more than one interrupt, please let me know, because I can not understand what I am doing wrong. XAxiDmaSimpleTransfer is inside a while loop, and every time an interrupt is detected, should repeat the transfer, but nothing...... ( the code is the one posted here).

 

Again thanks for your support @x_irie

0 Kudos