With the Cyclone V SoC, how can I use the FPGA fabric to access the HPS DMA?

[Altera_Forum]

Hello everyone, 

 I want to make something that records values that are in my FPGA. I thought of using the Cyclone V SoC DMA, I do have the Altera SoC kit.  With the Cyclone V SoC, how can I use the FPGA fabric to access the HPS DMA? I want to set a trigger in the HPS DMA, so that when I send a signal to high level at FPGA Fabric, the DMA starts to copy a FPGA FABRIC RAM content to the DDR connected in the HPS and visible by the Linux running in the ARM.  *Is this the easiest/fastest way of achieving what I want?

[Altera_Forum]

HPS and FPGA connected through AMBA bus. 

If HPS-DMA registers visible from FPGA, 2nd may signal to 1st. *And why you do not write data to HPS memory from FPGA direct in process accumulation as possible ?

[Altera_Forum]

There are up to 6 FPGA-to-SDRAM ports that you can push that data directly into the HPS SDRAM controller from the FPGA. If it was me I would just use those to push the data in directly and interrupt the processor after the data has been written to memory. Alternatively you could push the data through the F2H bridge into the ACP mapper so that the processor has a cache coherent view of the data automatically (depends on how much data you are sending over).

[Altera_Forum]

--- Quote Start ---  There are up to 6 FPGA-to-SDRAM ports that you can push that data directly into the HPS SDRAM controller from the FPGA. If it was me I would just use those to push the data in directly and interrupt the processor after the data has been written to memory. Alternatively you could push the data through the F2H bridge into the ACP mapper so that the processor has a cache coherent view of the data automatically (depends on how much data you are sending over).  --- Quote End ---    BadOmen do you have a link for a ref design or something that uses it that way? As far as I can see, I will need to limit a SDRAM area for the HPS, right? And reserve some for my FPGA fabric. I am finding very hard to find ref. designs for the HPS interacting with the FPGA.  Thanks for your answer, looks like the best way to do it. Maybe with a DMA (FPGA fabric) pushing to the SDRAM C

[Altera_Forum]

Actually I have a design planned to do exactly this but unfortunately it's not ready yet. You don't necessarily need to fence off a region of memory for the FPGA to access, you could dynamically allocate some space and pass the location to the FPGA logic to access later. 

 I think if you look at the golden hardware reference design there is a connection between some FPGA logic and the HPS SDRAM that you can take a look at. If I remember correctly there is a JTAG-to-Avalon memory mapped bridge connected which lets you peak and poke memory using system console. You'll want something more self contained but it would be a good starting point to look at to see how to configure the memory controller to give the FPGA access to it.

[Altera_Forum]

--- Quote Start ---  Actually I have a design planned to do exactly this but unfortunately it's not ready yet. You don't necessarily need to fence off a region of memory for the FPGA to access, you could dynamically allocate some space and pass the location to the FPGA logic to access later.  I think if you look at the golden hardware reference design there is a connection between some FPGA logic and the HPS SDRAM that you can take a look at. If I remember correctly there is a JTAG-to-Avalon memory mapped bridge connected which lets you peak and poke memory using system console. You'll want something more self contained but it would be a good starting point to look at to see how to configure the memory controller to give the FPGA access to it.  --- Quote End ---    Yup I saw this ref design, however it uses the F2H and H2F bridges and not the F2SDRAM, Can I just instiantiate a DMA IP in QSys and connect to the F2SDRAM bridge? QSys magically takes care of everything for me?  Could you share with me this "beta design" you have? My e-mail is wcprado { AT } hotmail { dot } com

[Altera_Forum]

You hit the nail on the head, Qsys will take care of any adaptation between the soft DMA logic and the interface into the HPS, that's why I use Qsys for integration because then I just worry about the IP being compliant to the interface spec and the tools will take care of gluing the IP together for me. 

 Unfortunately the design isn't very far along so it would be quicker to integrate it yourself. You would connect the lightweight H2F bridge to the slave port of the DMA so that the Cortex-A9 can control the DMA then connect the DMA masters to the HPS SDRAM ports. I said "ports" because you could have one port dedicated to reads and the other dedicated to writes and perform both reads and writes simulataneously and the HPS SDRAM controller will interleave them out to memory for you. If you use an existing DMA then I would select Avalon-MM ports for the SDRAM interface into the FPGA since the DMAs in Qsys are already Avalon-MM based (you can mix Avalon and AXI but that's just a waste of logic resources in this case).

[Altera_Forum]

--- Quote Start ---  You hit the nail on the head, Qsys will take care of any adaptation between the soft DMA logic and the interface into the HPS, that's why I use Qsys for integration because then I just worry about the IP being compliant to the interface spec and the tools will take care of gluing the IP together for me.  Unfortunately the design isn't very far along so it would be quicker to integrate it yourself. You would connect the lightweight H2F bridge to the slave port of the DMA so that the Cortex-A9 can control the DMA then connect the DMA masters to the HPS SDRAM ports. I said "ports" because you could have one port dedicated to reads and the other dedicated to writes and perform both reads and writes simulataneously and the HPS SDRAM controller will interleave them out to memory for you. If you use an existing DMA then I would select Avalon-MM ports for the SDRAM interface into the FPGA since the DMAs in Qsys are already Avalon-MM based (you can mix Avalon and AXI but that's just a waste of logic resources in this case).  --- Quote End ---   Thanks BadOmen, gonna try this.   I saw you talking about a SDRAM Memory Address Spammer or something like that, do I need it? What is it used for?

[Altera_Forum]

The F2S interfaces expose a 4GB address space so if your DMA is limitted in it's addressing capabilities you might need to put that address span bridge between it and the HPS. The DMA and SGDMA in Qsys only support up to 4GB but the mSGDMA up on alterawiki.com supports up to 64-bit addressing if you enable the enhanced descriptors. So if you used the mSGDMA you wouldn't need the address span bridge. 

 Regardless of the DMA addressing capabilities you might want to expose only a subset of the full 4GB SDRAM range so that the addressing logic in the DMA is not as wide. That typically drops the Fmax of the design when you try to master lots of memory so you might want to limit the address span using the address span bridge. For example lets say you only want to make addresses 0x1000_0000 to 0x100F_FFFF (1MB window) available you could setup the address span bridge to expose a 1MB range and have it hardcoded to start at 0x1000_0000.

[Altera_Forum]

--- Quote Start ---  The F2S interfaces expose a 4GB address space so if your DMA is limitted in it's addressing capabilities you might need to put that address span bridge between it and the HPS. The DMA and SGDMA in Qsys only support up to 4GB but the mSGDMA up on alterawiki.com supports up to 64-bit addressing if you enable the enhanced descriptors. So if you used the mSGDMA you wouldn't need the address span bridge.  Regardless of the DMA addressing capabilities you might want to expose only a subset of the full 4GB SDRAM range so that the addressing logic in the DMA is not as wide. That typically drops the Fmax of the design when you try to master lots of memory so you might want to limit the address span using the address span bridge. For example lets say you only want to make addresses 0x1000_0000 to 0x100F_FFFF (1MB window) available you could setup the address span bridge to expose a 1MB range and have it hardcoded to start at 0x1000_0000.  --- Quote End ---   I see! It makes sense.  I guess the same goes for the FPGA->SDRAM bridge with Altera Qsys DMA.  Just to make sure.  1. I've set the F2S SDRAM Bridge data to 64 bits WIDTH and write only, avalon-mm 2. I've added a SPAN EXTENDER, then i set the DATA PATH WIDTH to 64 bits and the ADDRESS WIDTH of the master to 32 bits (4 giga) and a default subwindow of 512kb 3. I've added an ALTERA DMA Controller and the WIDTH of the DMA LENGTH REGISTER is set to 32 bits 4. I've connected the DMA write master to the windowed slave on the SPAN EXTENDER 5. I've connected the expanded master to F2H_DATA  That's it? Now the Device Driver on HPS must set the right registers on the DMA Control port slave.

[Altera_Forum]

As long as when you say "F2H" you really mean F2S (i.e. the HPS SDRAM dedicated ports) then yes this is the correct way to set up the hardware. You can probably reduce the DMA length register width since you are only exposing 512kB of memory in the HPS at a time. 

 Make sure the DMA control port is connected to the lightweight HPS-to-FPGA bridge so that MPU can tell the DMA what to do.

[Altera_Forum]

Yes F2S, sorry for the typo. 

 Thanks for your help BadOmen, going to handle the FPGA Design to the firmware team and let's see if it works :)  Cheers

[Altera_Forum]

Where can find the reference design of F2S usage ?

[Altera_Forum]

So, I've tried this architecture and it worked!!!  

 However I am getting only 150MB/s throughtput, I expected around 300MB/s  I just took a look in the DMA Core and Enabled the Burst transfer. Then I configured it to 1024 max words. And I've also increased the FIFO depth to 256. I am going to try and see the results.  In the Address span extender, my data width is set to 64 bits. The burstcount Width is 1 bit and maximum pending reads is also set to 1.   Any ideas on a configuration that could help the throughput?       --- Quote Start ---  As long as when you say "F2H" you really mean F2S (i.e. the HPS SDRAM dedicated ports) then yes this is the correct way to set up the hardware. You can probably reduce the DMA length register width since you are only exposing 512kB of memory in the HPS at a time.  Make sure the DMA control port is connected to the lightweight HPS-to-FPGA bridge so that MPU can tell the DMA what to do.  --- Quote End ---  

[Altera_Forum]

Well with the Enabled Burst Transfer no more IRQs comes from the DMA.

[Altera_Forum]

I've tried some configurations with the DMA and the mem addr spammer but without success, can't get past 150MB/sec in the HW side... :(

[Altera_Forum]

So, I've increased the data width of the onchip ram to 64 bits and enabled only words/double-words transfers in the DMA. 

 With this I was able to get 224MB/Sec, a little bit better.   Final update:  The device driver wasn't enabling the double word transfer, with it enabled now I am hitting 324MBytes/Sec.

[Altera_Forum]

hi aprado,  

 I have do the QSYS interconnect as you mentions, but after trig DMA, it stay with BUSY bit, and can't go through...  Can you share your design for me ? ( FPGA project or QSYS file)  lucian.working@gmail.com  Thanks in advance.   Lucian

[Altera_Forum]

--- Quote Start ---  So, I've increased the data width of the onchip ram to 64 bits and enabled only words/double-words transfers in the DMA.  With this I was able to get 224MB/Sec, a little bit better.   Final update:  The device driver wasn't enabling the double word transfer, with it enabled now I am hitting 324MBytes/Sec.  --- Quote End ---    Hi Aprado,  Would it be possible that you posted your design or possibly email me? I'm trying to do a similar task but I'm having issues with the dma and how it gets controlled by hps.  nalkhateeb@teamist.com

[Altera_Forum]

--- Quote Start ---  Hi Aprado,  Would it be possible that you posted your design or possibly email me? I'm trying to do a similar task but I'm having issues with the dma and how it gets controlled by hps.  nalkhateeb@teamist.com  --- Quote End ---    I am also very interested in example driver code for controlling the DMA from the HPS. Would you please share? bill@procodeworks.com

[Altera_Forum]

Hi aprado, 

 Is it possible to share your design to s_kannoth14@informatik.uni-kl.de please. I am also have the same thing to do, but struggling to get it done.  Best Regards.