Re: Huge contiguous memory allocation in userspace

["Joseph A. Knapka" <jknapka@earthlink.net>]

Hi, Joseph,

Does the allocation succeed, and you get a segfault when trying to
access
the allocated memory? Or does the allocation just fail?

I don't know for sure what the cause of your problem is, but remember
that the virtual address space for user processes only extends from 0 to
PAGE_OFFSET, which is typically 3GB, so getting at any more real RAM
than
that is probably not possible. Also, in order to allocate a virtual
region
the kernel must be able to find a contiguous block of virtual space
large
enough to accommodate the request, so possibly virtual-space
fragmentation is
the cause of your trouble. In that case I would expect the malloc() call
to just fail.

You could get an authoritative answer by posting to the linux-mm list
(linux-mm@kvack.org).

HTH,

-- Joe

"Joseph P. Garcia" wrote:
> 
> A research group I am part of is having difficulty running in Linux a
> simulation that uses huge amounts of memory accessed via a single array.  We
> use the Portland Group's (www.pgroup.com) FORTRAN compiler (pghpf) to take
> advantage of our dual-processor system.  However, it has 4GB of real memory,
> and while kernels 2.4 and 2.2 w/bigmem can see all of the memory, our model
> can only allocate up to about 2GB of memory before it will segfault.  I see
> this also when I use C/GCC for a single huge malloc'ed or valloc'ed array,
> but in smaller chunks, I can allocate up to 3GB.
> 
> I have tried many kernels and a few methods in userspace.  The Portland Group
> assures me that their compiler does not have a limit in memory
> accessing/addressing.  http://his.luky.org/ML/linux-kernel.2000/msg02108.html
> mentions using mmap(), but I don't know how this would be used, especially in
> FORTRAN.
> 
> How do I go about accessing the remaining 2GB of real memory for use in this
> simulation?  Is there anyway to have virtually non-contiguous memory spaces
> remapped to be addressed contigously?  Or am I totally confused here?
> 
> I apologize if this is not the list I should be posting this to or if this
> has been covered before.  Any input would be appreceated.
> 
> Thank you for your time.
> 
> --
> Joseph P. Garcia
> Programmer - UW Lidar Group
> http://lidar.ssec.wisc.edu/
> -
> Kernelnewbies: Help each other learn about the Linux kernel.
> Archive:       http://mail.nl.linux.org/
> IRC Channel:   irc.openprojects.net / #kernelnewbies
> Web Page:      http://www.surriel.com/kernelnewbies.shtml

-- Joe Knapka
-
Kernelnewbies: Help each other learn about the Linux kernel.
Archive:       http://mail.nl.linux.org/
IRC Channel:   irc.openprojects.net / #kernelnewbies
Web Page:      http://www.surriel.com/kernelnewbies.shtml