Re: Huge contiguous memory allocation in userspace

[Bill Wendling <wendling@ganymede.isdn.uiuc.edu>]

Also sprach Joseph P. Garcia:
} A research group I am part of is having difficulty running in Linux a 
} simulation that uses huge amounts of memory accessed via a single array.  We 
} use the Portland Group's (www.pgroup.com) FORTRAN compiler (pghpf) to take 
} advantage of our dual-processor system.  However, it has 4GB of real memory, 
} and while kernels 2.4 and 2.2 w/bigmem can see all of the memory, our model 
} can only allocate up to about 2GB of memory before it will segfault.  I see 
} this also when I use C/GCC for a single huge malloc'ed or valloc'ed array, 
} but in smaller chunks, I can allocate up to 3GB.
} 
} I have tried many kernels and a few methods in userspace.  The Portland Group 
} assures me that their compiler does not have a limit in memory 
} accessing/addressing.  http://his.luky.org/ML/linux-kernel.2000/msg02108.html 
} mentions using mmap(), but I don't know how this would be used, especially in 
} FORTRAN.
} 
} How do I go about accessing the remaining 2GB of real memory for use in this 
} simulation?  Is there anyway to have virtually non-contiguous memory spaces 
} remapped to be addressed contigously?  Or am I totally confused here?
} 
} I apologize if this is not the list I should be posting this to or if this 
} has been covered before.  Any input would be appreceated. 
} 
Hello Joseph,

These are my first thoughts, which you may have already thought of but:
2GB is getting pretty close to the limit of a 32bit system, isn't it? You
may be running up against a physical barrier (i.e., your OS will want to
segment the address space).

Is it necessary to have the entire array in memory at one time? If not,
then maybe using a product like HDF5 (http://hdf.ncsa.uiuc.edu/) to read
specific pieces of the dataset for you would help...

-- 
|| Bill Wendling			wendling@ganymede.isdn.uiuc.edu
-
Kernelnewbies: Help each other learn about the Linux kernel.
Archive:       http://mail.nl.linux.org/
IRC Channel:   irc.openprojects.net / #kernelnewbies
Web Page:      http://www.surriel.com/kernelnewbies.shtml