Re: A proposal for a General Clustering Framework

["Bill Todd" <billtodd@foo.mv.com>]

An aspect that I haven't yet seen covered (though I might have missed it) is
the efficient processing of large chunks of data - e.g., file/database pages
(which these days can easily reach 128 KB in size and in the future might
reasonably get even larger - in many cases, any size where the seek
time/rotational latency eclipses the transfer time can be 'reasonable') and
even larger chunks of streaming data.

Facilities like (Illinois) Fast Messages at least potentially allow (FM's
ability may depend upon the implementation and/or its whims of the moment,
but one can get down-and-dirty in, e.g., NT and provide stronger guarantees)
the receiver of such data to use the start of the message to recognize the
contents and identify an appropriate receptacle (e.g., pre-allocated target
buffer[s]) where the payload can be dumped without extraneous copy activity
(copying being a non-negligible concern with payloads in the 100 KB -
multi-MB range).

It may be due to simple ignorance on my part, but it's not clear to me how a
generic XML processor would allow such optimization - or other FM-like
facilities such as the ability to process some short messages with
'handlers' at interrupt level when appropriate.

- bill



Linux-cluster: generic cluster infrastructure for Linux
Archive:       http://mail.nl.linux.org/linux-cluster/