Re: A proposal for a General Clustering Framework

[Alan Robertson <alanr@unix.sh>]

Peter Badovinatz wrote:
> 
> --- Alan Robertson <alanr@unix.sh> wrote:
> > Bill Todd wrote:
> > >
> > > An aspect that I haven't yet seen covered (though I might have missed it)
> > is
> > > the efficient processing of large chunks of data - e.g., file/database
> > pages
> > > (which these days can easily reach 128 KB in size and in the future might
> > > reasonably get even larger - in many cases, any size where the seek
> > > time/rotational latency eclipses the transfer time can be 'reasonable') and
> > > even larger chunks of streaming data.
> <snip>
> > The short answer to this is that this protocol is for control messages, not
> > bulk data transfers.  The needs of the two are somewhat different, in some
> > ways, radically different.  My belief is that no single mechanism can meet
> > the needs of the two parts well.
> >
> >       -- Alan Robertson
> >          alanr@unix.sh
> >
> Our experience with the RSCT clustering on IBM MPP systems was that our control
> level protocols (David Brower's message a few back in this thread had all of
> the relevant references to this work!) were NOT useful to the data transfer
> layers.  We support both failover (HACMP, RVSD) and cluster file systems (GPFS)
> with the RSCT clustering.  Our 'clients' do their own bulk data transfer.
> 
> Our protocols are oriented to multi-cast, small size, reliable, minimal
> overhead, etc., kinds of things we're talking about here.  The bulk data was
> mostly point-to-point, high performance, streaming type data.  We provided
> consistent membership, failover, events; they moved their data and used our
> notifications to do recovery.

Well said.  This is also the approach used by heartbeat, and the approach
I'd recommend here also...

	-- Alan Robertson
	   alanr@unix.sh

Linux-cluster: generic cluster infrastructure for Linux
Archive:       http://mail.nl.linux.org/linux-cluster/