Re: A proposal for a General Clustering Framework

[David Brower <David.Brower@oracle.com>]

Jeff Darcy wrote:
> 
> As for efficiency, this is where we get back to your point highlighted
> previously.  RPC forces *gross* algorithmic inefficiencies for operations
> needed by a cluster infrastructure.  RPC is oriented toward point-to-point
> or broadcast communication, while the most useful conceptual structure for
> many things in a cluster does(membership, voting, some parts of
> heartbeating) is likely to be a ring.  Aggregating point-to-point RPC into a
> star topology based around a temporarily-elected communications server
> doesn't work. Leader election is one of the services a cluster
> infrastructure *provides*, not one it consumes from elsewhere.  The only
> alternative implementable via RPC is all-to-all, which simply doesn't scale.
> In short, the conceptual model of RPC forces logical-topology decisions in
> directions that are far from optimal for clusters.
> 
> Please, look up some of these algorithms in Lynch or Mullender.  Think for a
> while about whether they can be adapted to use RPC, efficiently and without
> creating SPOFs.  Think about whether the kinds of timeouts and
> retransmissions inherent in the RPC model really match the requirements for
> those same sorts of things in a cluster environment.  RPC is not a bad
> thing, but it's a technological mismatch for cluster requirements.

I will confess to not understanding what Mr. Darcy has written
here.  I do not see how a point to point RPC from one node to another 
is any different than a single hop message in a ring.  I don't see where
timeouts and retransmissions come into play; I don't understand what
SPOF he thinks is present in an RPC.   We've already been assuming
the use of RPC-like encoding for messages that are unacknowledged
and possibly broad- or multicast.   These are capabilities present
in many RPC systems (but not XML-RPC in present form).

I think Mr. Darcy is making conclusions based on knowledge of some 
particular RPC systems.  He may be assuming the presence of routing 
daemons and things like name services in the RPC path; there is 
nothing requiring such things in many systems.

> > Wombat is right on another point that the ordering needed during
> > cluster transition (barrier services) aren't well served by an
> > event system.  I think that is an application for RPC.
> 
> See above for an explanation of why that is not so.

I still don't see it.  Elections involve messages between
voters; those messages could be RPCs.  Maybe they want to
be reliable/acknowledged messagees, maybe not, depending on
the election algorithm. 
 
> > My
> > impression is that many cluster people (including those I
> > work with) aren't adequately comfortable or familiar with
> > RPC systems
> 
> That's insulting.  It would be just as true to say that "RPC people" aren't
> adequately familiar with anything *but* the RPC model, and in fact I'm
> tempted to do so, but I think this conversation can move along more
> productively without such pissing contests.

Sorry, you found it insulting. It wasn't intended to be 
anything but a prod to do more research before concluding
RPC systems are inappropriate. 

If I were going to slam RPC, and XML-RPC/SOAP in particular, 
it would be the cpu expense of packing/unpacking the
messages-- and that is only a problem for the heartbeats, which
are frequent enough to cause background load.  I still don't
agree with bandwidth complaints, because I don't believe a 
cluster of really low-bandwidth is going to want to heartbeat
at a rate that presses the speed of the link for many reasons.

thanks,
-dB

Linux-cluster: generic cluster infrastructure for Linux
Archive:       http://mail.nl.linux.org/linux-cluster/