Apache

NAVIGATION
CATEGORIES
REFERRENCE
LINKS
  • Adding ClusterManager code the the codebase

    29 answers - 1216 bytes - related search similar search Add To My Delicious Add To My Stumble Upon Add To My Google Mark Add To My Facebook Add To My Digg Add To My Reddit

    Hi All,
    As we discussed and agreed sometime back we will be adding an abstraction
    layer to Axis2 to make it possible to transparently plug-in different
    clustering implementations. abstraction was named 'ClusterManager' and
    the interface that was proposed is given in [1] (with some slight
    modifications).
    I am hoping to go a step forward my adding this interface to the Axis2 code
    base. At initiation Axis2 will load a ClusterManager implementation object
    (if configured) and will call the methods of it whenever respective events
    occur.
    Chamikara
    [1] ClusterManager interface
    public abstract class ClusterManager {
    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context, String
    propertyName);
    public abstract void touchProperty (AbstractContext context, String
    propertyName);
    public abstract void commit ();
    }
  • No.1 | | 1660 bytes | |

    Chamikara,

    I do have half baked impl sitting on my laptop based on a slightly modified
    cluster manager.
    I will soon check this into a branch and you guys can then start playing
    around with it.

    Sorry about this taking time, I was so occupied with the Qpid work.

    Regarda,

    Rajith

    2/1/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:

    Hi All,

    As we discussed and agreed sometime back we will be adding an abstraction
    layer to Axis2 to make it possible to transparently plug-in different
    clustering implementations. abstraction was named 'ClusterManager' and
    the interface that was proposed is given in [1] (with some slight
    modifications).

    I am hoping to go a step forward my adding this interface to the Axis2
    code base. At initiation Axis2 will load a ClusterManager implementation
    object (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context, String
    propertyName);
    public abstract void touchProperty (AbstractContext context, String
    propertyName);
    public abstract void commit ();

    }
    >
    >
    >
  • No.2 | | 1995 bytes | |

    Rajith,

    What we are adding here is the interface that we agreed sometime back. So
    nothing prevents your implementation from working on top of that.

    BTW, what were the modifications that were done to the interface.

    Chamikara

    2/2/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    Chamikara,

    I do have half baked impl sitting on my laptop based on a slightly
    modified cluster manager.
    I will soon check this into a branch and you guys can then start playing
    around with it.

    Sorry about this taking time, I was so occupied with the Qpid work.

    Regarda,

    Rajith

    2/1/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:

    Hi All,

    As we discussed and agreed sometime back we will be adding an
    abstraction layer to Axis2 to make it possible to transparently plug-in
    different clustering implementations. abstraction was named
    'ClusterManager' and the interface that was proposed is given in [1] (with
    some slight modifications).

    I am hoping to go a step forward my adding this interface to the Axis2
    code base. At initiation Axis2 will load a ClusterManager implementation
    object (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context, String
    propertyName);
    public abstract void touchProperty (AbstractContext context, String
    propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
  • No.3 | | 2485 bytes | |

    Yes the implementation is build on top of the cluster manager.
    I changed the method signatures slightly. But it's only a prototype.

    I will not check it into the trunk, untill we run a successfull demo and
    everybody is comfortable.
    I will put that in a branch and then you guys can also make changes.

    Regards,

    Rajith

    2/2/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:

    Rajith,

    What we are adding here is the interface that we agreed sometime back. So
    nothing prevents your implementation from working on top of that.

    BTW, what were the modifications that were done to the interface.

    Chamikara
    >
    >
    >

    2/2/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    Chamikara,

    I do have half baked impl sitting on my laptop based on a slightly
    modified cluster manager.
    I will soon check this into a branch and you guys can then start playing
    around with it.

    Sorry about this taking time, I was so occupied with the Qpid work.

    Regarda,

    Rajith

    2/1/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:

    Hi All,

    As we discussed and agreed sometime back we will be adding an
    abstraction layer to Axis2 to make it possible to transparently plug-in
    different clustering implementations. abstraction was named
    'ClusterManager' and the interface that was proposed is given in [1] (with
    some slight modifications).

    I am hoping to go a step forward my adding this interface to the Axis2
    code base. At initiation Axis2 will load a ClusterManager implementation
    object (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
  • No.4 | | 3654 bytes | |

    Chamikara,

    The code is available under branches/clustering.

    Here are the changes I made to ClusterManager interface. course what I
    did is just a prototype so feel free to experiment and change things.

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);

    public abstract void updateState (AbstractContext context);

    As you can the changes are where I replaced the add/remove/touch property
    methods with a single updateState method.
    I found it to be very cumbersome/expensive to intercept the property changes
    and keep track of them on a per occurence basis.
    Instead a single updateState method called at the end of the invocation
    provides better alternative to handle the state changes. YMMV.

    looking forward to your comments on the prototype.

    Regards,

    Rajith

    2/2/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:
    Yes the implementation is build on top of the cluster manager.
    I changed the method signatures slightly. But it's only a prototype.

    I will not check it into the trunk, untill we run a successfull demo and
    everybody is comfortable.
    I will put that in a branch and then you guys can also make changes.

    Regards,

    Rajith
    >
    >
    >

    2/2/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Rajith,

    What we are adding here is the interface that we agreed sometime back.
    So nothing prevents your implementation from working on top of that.

    BTW, what were the modifications that were done to the interface.

    Chamikara
    >
    >
    >
    >
    >

    2/2/07, Rajith Attapattu < rajith77 (AT) gmail (DOT) comwrote:
    Chamikara,

    I do have half baked impl sitting on my laptop based on a slightly
    modified cluster manager.
    I will soon check this into a branch and you guys can then start
    playing around with it.

    Sorry about this taking time, I was so occupied with the Qpid work.

    Regarda,

    Rajith
    >
    >
    >

    2/1/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Hi All,

    As we discussed and agreed sometime back we will be adding an
    abstraction layer to Axis2 to make it possible to transparently plug-in
    different clustering implementations. abstraction was named
    'ClusterManager' and the interface that was proposed is given in [1] (with
    some slight modifications).

    I am hoping to go a step forward my adding this interface to the
    Axis2 code base. At initiation Axis2 will load a ClusterManager
    implementation object (if configured) and will call the methods of it
    whenever respective events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context,
    String propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >
    >
    >
    >
  • No.5 | | 5856 bytes | |

    Hi All,

    Since I am interested in distributed computing, I am closely monitoring this
    discussion on clustering. I am still going through the code to understand the
    Since I am new to this codebase I have very
    little understanding about most of the implementations. However I hope these
    comments will help you all to define a scalable, flexible clustering
    implementation for Axis2.

    In a distributed scenario having the ability to handle concurrency is very
    important. How do you tackle conflicting updates? There should be a way to
    rollback/commit. In otherwords the interface should be transactionable. Also
    having the ability to distinguish between read-only vs read/write access to
    the context is an opportunity to apply some optimizations to the actual
    implementation. I suggest that this interface should provide some means to do
    that too.

    I think the intention of abstracting the cluster manager to an interface is to
    provide extensibility so that we can have different cluster manager
    implementations based on different group communication techniques. But I
    think right now the interface has introduced more coupling.

    Following is an approach which could reduce coupling and enable more
    flexibility and extensibility.

    Do we have the ability to register a listener to the Context? What I mean is
    an instance of a class which implements an interface ContextListner. The
    ContextListner has one method "notify" which take a parameter of
    type "Event". The ClusterManager interface can extend the ContextListner
    (BTW, Should ClusterManger be an abstract class?)

    This way, in a particular node when there is change in the Context the
    ClusterManager instance in that node get notify. Then that ClusterManger
    instance can mulitcast (reliable mulitcasting) the change to other
    ClusterManger instances running in other nodes. Since each ClusterManager
    instance has a handler to the Context it can apply the change locally. If the
    reliable multicast is configured to keep the "total order" in message
    ordering you don't have to worry about the conflicts due to concurrency.

    This approach can be further optimized/enhanced based on different usecases.

    Thanks and best regards,
    Sanjaya

    * There is no difference
    Monday 05 February 2007 10:35, Rajith Attapattu wrote:
    Chamikara,

    The code is available under branches/clustering.

    Here are the changes I made to ClusterManager interface. course what I
    did is just a prototype so feel free to experiment and change things.

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);

    public abstract void updateState (AbstractContext context);

    As you can the changes are where I replaced the add/remove/touch property
    methods with a single updateState method.
    I found it to be very cumbersome/expensive to intercept the property
    changes and keep track of them on a per occurence basis.
    Instead a single updateState method called at the end of the invocation
    provides better alternative to handle the state changes. YMMV.

    looking forward to your comments on the prototype.

    Regards,

    Rajith

    2/2/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:
    Yes the implementation is build on top of the cluster manager.
    I changed the method signatures slightly. But it's only a prototype.

    I will not check it into the trunk, untill we run a successfull demo and

    everybody is comfortable.

    I will put that in a branch and then you guys can also make changes.

    Regards,

    Rajith

    2/2/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Rajith,

    What we are adding here is the interface that we agreed sometime back.

    So nothing prevents your implementation from working on top of that.

    BTW, what were the modifications that were done to the interface.

    Chamikara

    2/2/07, Rajith Attapattu < rajith77 (AT) gmail (DOT) comwrote:
    Chamikara,

    I do have half baked impl sitting on my laptop based on a slightly

    modified cluster manager.

    I will soon check this into a branch and you guys can then start

    playing around with it.

    Sorry about this taking time, I was so occupied with the Qpid work.

    Regarda,

    Rajith

    2/1/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Hi All,

    As we discussed and agreed sometime back we will be adding an

    abstraction layer to Axis2 to make it possible to transparently plug-in
    different clustering implementations. abstraction was named
    'ClusterManager' and the interface that was proposed is given in [1] (with
    some slight modifications).

    I am hoping to go a step forward my adding this interface to the

    Axis2 code base. At initiation Axis2 will load a ClusterManager
    implementation object (if configured) and will call the methods of it
    whenever respective events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context,

    String propertyName, propertyValue);

    public abstract void removeProperty (AbstractContext context,

    String propertyName);

    public abstract void touchProperty (AbstractContext context,

    String propertyName);

    public abstract void commit ();

    }
  • No.6 | | 6383 bytes | |

    Hi Sanjaya,

    2/5/07, Sanjaya Karunasena <sanjayak (AT) wso2 (DOT) comwrote:

    Hi All,

    Since I am interested in distributed computing, I am closely monitoring
    this
    discussion on clustering. I am still going through the code to understand
    the
    Since I am new to this codebase I have
    very
    little understanding about most of the implementations. However I hope
    these
    comments will help you all to define a scalable, flexible clustering
    implementation for Axis2.

    Great. Welcome to the club :-)

    In a distributed scenario having the ability to handle concurrency is very
    important. How do you tackle conflicting updates? There should be a way to
    rollback/commit. In otherwords the interface should be transactionable.
    Also
    having the ability to distinguish between read-only vs read/write access
    to
    the context is an opportunity to apply some optimizations to the actual
    implementation. I suggest that this interface should provide some means to
    do
    that too.

    Actually the updateState method we have now is doing something like
    commiting, so I suggest we rename that to 'commit'. We may also need to add
    too more methods to begin transactions and to rollback.

    I think the intention of abstracting the cluster manager to an interface is
    to
    provide extensibility so that we can have different cluster manager
    implementations based on different group communication techniques. But I
    think right now the interface has introduced more coupling.

    Following is an approach which could reduce coupling and enable more
    flexibility and extensibility.

    Do we have the ability to register a listener to the Context? What I mean
    is
    an instance of a class which implements an interface ContextListner. The
    ContextListner has one method "notify" which take a parameter of
    type "Event". The ClusterManager interface can extend the ContextListner
    (BTW, Should ClusterManger be an abstract class?)

    This way, in a particular node when there is change in the Context the
    ClusterManager instance in that node get notify. Then that ClusterManger
    instance can mulitcast (reliable mulitcasting) the change to other
    ClusterManger instances running in other nodes. Since eac h ClusterManager
    instance has a handler to the Context it can apply the change locally. If
    the
    reliable multicast is configured to keep the "total order" in message
    ordering you don't have to worry about the conflicts due to concurrency.

    This approach can be further optimized/enhanced based on different
    usecases.

    I think this approach is much flexible and reduces coupling. Rajith what do
    u think ?

    Chamikara

    Thanks and best regards,
    Sanjaya
    --
    * There is no difference
    Monday 05 February 2007 10:35, Rajith Attapattu wrote:
    Chamikara,

    The code is available under branches/clustering.

    Here are the changes I made to ClusterManager interface. course what
    I
    did is just a prototype so feel free to experiment and change things.

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);

    public abstract void updateState (AbstractContext context);

    As you can the changes are where I replaced the add/remove/touch
    property
    methods with a single updateState method.
    I found it to be very cumbersome/expensive to intercept the property
    changes and keep track of them on a per occurence basis.
    Instead a single updateState method called at the end of the invocation
    provides better alternative to handle the state changes. YMMV.

    looking forward to your comments on the prototype.

    Regards,

    Rajith

    2/2/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:
    Yes the implementation is build on top of the cluster manager.
    I changed the method signatures slightly. But it's only a prototype.

    I will not check it into the trunk, untill we run a successfull demo
    and

    everybody is comfortable.

    I will put that in a branch and then you guys can also make changes.

    Regards,

    Rajith

    2/2/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Rajith,

    What we are adding here is the interface that we agreed sometime
    back.

    So nothing prevents your implementation from working on top of that.

    BTW, what were the modifications that were done to the interface.

    Chamikara

    2/2/07, Rajith Attapattu < rajith77 (AT) gmail (DOT) comwrote:
    Chamikara,

    I do have half baked impl sitting on my laptop based on a slightly

    modified cluster manager.

    I will soon check this into a branch and you guys can then start

    playing around with it.

    Sorry about this taking time, I was so occupied with the Qpid
    work.

    Regarda,

    Rajith

    2/1/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Hi All,

    As we discussed and agreed sometime back we will be adding an

    abstraction layer to Axis2 to make it possible to transparently plug-in
    different clustering implementations. abstraction was named
    'ClusterManager' and the interface that was proposed is given in [1]
    (with
    some slight modifications).

    I am hoping to go a step forward my adding this interface to the
    --
    Axis2 code base. At initiation Axis2 will load a ClusterManager
    implementation object (if configured) and will call the methods of it
    whenever respective events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext
    context);
    public abstract void addProperty (AbstractContext context,

    String propertyName, propertyValue);

    public abstract void removeProperty (AbstractContext
    context,

    String propertyName);

    public abstract void touchProperty (AbstractContext context,

    String propertyName);

    public abstract void commit ();

    }
  • No.7 | | 9481 bytes | |

    Hello Sanjaya,

    comments are inline marked with [RA]

    Rajith Attapattu
    Red Hat.

    2/5/07, Sanjaya Karunasena <sanjayak (AT) wso2 (DOT) comwrote:

    Hi All,

    Since I am interested in distributed computing, I am closely monitoring
    this
    discussion on clustering. I am still going through the code to understand
    the
    Since I am new to this codebase I have
    very
    little understanding about most of the implementations. However I hope
    these
    comments will help you all to define a scalable, flexible clustering
    implementation for Axis2.

    [RA] thank you for your interest and welcome to scene.

    In a distributed scenario having the ability to handle concurrency is very
    important. How do you tackle conflicting updates? There should be a way to
    rollback/commit. In otherwords the interface should be transactionable.
    Also
    having the ability to distinguish between read-only vs read/write access
    to
    the context is an opportunity to apply some optimizations to the actual
    implementation.

    [RA] This is where locking comes into picture. There are 2 different types
    of locking mechanisms, pesimistic locking and optimistic locking.
    Most databases use pesimistic locking and is expensive and slow, while
    optimistic locking is relatively cheap. Negotiating concurrency issues is
    best handled by the underlying infrastructure. Apache Tribes (the impl I
    used and is using total ordering) and Richochet (garuntees are lower than
    VS) are both low level reliable multicasting infrastructure and have no
    support for locking.
    WADI from codehaus specializes in locking. But for now we decided sometime
    back to go without any locking for the following reasons.

    WADI is written for session replication and is difficult to adapt as is to
    the context replication scenario we have with out some modifications.
    Sanjiva and I debated on this issue for a while and agreed that we live with
    the concurrency issues for the time being.
    What we can do is to try to utilise WADI to provide locking support for our
    implementation as the second step.

    Detecting conflicts and rolling back is best decided at the infrastructure
    level (for example WADI) rather than at the application level.
    The logic for rolling back is complicated and if a rollback occurs we need
    to notify the web service client. Currently there is no standard WS way of
    doing that.
    (Not sure WS-Context has anything regarding this)

    We can have dirty reads (cheap) and do locking only for updates. But ymmv.

    I suggest that this interface should provide some means to do
    that too.

    [RA] As I said, the application should not be figuring out wether to
    rollback or not, the infrastructure should.
    so it should be the reverse. there should be a way for the infrastructure to
    notify the client about a rollback.

    I think the intention of abstracting the cluster manager to an interface is
    to
    provide extensibility so that we can have different cluster manager
    implementations based on different group communication techniques. But I
    think right now the interface has introduced more coupling.

    Following is an approach which could reduce coupling and enable more
    flexibility and extensibility.

    Do we have the ability to register a listener to the Context? What I mean
    is
    an instance of a class which implements an interface ContextListner. The
    ContextListner has one method "notify" which take a parameter of
    type "Event". The ClusterManager interface can extend the ContextListner
    (BTW, Should ClusterManger be an abstract class?)

    [RA} unfortunately your replication points can be outside of the context
    heirachy.
    For example instead of replicating every add/remove change happening inside
    an invocation we can choose to replicate the net result (change in state) at
    the end of the invocation. I think this is more efficient. The decesion to
    replicate something should not be limited to be taken within the context
    heirachy only.
    There should be flexibility to call the ClusterManager from anywhere. So a
    context listener is not sufficient.

    Furthermore your coupling is only reduced to the cluster manager. I also
    didn't like the prospect of calling cluster manager from all over the place,
    but given the current design of the context heirachy we have no choice.

    The other more cleaner approach that I proposed a while back was to use
    Aspects. We can use aspects to capture replication points using load time
    weaving in a more non intrusive manner. This way the context code is not
    littered with calls to the Cluster manager. I do have experimental version
    on my local copy, but it still needs more work.

    This way, in a particular node when there is change in the Context the
    ClusterManager instance in that node get notify. Then that ClusterManger
    instance can mulitcast (reliable mulitcasting) the change to other
    ClusterManger instances running in other nodes. Since each ClusterManager
    instance has a handler to the Context it can apply the change locally. If
    the
    reliable multicast is configured to keep the "total order" in message
    ordering you don't have to worry about the conflicts due to concurrency.

    [RA] This is what happens currently. Context creations are notified
    immediatly and state updates are notified at the end of an invocation.
    Tribes which is a total odering reliable multicast then does the work.
    However this will not prevent concurrency issues completely. Unless we do
    pesimistic locking we would still experiance dirty reads and potential
    write/write conflicts.

    This approach can be further optimized/enhanced based on different usecases.

    Thanks and best regards,
    Sanjaya
    --
    * There is no difference
    Monday 05 February 2007 10:35, Rajith Attapattu wrote:
    Chamikara,

    The code is available under branches/clustering.

    Here are the changes I made to ClusterManager interface. course what
    I
    did is just a prototype so feel free to experiment and change things.

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);

    public abstract void updateState (AbstractContext context);

    As you can the changes are where I replaced the add/remove/touch
    property
    methods with a single updateState method.
    I found it to be very cumbersome/expensive to intercept the property
    changes and keep track of them on a per occurence basis.
    Instead a single updateState method called at the end of the invocation
    provides better alternative to handle the state changes. YMMV.

    looking forward to your comments on the prototype.

    Regards,

    Rajith

    2/2/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:
    Yes the implementation is build on top of the cluster manager.
    I changed the method signatures slightly. But it's only a prototype.

    I will not check it into the trunk, untill we run a successfull demo
    and

    everybody is comfortable.

    I will put that in a branch and then you guys can also make changes.

    Regards,

    Rajith

    2/2/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Rajith,

    What we are adding here is the interface that we agreed sometime
    back.

    So nothing prevents your implementation from working on top of that.

    BTW, what were the modifications that were done to the interface.

    Chamikara

    2/2/07, Rajith Attapattu < rajith77 (AT) gmail (DOT) comwrote:
    Chamikara,

    I do have half baked impl sitting on my laptop based on a slightly

    modified cluster manager.

    I will soon check this into a branch and you guys can then start

    playing around with it.

    Sorry about this taking time, I was so occupied with the Qpid
    work.

    Regarda,

    Rajith

    2/1/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Hi All,

    As we discussed and agreed sometime back we will be adding an

    abstraction layer to Axis2 to make it possible to transparently plug-in
    different clustering implementations. abstraction was named
    'ClusterManager' and the interface that was proposed is given in [1]
    (with
    some slight modifications).

    I am hoping to go a step forward my adding this interface to the

    Axis2 code base. At initiation Axis2 will load a ClusterManager
    implementation object (if configured) and will call the methods of it
    whenever respective events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext
    context);
    public abstract void addProperty (AbstractContext context,

    String propertyName, propertyValue);

    public abstract void removeProperty (AbstractContext
    context,

    String propertyName);

    public abstract void touchProperty (AbstractContext context,

    String propertyName);

    public abstract void commit ();

    }
  • No.8 | | 9007 bytes | |

    Hi Rajith,

    Please find my comments below marked with [SK]

    Thanks
    Sanjaya

    Monday 05 February 2007 20:28, Rajith Attapattu wrote:
    Hello Sanjaya,

    comments are inline marked with [RA]

    Rajith Attapattu
    Red Hat.
    --
    In a distributed scenario having the ability to handle concurrency is very

    important. How do you tackle conflicting updates? There should be a way
    to rollback/commit. In otherwords the interface should be
    transactionable. Also
    having the ability to distinguish between read-only vs read/write access
    to
    the context is an opportunity to apply some optimizations to the actual
    implementation.

    [RA] This is where locking comes into picture. There are 2 different types
    of locking mechanisms, pesimistic locking and optimistic locking.
    Most databases use pesimistic locking and is expensive and slow, while
    optimistic locking is relatively cheap. Negotiating concurrency issues is
    best handled by the underlying infrastructure. Apache Tribes (the impl I
    used and is using total ordering) and Richochet (garuntees are lower than
    VS) are both low level reliable multicasting infrastructure and have no
    support for locking.
    WADI from codehaus specializes in locking. But for now we decided sometime
    back to go without any locking for the following reasons.

    WADI is written for session replication and is difficult to adapt as is to
    the context replication scenario we have with out some modifications.
    Sanjiva and I debated on this issue for a while and agreed that we live
    with the concurrency issues for the time being.
    What we can do is to try to utilise WADI to provide locking support for our
    implementation as the second step.

    [SK] Locking, which comes under pessimistic concurrency control is good for
    databases due to the high probability of conflicts. concurrency
    work best when there are high number of reads with intermittent writes. But
    which way to go depends on what do we expect in the context. We can put some
    constrains or let the end user to configure it. But I don't think this is a
    good place to over engineer things. We should keep this very simple. I guess
    this is similar to the HTTPSession. We don't expect application developers to
    use the HTTPSession as their in memory database. Am I right?

    Detecting conflicts and rolling back is best decided at the infrastructure
    level (for example WADI) rather than at the application level.
    The logic for rolling back is complicated and if a rollback occurs we need
    to notify the web service client. Currently there is no standard WS way of
    doing that.
    (Not sure WS-Context has anything regarding this)

    We can have dirty reads (cheap) and do locking only for updates. But ymmv.

    [SK] From the application developer point of view Axis2 itself is
    infrastructure. we should not pull him in to the picture. However
    application code which implements a refine requirement can provide some hints
    (like whether its K to have dirty reads).

    I suggest that this interface should provide some means to do

    that too.

    [RA] As I said, the application should not be figuring out wether to
    rollback or not, the infrastructure should.
    so it should be the reverse. there should be a way for the infrastructure
    to notify the client about a rollback.

    [SK] Agreed. But it will be good to shoot for some thing where there is no
    need to rollback.

    I think the intention of abstracting the cluster manager to an interface is

    to
    provide extensibility so that we can have different cluster manager
    implementations based on different group communication techniques. But I
    think right now the interface has introduced more coupling.

    Following is an approach which could reduce coupling and enable more
    flexibility and extensibility.

    Do we have the ability to register a listener to the Context? What I mean
    is
    an instance of a class which implements an interface ContextListner. The
    ContextListner has one method "notify" which take a parameter of
    type "Event". The ClusterManager interface can extend the ContextListner
    (BTW, Should ClusterManger be an abstract class?)

    [RA} unfortunately your replication points can be outside of the context
    heirachy.

    [SK] I am sorry, I did not understand this. What else do you want to
    replicate. In a cluster what we need is state information to be replicated.
    There are two places where usually an application developer maintain state,
    the session and the database. I guess with the Axis2 context option he get a
    third place. HTTPSession and databases should not be our problem. Let that to
    be handled by the infrastructure providers of those two. But yes, we need to
    have a story as to how to get end to end state replication working.

    For example instead of replicating every add/remove change happening
    inside an invocation we can choose to replicate the net result (change in
    state) at the end of the invocation. I think this is more efficient. The
    decesion to replicate something should not be limited to be taken within
    the context heirachy only.

    [SK] Yes this is efficient but leads to more conflicts. Why not let the end
    user (Application developer) decide that since he knows best the behavior of
    his application implementation.

    There should be flexibility to call the ClusterManager from anywhere. So a
    context listener is not sufficient.

    [SK] Well this sounds bit dangerous :-). I am sorry, I don't have enough
    context here.

    Furthermore your coupling is only reduced to the cluster manager. I also
    didn't like the prospect of calling cluster manager from all over the
    place, but given the current design of the context heirachy we have no
    choice.

    [SK] Well this actually could provide a good extension point to even
    application developers. In application server implementation of HTTPSession
    there is a way to register a listener. I have used it many times to clean up
    stale data in the session. But I may be wrong in this context.

    The other more cleaner approach that I proposed a while back was to use
    Aspects. We can use aspects to capture replication points using load time
    weaving in a more non intrusive manner. This way the context code is not
    littered with calls to the Cluster manager. I do have experimental version
    on my local copy, but it still needs more work.

    [SK] Aspect oriented approach work best when there are cross cutting concerns.
    I am not sure about the performance implications. I have used it for logging,
    tracing, throttling, etc. But certainly worth evaluating.

    This way, in a particular node when there is change in the Context the

    ClusterManager instance in that node get notify. Then that ClusterManger
    instance can mulitcast (reliable mulitcasting) the change to other
    ClusterManger instances running in other nodes. Since each ClusterManager
    instance has a handler to the Context it can apply the change locally. If
    the
    reliable multicast is configured to keep the "total order" in message
    ordering you don't have to worry about the conflicts due to concurrency.

    [RA] This is what happens currently. Context creations are notified
    immediatly and state updates are notified at the end of an invocation.
    Tribes which is a total odering reliable multicast then does the work.
    However this will not prevent concurrency issues completely. Unless we do
    pesimistic locking we would still experiance dirty reads and potential
    write/write conflicts.

    [SK] Well, total ordering to work, you need to maintain total ordering of
    events. In otherwords all the events occurred need to be notified. We can
    check with some carefull analysis whether causal ordering could work here. In
    that way things will be more efficient.

    Different group communication implementations are geared towards different QoS
    attributes. Application developer should have the flexibility to choose which
    one to go with. So we may want to enable different ClusterManger
    implementations to come in using different group communication
    implementations.

    However things can be futher simplify if we can make an assumption of a sticky
    session. I think that's some thing you can enforce and then manage simple
    complexities like transparent node recovery. With little feedback from the
    application developer this could also lead to flexible scalable
    implementation.

    I am just throwing out ideas so that we can trash out things. I suppose this
    is not information overloading :-).

    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
  • No.9 | | 1244 bytes | |

    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer much
    from the interface -- exactly which respective events are you referring
    to?
    -Bill

    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation object
    (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara

    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }

    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
  • No.10 | | 1494 bytes | |

    Bill,

    from what i understand, it's in a separate maven module. Not in
    kernel. So you can ignore it :)
    -- dims

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:
    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer much
    from the interface -- exactly which respective events are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation object
    (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >


    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
    --
  • No.11 | | 1935 bytes | |

    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the ConfigContext. This will
    be called in places like Context Creation and removal and may be property
    updates (hvnt fully decided yet). When Clustering is not present the cost
    should be reduced to a null check.

    Chamikara

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:

    Bill,

    from what i understand, it's in a separate maven module. Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:
    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer much
    from the interface -- exactly which respective events are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation object
    (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >


    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.12 | | 2173 bytes | |

    Chamikara,

    New notifications? like what we have already for services/modules?
    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the ConfigContext. This will
    be called in places like Context Creation and removal and may be property
    updates (hvnt fully decided yet). When Clustering is not present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module. Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer much
    from the interface -- exactly which respective events are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation object
    (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.13 | | 2768 bytes | |

    Hi Dims,

    As this is still is its design stage final approach hs not been decided. The
    current code is available in a branch so no affect on the performance of the
    trunk.

    A method of the ClusterManager instance hs to be called in events such as
    Context creation and removal. Currently these directly get called from
    respective places of the context classes (should happen only if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:

    Chamikara,

    New notifications? like what we have already for services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the ConfigContext. This
    will
    be called in places like Context Creation and removal and may be
    property
    updates (hvnt fully decided yet). When Clustering is not present the
    cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module. Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer
    much
    from the interface -- exactly which respective events are you
    referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation
    object
    (if configured) and will call the methods of it whenever
    respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context,
    String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.14 | | 3166 bytes | |

    That's what I inferred from the interface -- any time there is a context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for
    something arguably not core to a web services runtime.
    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events such
    as Context creation and removal. Currently these directly get called
    from respective places of the context classes (should happen only if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?
    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.15 | | 3582 bytes | |

    Hi Bill,

    This will be called only when clustering is enabled.

    Even then we most probably will not be calling it in every property change.
    Rather will be trying to detect updates to the property bags in certain
    places of the execution.

    Plus better approaches are always welcome :-)

    Chamikara

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:

    That's what I inferred from the interface -- any time there is a context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events such
    as Context creation and removal. Currently these directly get called
    from respective places of the context classes (should happen only if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.16 | | 1802 bytes | |

    Bill,

    If you addd <cluster class="
    "you will get
    replication.
    you can also just plug in any ClusterManager impl and the code will
    delegate to your impl.

    If that entry is missing the calls to ClusterManager goes to
    NullClusterManager which does nothing.
    Absolutely no performance implications (other than calling an empty method
    :)

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:

    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer much
    from the interface -- exactly which respective events are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation object
    (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >


    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
    --
  • No.17 | | 3909 bytes | |

    K Bill, not so fast :)

    I don't call the ClusterManager for every property Change.
    I have a slightly modified interface, not the one chamikara provided.

    I only replicate at the end of an invocation.
    So here are the no of calls per invocation.

    if first request, 3 calls.
    for Service Group ctx creation, one for service ctx creation, one for
    stateUpdate.

    for subsequent invocations
    only one call. State update.

    Now thats not bad.
    I don't think just 3 NP's at the max and 1 NP at the minimum is that much
    of a performance impact :)

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:

    That's what I inferred from the interface -- any time there is a context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events such
    as Context creation and removal. Currently these directly get called
    from respective places of the context classes (should happen only if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.18 | | 3910 bytes | |

    Chamikara please see my reply to Bill questions.
    As I mentioned I check and replicate state only at the end of an invocation.
    Thats why I changed the ClusterManager interface slightly.

    Rajith Attapattu
    Red Hat.

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:

    Hi Bill,

    This will be called only when clustering is enabled.

    Even then we most probably will not be calling it in every property
    change. Rather will be trying to detect updates to the property bags in
    certain places of the execution.

    Plus better approaches are always welcome :-)

    Chamikara
    --
    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.com wrote:

    That's what I inferred from the interface -- any time there is a context

    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on the

    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events such
    as Context creation and removal. Currently these directly get called
    from respective places of the context classes (should happen only if a

    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.19 | | 4296 bytes | |

    Further more if doing an empty method is a problem then if the cluster tag
    is empty I could just make the
    axisConfig.getClusterManager() return null instead of a NullClusterManager.

    This way I could do 3 checks for null instead of 3 NP calls.

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    K Bill, not so fast :)

    I don't call the ClusterManager for every property Change.
    I have a slightly modified interface, not the one chamikara provided.

    I only replicate at the end of an invocation.
    So here are the no of calls per invocation.

    if first request, 3 calls.
    for Service Group ctx creation, one for service ctx creation, one for
    stateUpdate.

    for subsequent invocations
    only one call. State update.

    Now thats not bad.
    I don't think just 3 NP's at the max and 1 NP at the minimum is that
    much of a performance impact :)

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:

    That's what I inferred from the interface -- any time there is a context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events such
    as Context creation and removal. Currently these directly get called
    from respective places of the context classes (should happen only if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.20 | | 4536 bytes | |

    +1 check for nulls is faster then NP method calls.

    PS: Spent way too much time last few days on exactly such calls :)

    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:
    Further more if doing an empty method is a problem then if the cluster tag
    is empty I could just make the
    axisConfig.getClusterManager() return null instead of a NullClusterManager.

    This way I could do 3 checks for null instead of 3 NP calls.

    Regards,

    Rajith Attapattu
    Red Hat.
    --
    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:
    K Bill, not so fast :)

    I don't call the ClusterManager for every property Change.
    I have a slightly modified interface, not the one chamikara provided.

    I only replicate at the end of an invocation.
    So here are the no of calls per invocation.

    if first request, 3 calls.
    for Service Group ctx creation, one for service ctx creation, one for
    stateUpdate.

    for subsequent invocations
    only one call. State update.

    Now thats not bad.
    I don't think just 3 NP's at the max and 1 NP at the minimum is that
    much of a performance impact :)

    Regards,

    Rajith Attapattu
    Red Hat.
    >
    >
    >

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:
    That's what I inferred from the interface -- any time there is a context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events such
    as Context creation and removal. Currently these directly get called
    from respective places of the context classes (should happen only if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.21 | | 4087 bytes | |

    Yeah, that seems improves the performance. +1 for the change.

    Chamikara

    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    Chamikara please see my reply to Bill questions.
    As I mentioned I check and replicate state only at the end of an
    invocation.
    Thats why I changed the ClusterManager interface slightly.

    Rajith Attapattu
    Red Hat.

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:

    Hi Bill,

    This will be called only when clustering is enabled.

    Even then we most probably will not be calling it in every property
    change. Rather will be trying to detect updates to the property bags in
    certain places of the execution.

    Plus better approaches are always welcome :-)

    Chamikara
    --
    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.com wrote:

    That's what I inferred from the interface -- any time there is a
    context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on
    the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events
    such
    as Context creation and removal. Currently these directly get called
    from respective places of the context classes (should happen only if
    a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath < chamikaramj (AT) gmail (DOT) com>
    wrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.

    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty
    (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.22 | | 4551 bytes | |

    Hi Rajith,

    Yes. This was one of the comments in my list :-)
    Lets do it that way and get rid of the NullClusterManager.

    Chamikara

    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    Further more if doing an empty method is a problem then if the cluster tag
    is empty I could just make the
    axisConfig.getClusterManager() return null instead of a
    NullClusterManager.

    This way I could do 3 checks for null instead of 3 NP calls.

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    K Bill, not so fast :)

    I don't call the ClusterManager for every property Change.
    I have a slightly modified interface, not the one chamikara provided.

    I only replicate at the end of an invocation.
    So here are the no of calls per invocation.

    if first request, 3 calls.
    for Service Group ctx creation, one for service ctx creation, one
    for stateUpdate.

    for subsequent invocations
    only one call. State update.

    Now thats not bad.
    I don't think just 3 NP's at the max and 1 NP at the minimum is that
    much of a performance impact :)

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:

    That's what I inferred from the interface -- any time there is a
    context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops) for

    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on
    the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events
    such
    as Context creation and removal. Currently these directly get called

    from respective places of the context classes (should happen only if
    a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal and
    may be property
    updates (hvnt fully decided yet). When Clustering is not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module.

    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to clustering? I
    can't infer much
    from the interface -- exactly which respective events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext
    context);
    public abstract void addContext (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty
    (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.23 | | 12977 bytes | |

    Hello Sanjaya,

    Thanks for the reply, comments marked with [RA]

    Regards,
    Rajith Attapattu
    Red Hat.

    2/5/07, Sanjaya Karunasena < sanjayak (AT) wso2 (DOT) comwrote:
    Hi Rajith,

    Please find my comments below marked with [SK]

    Thanks
    Sanjaya

    Monday 05 February 2007 20:28, Rajith Attapattu wrote:
    Hello Sanjaya,

    comments are inline marked with [RA]

    Rajith Attapattu
    Red Hat.
    --
    In a distributed scenario having the ability to handle concurrency is
    very

    important. How do you tackle conflicting updates? There should be a
    way
    to rollback/commit. In otherwords the interface should be
    transactionable. Also
    having the ability to distinguish between read-only vs read/write
    access
    to
    the context is an opportunity to apply some optimizations to the
    actual
    implementation.

    [RA] This is where locking comes into picture. There are 2 different
    types
    of locking mechanisms, pesimistic locking and optimistic locking.
    Most databases use pesimistic locking and is expensive and slow, while
    optimistic locking is relatively cheap. Negotiating concurrency issues
    is
    best handled by the underlying infrastructure. Apache Tribes (the impl I
    used and is using total ordering) and Richochet (garuntees are lower
    than
    VS) are both low level reliable multicasting infrastructure and have no
    support for locking.
    WADI from codehaus specializes in locking. But for now we decided
    sometime
    back to go without any locking for the following reasons.

    WADI is written for session replication and is difficult to adapt as is
    to
    the context replication scenario we have with out some modifications.
    Sanjiva and I debated on this issue for a while and agreed that we live
    with the concurrency issues for the time being.
    What we can do is to try to utilise WADI to provide locking support for
    our
    implementation as the second step.

    [SK] Locking, which comes under pessimistic concurrency control is good
    for
    databases due to the high probability of conflicts. concurrency
    work best when there are high number of reads with intermittent writes.
    But
    which way to go depends on what do we expect in the context. We can put
    some
    constrains or let the end user to configure it. But I don't think this is
    a
    good place to over engineer things. We should keep this very simple. I
    guess
    this is similar to the HTTPSession. We don't expect application developers
    to
    use the HTTPSession as their in memory database. Am I right?

    [RA] Yes, pessimistic locking is expensive and slow. If we really want to do
    locking, then we should do optimistic locking (WADI supports this).
    But for the first phase I would like to go without locking. Tomcat
    clustering is built on Tribes and they have no distributed locking.
    As you correctly pointed out the contexts are like the HTTPSession and
    shouldn't be abused by the Web Service developer to shove anything and
    everything into it and do frequent writes. This should be clearly stated in
    documentation.

    The expectation is to use some simple state (mostly read only with
    intermittent writes), just like a HTTPSession.

    However there is a difference btw the HTTPSession case and the context case
    that makes things a bit hard. With HTTPSession if we use session affinity
    a.k.a sticky sessions then we make sure requests for the same session is
    routed to the same node. So even w/o locking we can get rid of the
    concurrency issues. The replication happens merely for failover.

    But with Axis2 (if HTTP is not used as transport) we cannot do session
    afinity. So we might face concurrency issues. Unless we use Synapse as a
    Load Balancer that knows the interpret the serviceGroupId as the session
    key.

    Detecting conflicts and rolling back is best decided at the
    infrastructure
    level (for example WADI) rather than at the application level.
    The logic for rolling back is complicated and if a rollback occurs we
    need
    to notify the web service client. Currently there is no standard WS way
    of
    doing that.
    (Not sure WS-Context has anything regarding this)

    We can have dirty reads (cheap) and do locking only for updates. But
    ymmv.

    [SK] From the application developer point of view Axis2 itself is
    infrastructure. we should not pull him in to the picture.
    However
    application code which implements a refine requirement can provide some
    hints
    (like whether its K to have dirty reads).

    [RA] The only way to avoid dirty reads completly is to do pessimistic
    locking :(
    For now lets keep things simple. Later let us try to come up with a way to
    layer some locking mechanism on top of the cluster impl.

    I suggest that this interface should provide some means to do

    that too.

    [RA] As I said, the application should not be figuring out wether to
    rollback or not, the infrastructure should.
    so it should be the reverse. there should be a way for the
    infrastructure
    to notify the client about a rollback.
    --
    [SK] Agreed. But it will be good to shoot for some thing where there is no

    need to rollback.

    I think the intention of abstracting the cluster manager to an interface
    is

    to
    provide extensibility so that we can have different cluster manager
    implementations based on different group communication techniques. But
    I
    think right now the interface has introduced more coupling.

    Following is an approach which could reduce coupling and enable more
    flexibility and extensibility.

    Do we have the ability to register a listener to the Context? What I
    mean
    is
    an instance of a class which implements an interface ContextListner.
    The
    ContextListner has one method "notify" which take a parameter of
    type "Event". The ClusterManager interface can extend the
    ContextListner
    (BTW, Should ClusterManger be an abstract class?)

    [RA} unfortunately your replication points can be outside of the context

    heirachy.

    [SK] I am sorry, I did not understand this. What else do you want to
    replicate. In a cluster what we need is state information to be
    replicated.
    There are two places where usually an application developer maintain
    state,
    the session and the database. I guess with the Axis2 context option he get
    a
    third place. HTTPSession and databases should not be our problem. Let that
    to
    be handled by the infrastructure providers of those two. But yes, we need
    to
    have a story as to how to get end to end state replication working.

    [RA]Sorry for not being clear. What I meant was the decesion points to
    replicate (the exact replication points)
    may fall outside of the context heirachy. That's why I think we need the
    ClusterManager (atleast for the time being).
    For example update state.
    We can do this on per updateBasis (we can have a event listener like u said
    that creates event for an add/update/remove proeprty),
    instead, it's easy to call updateState at the end of the invocation from the
    engine or anywhere deem suitable and then figure out the net state change
    and then replicate. So we need more flexibility than events.
    Does that make sense ?

    For example instead of replicating every add/remove change happening
    inside an invocation we can choose to replicate the net result (change
    in
    state) at the end of the invocation. I think this is more
    efficient. The
    decesion to replicate something should not be limited to be taken within
    the context heirachy only.

    [SK] Yes this is efficient but leads to more conflicts. Why not let the
    end
    user (Application developer) decide that since he knows best the behavior
    of
    his application implementation.
    [RA] If you look at HTTPSession, the service author have no way of saying
    how or when to replicate.
    If the service author calls abuses this privilege then we have a
    performance issue on hand.
    Also I am not sure how the above way will lead to less conflicts? Can u
    explain more?

    There should be flexibility to call the ClusterManager from anywhere. So
    a
    context listener is not sufficient.

    [SK] Well this sounds bit dangerous :-). I am sorry, I don't have enough
    context here.

    [RA] Sorry what I meant was the decesion to replicate or points of
    replication can be anywhere within the system.
    (Not the state, the state is only encapsulated inside the ctx heirachy).

    Furthermore your coupling is only reduced to the cluster manager. I also
    didn't like the prospect of calling cluster manager from all over the
    place, but given the current design of the context heirachy we have no
    choice.

    [SK] Well this actually could provide a good extension point to even
    application developers. In application server implementation of
    HTTPSession
    there is a way to register a listener. I have used it many times to clean
    up
    stale data in the session. But I may be wrong in this context.

    The other more cleaner approach that I proposed a while back was to use
    Aspects. We can use aspects to capture replication points using load
    time
    weaving in a more non intrusive manner. This way the context code is not
    littered with calls to the Cluster manager. I do have experimental
    version
    on my local copy, but it still needs more work.
    --
    [SK] Aspect oriented approach work best when there are cross cutting
    concerns.
    I am not sure about the performance implications. I have used it for
    logging,
    tracing, throttling, etc. But certainly worth evaluating.
    [RA] I don't think it's a big problem with performance, but dims pointed out
    other issues where byte code manipulation may create to other non technical
    realted issues.
    My point cuts are the replication points. AP will allow me to intercept
    these points in a non intrusive manner.

    This way, in a particular node when there is change in the Context the

    ClusterManager instance in that node get notify. Then that
    ClusterManger
    instance can mulitcast (reliable mulitcasting) the change to other
    ClusterManger instances running in other nodes. Since each
    ClusterManager
    instance has a handler to the Context it can apply the change locally.
    If
    the
    reliable multicast is configured to keep the "total order" in message
    ordering you don't have to worry about the conflicts due to
    concurrency.

    [RA] This is what happens currently. Context creations are notified
    immediatly and state updates are notified at the end of an invocation.
    Tribes which is a total odering reliable multicast then does the work.
    However this will not prevent concurrency issues completely. Unless we
    do
    pesimistic locking we would still experiance dirty reads and potential
    write/write conflicts.
    --
    [SK] Well, total ordering to work, you need to maintain total ordering of
    events. In otherwords all the events occurred need to be notified. We can
    check with some carefull analysis whether causal ordering could work here.
    In
    that way things will be more efficient.

    [RA] I made mistake, Phlips confirmed that he hasn't implemented total
    ordering. Even JGroups doesn't have it.
    There is evs4j from Guglielmo Lichtner which claims total odering. The
    others seem to have different levels of garuntees.

    Different group communication implementations are geared towards different
    QoS
    attributes. Application developer should have the flexibility to choose
    which
    one to go with. So we may want to enable different ClusterManger
    implementations to come in using different group communication
    implementations.

    [RA] It is simple, all you need to do is to add the following to your
    axis2.xml
    <cluster class="" />

    You can put any implementation that implements the ClusterManager interface.

    However things can be futher simplify if we can make an assumption of a
    sticky
    session. I think that's some thing you can enforce and then manage simple
    complexities like transparent node recovery. With little feedback from the
    application developer this could also lead to flexible scalable
    implementation.

    [RA] see my above comment on session affinity and why it may not work for
    axis2.

    I am just throwing out ideas so that we can trash out things. I suppose
    this
    is not information overloading :-).

    [RA] not at all, I really like this conversation.

    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
    --
  • No.24 | | 4924 bytes | |

    Using the null object pattern is a bit more elegant :)
    But I certainly understand the importance of performance.

    As dims pointed out if a developer forgets to do the null check let them pay
    for it :)
    I will make this change ASAP.

    Rajith

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:

    Hi Rajith,

    Yes. This was one of the comments in my list :-)
    Lets do it that way and get rid of the NullClusterManager.

    Chamikara
    --
    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    Further more if doing an empty method is a problem then if the cluster
    tag is empty I could just make the
    axisConfig.getClusterManager() return null instead of a
    NullClusterManager.

    This way I could do 3 checks for null instead of 3 NP calls.

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    K Bill, not so fast :)

    I don't call the ClusterManager for every property Change.
    I have a slightly modified interface, not the one chamikara provided.

    I only replicate at the end of an invocation.
    So here are the no of calls per invocation.

    if first request, 3 calls.
    for Service Group ctx creation, one for service ctx creation, one
    for stateUpdate.

    for subsequent invocations
    only one call. State update.

    Now thats not bad.
    I don't think just 3 NP's at the max and 1 NP at the minimum is that
    much of a performance impact :)

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:

    That's what I inferred from the interface -- any time there is a
    context
    change or a property change, you want your interface to be invoked.
    That's an awful lot of method invocations (even if they're no-ops)
    for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect on
    the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in events
    such
    as Context creation and removal. Currently these directly get
    called
    from respective places of the context classes (should happen only
    if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) com>
    wrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal
    and
    may be property
    updates (hvnt fully decided yet). When Clustering is not

    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven
    module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone who
    doesn't want
    clustering/this particular approach to
    clustering? I
    can't infer much
    from the interface -- exactly which respective
    events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara
    Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init
    (ConfigurationContext
    context);
    public abstract void addContext
    (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty
    (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.25 | | 5039 bytes | |

    Great. :-)

    Chamikara

    2/7/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    Using the null object pattern is a bit more elegant :)
    But I certainly understand the importance of performance.

    As dims pointed out if a developer forgets to do the null check let them
    pay for it :)
    I will make this change ASAP.

    Rajith

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) comwrote:

    Hi Rajith,

    Yes. This was one of the comments in my list :-)
    Lets do it that way and get rid of the NullClusterManager.

    Chamikara
    --
    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    Further more if doing an empty method is a problem then if the cluster
    tag is empty I could just make the
    axisConfig.getClusterManager() return null instead of a
    NullClusterManager.

    This way I could do 3 checks for null instead of 3 NP calls.

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Rajith Attapattu <rajith77 (AT) gmail (DOT) comwrote:

    K Bill, not so fast :)

    I don't call the ClusterManager for every property Change.
    I have a slightly modified interface, not the one chamikara
    provided.

    I only replicate at the end of an invocation.
    So here are the no of calls per invocation.

    if first request, 3 calls.
    for Service Group ctx creation, one for service ctx creation,
    one for stateUpdate.

    for subsequent invocations
    only one call. State update.

    Now thats not bad.
    I don't think just 3 NP's at the max and 1 NP at the minimum is
    that much of a performance impact :)

    Regards,

    Rajith Attapattu
    Red Hat.

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:

    That's what I inferred from the interface -- any time there is a
    context
    change or a property change, you want your interface to be
    invoked.
    That's an awful lot of method invocations (even if they're no-ops)
    for
    something arguably not core to a web services runtime.

    -Bill

    Tue, 2007-02-06 at 21:32 +0530, Chamikara Jayalath wrote:
    Hi Dims,

    As this is still is its design stage final approach hs not been
    decided. The current code is available in a branch so no affect
    on the
    performance of the trunk.

    A method of the ClusterManager instance hs to be called in
    events such
    as Context creation and removal. Currently these directly get
    called
    from respective places of the context classes (should happen
    only if a
    ClusterManager hs been set i.e. when clustering is enabled).

    Chamikara
    --
    2/6/07, Davanum Srinivas < davanum (AT) gmail (DOT) comwrote:
    Chamikara,

    New notifications? like what we have already for
    services/modules?

    -- dims

    2/6/07, Chamikara Jayalath <chamikaramj (AT) gmail (DOT) com>
    wrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the
    ConfigContext. This will
    be called in places like Context Creation and removal
    and
    may be property
    updates (hvnt fully decided yet). When Clustering is
    not
    present the cost
    should be reduced to a null check.

    Chamikara
    >
    >
    >
    >

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven
    module.
    Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy < nagy (AT) watson (DOT) ibm.com wrote:
    What is the cost of this going to be for someone
    who
    doesn't want
    clustering/this particular approach to
    clustering? I
    can't infer much
    from the interface -- exactly which respective
    events
    are you referring
    to?

    -Bill
    --
    Thu, 2007-02-01 at 23:16 +0530, Chamikara
    Jayalath
    wrote:
    At initiation Axis2 will load a ClusterManager
    implementation object
    (if configured) and will call the methods of it
    whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init
    (ConfigurationContext
    context);
    public abstract void addContext
    (AbstractContext
    context);
    public abstract void removeContext
    (AbstractContext context);
    public abstract void addProperty
    (AbstractContext
    context, String
    propertyName, propertyValue);
    public abstract void removeProperty
    (AbstractContext context,
    String propertyName);
    public abstract void touchProperty
    (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }
    >
    >
    >
    >
    >
    >


    To unsubscribe, e-mail:
    axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail:
    axis-dev-help (AT) ws (DOT) apache.org
    >
    >
    >
    >
  • No.26 | | 917 bytes | |

    Hi All;

    I am not sure if this is the best place for this question, but I am
    stuck.

    I have a Axis2 client which I created from a service's WSDL using the
    xmlbeans data binding. I am now trying to connect to the live
    service, but the service is behind a proxy server. The proxy server
    monitors the HTTP headers, and based on custom field in the header
    will route the request to the correct machine. I am trying to add
    the HTTP Header information to my code for the client, but I am not
    sure how to do this. So far the best I have been able to do is this:

    //Set the HTTP Headers
    ServiceClient sc = stub._getServiceClient();
    options = ();
    options.setProperty("myKey", "myValue");
    (options);
    stub._setServiceClient(sc);

    I am hoping somebody may be able to help me here, or at least point
    me to some reference that discuss this

    Rob
  • No.27 | | 2049 bytes | |

    Hi Chamikara,

    I think it should be a configuration parameter which says whether clustering
    is enabled or disabled, defult to disabled.

    Sanjaya

    Tuesday 06 February 2007 20:46, Chamikara Jayalath wrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the ConfigContext. This will
    be called in places like Context Creation and removal and may be property
    updates (hvnt fully decided yet). When Clustering is not present the cost
    should be reduced to a null check.

    Chamikara

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module. Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:
    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer much
    from the interface -- exactly which respective events are you referring
    to?

    -Bill

    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation object
    (if configured) and will call the methods of it whenever respective
    events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context, String
    propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }

    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
  • No.28 | | 2322 bytes | |

    Let me take my comment back.

    I just looked at the way Rujith configure the cluster manager. So if that property is not there you can simplify this to a simple null check.

    Thanks
    Sanjaya

    Wednesday 07 February 2007 08:17, Sanjaya Karunasena wrote:
    Hi Chamikara,

    I think it should be a configuration parameter which says whether
    clustering is enabled or disabled, defult to disabled.

    Sanjaya

    Tuesday 06 February 2007 20:46, Chamikara Jayalath wrote:
    Hi Bill, Dims,

    The ClusterManager impl will be initiated with the ConfigContext. This
    will be called in places like Context Creation and removal and may be
    property updates (hvnt fully decided yet). When Clustering is not present
    the cost should be reduced to a null check.

    Chamikara

    2/6/07, Davanum Srinivas <davanum (AT) gmail (DOT) comwrote:
    Bill,

    from what i understand, it's in a separate maven module. Not in
    kernel. So you can ignore it :)

    -- dims

    2/6/07, Bill Nagy <nagy (AT) watson (DOT) ibm.comwrote:
    What is the cost of this going to be for someone who doesn't want
    clustering/this particular approach to clustering? I can't infer
    much from the interface -- exactly which respective events are you
    referring to?

    -Bill

    Thu, 2007-02-01 at 23:16 +0530, Chamikara Jayalath wrote:
    At initiation Axis2 will load a ClusterManager implementation
    object (if configured) and will call the methods of it whenever
    respective events occur.

    Chamikara
    --
    [1] ClusterManager interface

    public abstract class ClusterManager {

    public abstract void init (ConfigurationContext context);
    public abstract void addContext (AbstractContext context);
    public abstract void removeContext (AbstractContext context);
    public abstract void addProperty (AbstractContext context,
    String propertyName, propertyValue);
    public abstract void removeProperty (AbstractContext context,
    String propertyName);
    public abstract void touchProperty (AbstractContext context,
    String propertyName);
    public abstract void commit ();

    }

    To unsubscribe, e-mail: axis-dev-unsubscribe (AT) ws (DOT) apache.org
    For additional commands, e-mail: axis-dev-help (AT) ws (DOT) apache.org
  • No.29 | | 2357 byt