a thought, asynchronous object serialization

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

a thought, asynchronous object serialization

wouter.hendrickx

Hi,

 

Yesterday I came up with a wild though concerning async object serialization, and I’m interested in feedback on the usefulness / feasibility of the idea…

 

While sending objects back and forth over an ObjectOutputStream wrapping a (classical non NIO) socket’s outputstream, I was wondering whether it would be possible to make a nonblocking implementation of what I was doing… JAVA’s implementation of Object streams is inherently synchronous, forcing both the reading and writing end to block upon any read/write operation. It would be possible to create a (pseudo-) nonblocking alternative if it was possible to determine the size of a completely serialized object beforehand, and then sending the size before you actually transmit the object. That way, the receiving end will know how many bytes are needed before the Object can be deserialized again in one go… (so no more socket operations are needed to deserialize the object)…

 

As I said, yesterday I came up with the idea that it is (at least theoretically) possible to make an asynchronous implementation of the serialization process! Indeed, as the serialized data conforms to a `grammar’ of some sort (I’m not sure whether this is the right term to use here), it is perfectly possible to process a few bytes, yielding a state containing a partially constructed object, and then continuing object reconstruction when more data comes available.

 

Such implementation would make it possible to accept socket connections, and to start reading whole objects from these connections, in an asynchronous matter, as there is now no need to read whole objects in one blow. This eliminates the need of a dedicated thread looping over calls to objectOutputStream.readObject() …

 

As my explanation is probably unclear, let’s write some pseudo-code

 

interface ObjectHandler {

    /**

     * Process a newly deserialized object

     * E.g. put object in some sort of queue, ...

     * @param object

     */

    void handle( Object object );

}

 

public interface ObjectDeserializer {

    /**

     * Called from the asynchronous socket API to offer new bytes

     * to the deserializer

     * If one or more complete object are read during the processing

     * of the buffer, they are passed on to the registered object

     * handler...

     * @param buffer

     */

    void offer( byte[] buffer );

 

    /**

     * Register a handler with this deserializer

     * @param handler

     */

    void setHandler( ObjectHandler handler );

}

 

 

So the class ObjectDeserializer will form the link between the low-level socket API and a (sort-of) asynchronous object producing API…

In my view, an implementation of ObjectDeserializer for the Hessian binary web protocol is certainly possible, on top of that, I suppose plain old java Object streams can have an asynchronous implementation, although it will be quite tedious to implement ;-)

 

I see applications for such a solution in software where lots of Object communication between many endpoints, in a send-and-forget manner. I came up with the idea while experimenting and thinking about point-to-point object communications within a (larger) computer cluster.

 

The reason I post this, is to poll whether there would be interest in such an implementation, and to communicate my idea to the wider world ;)

 

Kind Regards,

 

Wouter

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: a thought, asynchronous object serialization

charlie hunt-3
Hi Wouter,

If I'm correctly understanding what you are suggesting, and I think I am
....

Something that might be of interest to you and maybe you have already
looked at it is GlassFish CORBA (https://glassfish-corba.dev.java.net).
Although CORBA uses CDR, much of the functionality I think you'd want to
be able to do is gonna be similar, i.e. non-blocking Java NIO &
fragmented message support). There was also some work done in the past
to do Java serialization over GIOP. But, that has not been tested
thoroughly nor is it an "officially" supported configuration.

What I am trying to suggest is that you might find looking at how
GlassFish CORBA handles message fragments and also uses Java NIO useful
to formulate some additional ideas? I should also let you know that
GlassFish CORBA is in the process of migrating its use of non-blocking
Java NIO to use Grizzly. Some time in the near future you will see these
changes propagated into the GlassFish CORBA repository.

You might also find it useful to take a look at the CORBA specification
area where it talks about fragment messages or Fragment Message types.

hths,

charlie ...

[hidden email] wrote:

>
> Hi,
>
> Yesterday I came up with a wild though concerning async object
> serialization, and I’m interested in feedback on the usefulness /
> feasibility of the idea…
>
> While sending objects back and forth over an ObjectOutputStream
> wrapping a (classical non NIO) socket’s outputstream, I was wondering
> whether it would be possible to make a nonblocking implementation of
> what I was doing… JAVA’s implementation of Object streams is
> inherently synchronous, forcing both the reading and writing end to
> block upon any read/write operation. It would be possible to create a
> (pseudo-) nonblocking alternative if it was possible to determine the
> size of a completely serialized object beforehand, and then sending
> the size before you actually transmit the object. That way, the
> receiving end will know how many bytes are needed before the Object
> can be deserialized again in one go… (so no more socket operations are
> needed to deserialize the object)…
>
> As I said, yesterday I came up with the idea that it is (at least
> theoretically) possible to make an asynchronous implementation of the
> serialization process! Indeed, as the serialized data conforms to a
> `grammar’ of some sort (I’m not sure whether this is the right term to
> use here), it is perfectly possible to process a few bytes, yielding a
> state containing a partially constructed object, and then continuing
> object reconstruction when more data comes available.
>
> Such implementation would make it possible to accept socket
> connections, and to start reading whole objects from these
> connections, in an asynchronous matter, as there is now no need to
> read whole objects in one blow. This eliminates the need of a
> dedicated thread looping over calls to objectOutputStream.readObject() …
>
> As my explanation is probably unclear, let’s write some pseudo-code
>
> interface ObjectHandler {
>
> /**
>
> * Process a newly deserialized object
>
> * E.g. put object in some sort of queue, ...
>
> * @param object
>
> */
>
> void handle( Object object );
>
> }
>
> public interface ObjectDeserializer {
>
> /**
>
> * Called from the asynchronous socket API to offer new bytes
>
> * to the deserializer
>
> * If one or more complete object are read during the processing
>
> * of the buffer, they are passed on to the registered object
>
> * handler...
>
> * @param buffer
>
> */
>
> void offer( byte[] buffer );
>
> /**
>
> * Register a handler with this deserializer
>
> * @param handler
>
> */
>
> void setHandler( ObjectHandler handler );
>
> }
>
> So the class ObjectDeserializer will form the link between the
> low-level socket API and a (sort-of) asynchronous object producing API…
>
> In my view, an implementation of ObjectDeserializer for the Hessian
> binary web protocol is certainly possible, on top of that, I suppose
> plain old java Object streams can have an asynchronous implementation,
> although it will be quite tedious to implement ;-)
>
> I see applications for such a solution in software where lots of
> Object communication between many endpoints, in a send-and-forget
> manner. I came up with the idea while experimenting and thinking about
> point-to-point object communications within a (larger) computer cluster.
>
> The reason I post this, is to poll whether there would be interest in
> such an implementation, and to communicate my idea to the wider world ;)
>
> Kind Regards,
>
> Wouter
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: a thought, asynchronous object serialization

Jeanfrancois Arcand-2
In reply to this post by wouter.hendrickx
Hi,

sorry for the dealy...was on vacation and away from computers for two
weeks :-)

[hidden email] wrote:

> Hi,
>
>  
>
> Yesterday I came up with a wild though concerning async object
> serialization, and I’m interested in feedback on the usefulness /
> feasibility of the idea…
>
>  
>
> While sending objects back and forth over an ObjectOutputStream wrapping
> a (classical non NIO) socket’s outputstream, I was wondering whether it
> would be possible to make a nonblocking implementation of what I was
> doing… JAVA’s implementation of Object streams is inherently
> synchronous, forcing both the reading and writing end to block upon any
> read/write operation. It would be possible to create a (pseudo-)
> nonblocking alternative if it was possible to determine the size of a
> completely serialized object beforehand, and then sending the size
> before you actually transmit the object. That way, the receiving end
> will know how many bytes are needed before the Object can be
> deserialized again in one go… (so no more socket operations are needed
> to deserialize the object)…
>
>  
>
> As I said, yesterday I came up with the idea that it is (at least
> theoretically) possible to make an asynchronous implementation of the
> serialization process! Indeed, as the serialized data conforms to a
> `grammar’ of some sort (I’m not sure whether this is the right term to
> use here), it is perfectly possible to process a few bytes, yielding a
> state containing a partially constructed object, and then continuing
> object reconstruction when more data comes available.
>
>  
>
> Such implementation would make it possible to accept socket connections,
> and to start reading whole objects from these connections, in an
> asynchronous matter, as there is now no need to read whole objects in
> one blow. This eliminates the need of a dedicated thread looping over
> calls to objectOutputStream.readObject() …
>
>  
>
> As my explanation is probably unclear, let’s write some pseudo-code
>
>  
>
> interface ObjectHandler {
>
>     /**
>
>      * Process a newly deserialized object
>
>      * E.g. put object in some sort of queue, ...
>
>      * @param object
>
>      */
>
>     void handle( Object object );
>
> }
>
>  
>
> public interface ObjectDeserializer {
>
>     /**
>
>      * Called from the asynchronous socket API to offer new bytes
>
>      * to the deserializer
>
>      * If one or more complete object are read during the processing
>
>      * of the buffer, they are passed on to the registered object
>
>      * handler...
>
>      * @param buffer
>
>      */
>
>     void offer( byte[] buffer );
>
>  
>
>     /**
>
>      * Register a handler with this deserializer
>
>      * @param handler
>
>      */
>
>     void setHandler( ObjectHandler handler );
>
> }
>
>  
>
>  
>
> So the class ObjectDeserializer will form the link between the low-level
> socket API and a (sort-of) asynchronous object producing API…
>
> In my view, an implementation of ObjectDeserializer for the Hessian
> binary web protocol is certainly possible, on top of that, I suppose
> plain old java Object streams can have an asynchronous implementation,
> although it will be quite tedious to implement ;-)
>
>  
>
> I see applications for such a solution in software where lots of Object
> communication between many endpoints, in a send-and-forget manner. I
> came up with the idea while experimenting and thinking about
> point-to-point object communications within a (larger) computer cluster.
>
>  
>
> The reason I post this, is to poll whether there would be interest in
> such an implementation, and to communicate my idea to the wider world ;)

GlassFish deployment is based on object serialization via a Servlet. The
way I solved the problem for them (object deserialization) was to use a
temporary Selector pool to make sure the InputStream the Servlet is
using "blocks' until the entire object is read/deserialized. I would be
interested to explore your solution to see if that improve the current
design.

Thanks!

-- Jeanfrancois


>
>  
>
> Kind Regards,
>
>  
>
> Wouter
>
>  
>
>  
>
>  
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]