Varint32 Protobuf Filter

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Varint32 Protobuf Filter

Chris Molozian
This post has NOT been accepted by the mailing list yet.
Hi,

I'm relatively new to working with Grizzly. I'm currently trying to implement encoding and decoding support for Varint32 encoded message headers with Protocol Buffers messages. I've tried to take inspiration from the Netty4 classes:

https://github.com/netty/netty/blob/master/codec/src/main/java/io/netty/handler/codec/protobuf/ProtobufVarint32FrameDecoder.java
https://github.com/netty/netty/blob/master/codec/src/main/java/io/netty/handler/codec/protobuf/ProtobufVarint32LengthFieldPrepender.java

The current development effort is available here:
https://github.com/novabyte/grizzly-protobuf

I've written some tests to verify the implementation and unfortunately it's not working properly yet. I think it's an issue with my decoder. It's not detecting the varint32 encoded header properly and so I cannot decode the protobuf message.

I'm still debugging the issue to try and understand what I'm doing wrong. I'd appreciate any advice anyone can give about the performance of the implementation in general or the specifics of the decoding problem.

Many Thanks,

Chris
Reply | Threaded
Open this post in threaded view
|

Re: Varint32 Protobuf Filter

cdollins
This post has NOT been accepted by the mailing list yet.
This post was updated on .
This used to already be apart of the project, but I'm not sure that it ever
made it into the repo.

See https://github.com/cdollins/grizzly-google-protobufs for my version.

Thanks,
Chad


On Tue, Sep 3, 2013 at 6:56 AM, Chris Molozian [via Grizzly] <
ml-node+s1045725n5710209h4@n5.nabble.com> wrote:

> Hi,
>
> I'm relatively new to working with Grizzly. I'm currently trying to
> implement encoding and decoding support for Varint32 encoded message
> headers with Protocol Buffers messages. I've tried to take inspiration from
> the Netty4 classes:
>
>
> https://github.com/netty/netty/blob/master/codec/src/main/java/io/netty/handler/codec/protobuf/ProtobufVarint32FrameDecoder.java
>
> https://github.com/netty/netty/blob/master/codec/src/main/java/io/netty/handler/codec/protobuf/ProtobufVarint32LengthFieldPrepender.java
>
> The current development effort is available here:
> https://github.com/novabyte/grizzly-protobuf
>
> I've written some tests to verify the implementation and unfortunately
> it's not working properly yet. I think it's an issue with my decoder. It's
> not detecting the varint32 encoded header properly and so I cannot decode
> the protobuf message.
>
> I'm still debugging the issue to try and understand what I'm doing wrong.
> I'd appreciate any advice anyone can give about the performance of the
> implementation in general or the specifics of the decoding problem.
>
> Many Thanks,
>
> Chris
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://grizzly.1045725.n5.nabble.com/Varint32-Protobuf-Filter-tp5710209.html
>  To start a new topic under Grizzly - Users, email
> ml-node+s1045725n3726882h67@n5.nabble.com
> To unsubscribe from Grizzly, click here<http://grizzly.1045725.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3726881&code=Y2hhZC5kb2xsaW5zQGdtYWlsLmNvbXwzNzI2ODgxfDE0MDE2NTg2NzA=>
> .
> NAML<http://grizzly.1045725.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
Reply | Threaded
Open this post in threaded view
|

Re: Varint32 Protobuf Filter

Chris Molozian
This post has NOT been accepted by the mailing list yet.
Hey Chad,

Thanks for your link, I hadn't seen any code from Grizzly users implementing Protocol Buffers. There are some old mailing list posts about Protocol Buffers being up for discussion in the development meetings but I didn't think they amounted to anything.

I took a look through your code but as far as I'm aware it expects a fixed size header that contains the size of the Protobuf message being encoded/decoded. Does your codebase implement varint32 encoding of the message size as the header?

Cheers,

Chris
Reply | Threaded
Open this post in threaded view
|

Re: Varint32 Protobuf Filter

Chris Molozian
This post has NOT been accepted by the mailing list yet.
Hey all,

I've fixed the bug in my varint32 length header implementation. I've pushed the changes to GitHub ;)

Cheers,

Chris