Re: [Q] Testing ProtocolFilter

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [Q] Testing ProtocolFilter

Erik Svensson-2
On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]> wrote:

> I would like to know, how can I test my protocol filter. I would like to
> simulate, using unit tests, a truncated message.
>
>  
>
> I try invoking the connectorHandler's send() method twice (part1 and
> part2 of my message) with a Thread.sleep in between but the protocol
> parsers doesn't think tokens are missing, it sees the two different
> operations as two different message. Does this has something with TCP/IP
> where the stack is configured to "know" the packet received has been
> truncated or do I need to investigate my ProtocolFilter further. How
> should I test this?
>
>  
>
>             client.connect(new
> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>
>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
> String("\r\nPA").getBytes());
>
>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
> String("SS\r\n\0").getBytes());
>
>             client.send(outputByteBuffer1);
>
>             Thread.sleep(25);
>
>             client.send(outputByteBuffer2);
>

I haven't looked at your previously posted code but I've written a
ProtocolParser. tcp knows nothing about messages nor does it know anything
about the semantics of your message.
Your ProtocolParser needs to be able to determine if it has received a
complete message and then return true when the 'I need more data'-method is
called.

I also found that I had to do a compact() on my incoming bytebuffer when the
ReleaseBuffer() is called.
Otherwise, if you can't read a complete message the bytebuffer will just
fill up until there's no more space and you don't get any more messages.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Re: [Q] Testing ProtocolFilter

CigarMan
Have you filled up a ticket about the compacting issue?

Have you posted the code for your protocol parser? Here's mine, maybe
you can give me some tips! :.)

I will add your buffer compacting tip!

Simon

-----Original Message-----
From: Erik Svensson [mailto:[hidden email]]
Sent: March-07-08 10:56 AM
To: [hidden email]
Subject: Re: [Q] Testing ProtocolFilter

On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
wrote:

> I would like to know, how can I test my protocol filter. I would like
to
> simulate, using unit tests, a truncated message.
>
>  
>
> I try invoking the connectorHandler's send() method twice (part1 and
> part2 of my message) with a Thread.sleep in between but the protocol
> parsers doesn't think tokens are missing, it sees the two different
> operations as two different message. Does this has something with
TCP/IP

> where the stack is configured to "know" the packet received has been
> truncated or do I need to investigate my ProtocolFilter further. How
> should I test this?
>
>  
>
>             client.connect(new
> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>
>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
> String("\r\nPA").getBytes());
>
>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
> String("SS\r\n\0").getBytes());
>
>             client.send(outputByteBuffer1);
>
>             Thread.sleep(25);
>
>             client.send(outputByteBuffer2);
>
I haven't looked at your previously posted code but I've written a
ProtocolParser. tcp knows nothing about messages nor does it know
anything
about the semantics of your message.
Your ProtocolParser needs to be able to determine if it has received a
complete message and then return true when the 'I need more data'-method
is
called.

I also found that I had to do a compact() on my incoming bytebuffer when
the
ReleaseBuffer() is called.
Otherwise, if you can't read a complete message the bytebuffer will just
fill up until there's no more space and you don't get any more messages.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

MyProtocolParser.java (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Q] Testing ProtocolFilter

Erik Svensson-3

On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:

> Have you filled up a ticket about the compacting issue?

That I haven't.

> Have you posted the code for your protocol parser? Here's mine, maybe
> you can give me some tips! :.)

I haven't done that either. We're looking at using grizzly inhouse  
and it was in that context I wrote
the protocol parser. I had thoughts about writing a tutorial/blog  
post about it but there aren't all that much time right now.
At work I have more pressing concerns. I

I attach my protocol parser to this mail so you can look at it. It's  
not tested with 1.7.2, though.

Basically it uses a class XFlowMessage that contains the parser to  
parse the byte buffer into xflow messages.

> I will add your buffer compacting tip!

I'll look at your code and see if I can come up with something. Just  
the thing the wife wants me to do on a friday evening :-)

cheers

/Erik



>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 10:56 AM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
> wrote:
>
>> I would like to know, how can I test my protocol filter. I would like
> to
>> simulate, using unit tests, a truncated message.
>>
>>
>>
>> I try invoking the connectorHandler's send() method twice (part1 and
>> part2 of my message) with a Thread.sleep in between but the protocol
>> parsers doesn't think tokens are missing, it sees the two different
>> operations as two different message. Does this has something with
> TCP/IP
>> where the stack is configured to "know" the packet received has been
>> truncated or do I need to investigate my ProtocolFilter further. How
>> should I test this?
>>
>>
>>
>>             client.connect(new
>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>
>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>> String("\r\nPA").getBytes());
>>
>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>> String("SS\r\n\0").getBytes());
>>
>>             client.send(outputByteBuffer1);
>>
>>             Thread.sleep(25);
>>
>>             client.send(outputByteBuffer2);
>>
>
> I haven't looked at your previously posted code but I've written a
> ProtocolParser. tcp knows nothing about messages nor does it know
> anything
> about the semantics of your message.
> Your ProtocolParser needs to be able to determine if it has received a
> complete message and then return true when the 'I need more data'-
> method
> is
> called.
>
> I also found that I had to do a compact() on my incoming bytebuffer  
> when
> the
> ReleaseBuffer() is called.
> Otherwise, if you can't read a complete message the bytebuffer will  
> just
> fill up until there's no more space and you don't get any more  
> messages.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
> <MyProtocolParser.java>-----------------------------------------------
> ----------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

XFlowProtocolParser.java (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Re: [Q] Testing ProtocolFilter

CigarMan
Thanks!

Simon

-----Original Message-----
From: Erik Svensson [mailto:[hidden email]]
Sent: March-07-08 1:35 PM
To: [hidden email]
Subject: Re: [Q] Testing ProtocolFilter


On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:

> Have you filled up a ticket about the compacting issue?

That I haven't.

> Have you posted the code for your protocol parser? Here's mine, maybe
> you can give me some tips! :.)

I haven't done that either. We're looking at using grizzly inhouse  
and it was in that context I wrote
the protocol parser. I had thoughts about writing a tutorial/blog  
post about it but there aren't all that much time right now.
At work I have more pressing concerns. I

I attach my protocol parser to this mail so you can look at it. It's  
not tested with 1.7.2, though.

Basically it uses a class XFlowMessage that contains the parser to  
parse the byte buffer into xflow messages.

> I will add your buffer compacting tip!

I'll look at your code and see if I can come up with something. Just  
the thing the wife wants me to do on a friday evening :-)

cheers

/Erik



>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 10:56 AM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
> wrote:
>
>> I would like to know, how can I test my protocol filter. I would like
> to
>> simulate, using unit tests, a truncated message.
>>
>>
>>
>> I try invoking the connectorHandler's send() method twice (part1 and
>> part2 of my message) with a Thread.sleep in between but the protocol
>> parsers doesn't think tokens are missing, it sees the two different
>> operations as two different message. Does this has something with
> TCP/IP
>> where the stack is configured to "know" the packet received has been
>> truncated or do I need to investigate my ProtocolFilter further. How
>> should I test this?
>>
>>
>>
>>             client.connect(new
>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>
>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>> String("\r\nPA").getBytes());
>>
>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>> String("SS\r\n\0").getBytes());
>>
>>             client.send(outputByteBuffer1);
>>
>>             Thread.sleep(25);
>>
>>             client.send(outputByteBuffer2);
>>
>
> I haven't looked at your previously posted code but I've written a
> ProtocolParser. tcp knows nothing about messages nor does it know
> anything
> about the semantics of your message.
> Your ProtocolParser needs to be able to determine if it has received a
> complete message and then return true when the 'I need more data'-
> method
> is
> called.
>
> I also found that I had to do a compact() on my incoming bytebuffer  
> when
> the
> ReleaseBuffer() is called.
> Otherwise, if you can't read a complete message the bytebuffer will  
> just
> fill up until there's no more space and you don't get any more  
> messages.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
> <MyProtocolParser.java>-----------------------------------------------

> ----------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [Q] Testing ProtocolFilter

Erik Svensson-3
>

Simon,

I found something a bit odd but that might be because you've edited  
the code before posting it.
In the hasNextMessage() method there's this snippet of code:

  86 }
  87         /*
  88          * Validates that the buffer contains at least 2 bytes  
so we can proceed to input validation
  89          */
  90         if(bufferContent.length < 2)
  91         {
  92         isExpectingMoreData = true;
  93         return !isExpectingMoreData;
  94
  95         // 1.1)
  96         if(CR == bufferContent[0])
  97         {
  98         if(LF == bufferContent[1])
  99         {
100         // 1.1.1)
101     if(bufferContentString.contains(PASS.toString()))
102     {
103     if(LOG.isTraceEnabled())
104     {
105     LOG.trace("Found: PASS");
106     }
107     isExpectingMoreData = false;
108                     message = PASS.toString();
109                     position = PASS.toBytes().length;
110     }

Note on line 93 the return without any closing curly bracket.  
However, this shouldn't compile
as the code below is unreachable. Am I correct in assuming that there  
should be
a closing bracket on line 94?
(the indention is a bit off due to being cut'n'pasted into my mail  
client).

cheers
Erik





> Thanks!
>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 1:35 PM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
>
> On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:
>
>> Have you filled up a ticket about the compacting issue?
>
> That I haven't.
>
>> Have you posted the code for your protocol parser? Here's mine, maybe
>> you can give me some tips! :.)
>
> I haven't done that either. We're looking at using grizzly inhouse
> and it was in that context I wrote
> the protocol parser. I had thoughts about writing a tutorial/blog
> post about it but there aren't all that much time right now.
> At work I have more pressing concerns. I
>
> I attach my protocol parser to this mail so you can look at it. It's
> not tested with 1.7.2, though.
>
> Basically it uses a class XFlowMessage that contains the parser to
> parse the byte buffer into xflow messages.
>
>> I will add your buffer compacting tip!
>
> I'll look at your code and see if I can come up with something. Just
> the thing the wife wants me to do on a friday evening :-)
>
> cheers
>
> /Erik
>
>
>
>>
>> Simon
>>
>> -----Original Message-----
>> From: Erik Svensson [mailto:[hidden email]]
>> Sent: March-07-08 10:56 AM
>> To: [hidden email]
>> Subject: Re: [Q] Testing ProtocolFilter
>>
>> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
>> wrote:
>>
>>> I would like to know, how can I test my protocol filter. I would  
>>> like
>> to
>>> simulate, using unit tests, a truncated message.
>>>
>>>
>>>
>>> I try invoking the connectorHandler's send() method twice (part1 and
>>> part2 of my message) with a Thread.sleep in between but the protocol
>>> parsers doesn't think tokens are missing, it sees the two different
>>> operations as two different message. Does this has something with
>> TCP/IP
>>> where the stack is configured to "know" the packet received has been
>>> truncated or do I need to investigate my ProtocolFilter further. How
>>> should I test this?
>>>
>>>
>>>
>>>             client.connect(new
>>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>>
>>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>>> String("\r\nPA").getBytes());
>>>
>>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>>> String("SS\r\n\0").getBytes());
>>>
>>>             client.send(outputByteBuffer1);
>>>
>>>             Thread.sleep(25);
>>>
>>>             client.send(outputByteBuffer2);
>>>
>>
>> I haven't looked at your previously posted code but I've written a
>> ProtocolParser. tcp knows nothing about messages nor does it know
>> anything
>> about the semantics of your message.
>> Your ProtocolParser needs to be able to determine if it has  
>> received a
>> complete message and then return true when the 'I need more data'-
>> method
>> is
>> called.
>>
>> I also found that I had to do a compact() on my incoming bytebuffer
>> when
>> the
>> ReleaseBuffer() is called.
>> Otherwise, if you can't read a complete message the bytebuffer will
>> just
>> fill up until there's no more space and you don't get any more
>> messages.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>> <MyProtocolParser.java>----------------------------------------------
>> -
>
>> ----------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Re: [Q] Testing ProtocolFilter

CigarMan
You are most certainly right, a curly brace is missing on line 94.
Thanks,

Simon

-----Original Message-----
From: Erik Svensson [mailto:[hidden email]]
Sent: March-07-08 2:49 PM
To: [hidden email]
Subject: Re: [Q] Testing ProtocolFilter

>

Simon,

I found something a bit odd but that might be because you've edited  
the code before posting it.
In the hasNextMessage() method there's this snippet of code:

  86 }
  87         /*
  88          * Validates that the buffer contains at least 2 bytes  
so we can proceed to input validation
  89          */
  90         if(bufferContent.length < 2)
  91         {
  92         isExpectingMoreData = true;
  93         return !isExpectingMoreData;
  94
  95         // 1.1)
  96         if(CR == bufferContent[0])
  97         {
  98         if(LF == bufferContent[1])
  99         {
100         // 1.1.1)
101
if(bufferContentString.contains(PASS.toString()))
102     {
103     if(LOG.isTraceEnabled())
104     {
105     LOG.trace("Found:
PASS");
106     }
107     isExpectingMoreData = false;
108                     message = PASS.toString();
109                     position = PASS.toBytes().length;
110     }

Note on line 93 the return without any closing curly bracket.  
However, this shouldn't compile
as the code below is unreachable. Am I correct in assuming that there  
should be
a closing bracket on line 94?
(the indention is a bit off due to being cut'n'pasted into my mail  
client).

cheers
Erik





> Thanks!
>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 1:35 PM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
>
> On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:
>
>> Have you filled up a ticket about the compacting issue?
>
> That I haven't.
>
>> Have you posted the code for your protocol parser? Here's mine, maybe
>> you can give me some tips! :.)
>
> I haven't done that either. We're looking at using grizzly inhouse
> and it was in that context I wrote
> the protocol parser. I had thoughts about writing a tutorial/blog
> post about it but there aren't all that much time right now.
> At work I have more pressing concerns. I
>
> I attach my protocol parser to this mail so you can look at it. It's
> not tested with 1.7.2, though.
>
> Basically it uses a class XFlowMessage that contains the parser to
> parse the byte buffer into xflow messages.
>
>> I will add your buffer compacting tip!
>
> I'll look at your code and see if I can come up with something. Just
> the thing the wife wants me to do on a friday evening :-)
>
> cheers
>
> /Erik
>
>
>
>>
>> Simon
>>
>> -----Original Message-----
>> From: Erik Svensson [mailto:[hidden email]]
>> Sent: March-07-08 10:56 AM
>> To: [hidden email]
>> Subject: Re: [Q] Testing ProtocolFilter
>>
>> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
>> wrote:
>>
>>> I would like to know, how can I test my protocol filter. I would  
>>> like
>> to
>>> simulate, using unit tests, a truncated message.
>>>
>>>
>>>
>>> I try invoking the connectorHandler's send() method twice (part1 and
>>> part2 of my message) with a Thread.sleep in between but the protocol
>>> parsers doesn't think tokens are missing, it sees the two different
>>> operations as two different message. Does this has something with
>> TCP/IP
>>> where the stack is configured to "know" the packet received has been
>>> truncated or do I need to investigate my ProtocolFilter further. How
>>> should I test this?
>>>
>>>
>>>
>>>             client.connect(new
>>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>>
>>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>>> String("\r\nPA").getBytes());
>>>
>>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>>> String("SS\r\n\0").getBytes());
>>>
>>>             client.send(outputByteBuffer1);
>>>
>>>             Thread.sleep(25);
>>>
>>>             client.send(outputByteBuffer2);
>>>
>>
>> I haven't looked at your previously posted code but I've written a
>> ProtocolParser. tcp knows nothing about messages nor does it know
>> anything
>> about the semantics of your message.
>> Your ProtocolParser needs to be able to determine if it has  
>> received a
>> complete message and then return true when the 'I need more data'-
>> method
>> is
>> called.
>>
>> I also found that I had to do a compact() on my incoming bytebuffer
>> when
>> the
>> ReleaseBuffer() is called.
>> Otherwise, if you can't read a complete message the bytebuffer will
>> just
>> fill up until there's no more space and you don't get any more
>> messages.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>> <MyProtocolParser.java>----------------------------------------------

>> -
>
>> ----------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [Q] Testing ProtocolFilter

Erik Svensson-3
To keep up the flowing commentary... :-)

This is turning out to be something of a flow-of-consciousness thing!

On lines 65 - 70:

  65         ByteBuffer roDuplicateBuffer =  
byteBuffer.asReadOnlyBuffer();
  66
  67         if (byteBuffer.position() == 0){
  68             isExpectingMoreData = true;
  69             return false;
  70         }

I'm wondering a bit about this. If the incoming buffers position is  
'0' (before flip()) means that the buffer is empty
(ie the eq to hasRemaining() == false). 'isExpectingMoreData' means  
that you need more data to fill out a complete message.
I cannot see how you can determine that from just looking at the  
position of the byte buffer.
I might be misunderstanding somehting from your program logic.

cheers
/Erik

On Mar 7, 2008, at 20:51 , Simon Trudeau wrote:

> You are most certainly right, a curly brace is missing on line 94.
> Thanks,
>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 2:49 PM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
>>
>
> Simon,
>
> I found something a bit odd but that might be because you've edited
> the code before posting it.
> In the hasNextMessage() method there's this snippet of code:
>
>   86 }
>   87         /*
>   88          * Validates that the buffer contains at least 2 bytes
> so we can proceed to input validation
>   89          */
>   90         if(bufferContent.length < 2)
>   91         {
>   92         isExpectingMoreData = true;
>   93         return !isExpectingMoreData;
>   94
>   95         // 1.1)
>   96         if(CR == bufferContent[0])
>   97         {
>   98         if(LF == bufferContent[1])
>   99         {
> 100         // 1.1.1)
> 101
> if(bufferContentString.contains(PASS.toString()))
> 102     {
> 103     if(LOG.isTraceEnabled())
> 104     {
> 105     LOG.trace("Found:
> PASS");
> 106     }
> 107     isExpectingMoreData = false;
> 108                     message = PASS.toString();
> 109                     position = PASS.toBytes().length;
> 110     }
>
> Note on line 93 the return without any closing curly bracket.
> However, this shouldn't compile
> as the code below is unreachable. Am I correct in assuming that there
> should be
> a closing bracket on line 94?
> (the indention is a bit off due to being cut'n'pasted into my mail
> client).
>
> cheers
> Erik
>
>
>
>
>
>> Thanks!
>>
>> Simon
>>
>> -----Original Message-----
>> From: Erik Svensson [mailto:[hidden email]]
>> Sent: March-07-08 1:35 PM
>> To: [hidden email]
>> Subject: Re: [Q] Testing ProtocolFilter
>>
>>
>> On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:
>>
>>> Have you filled up a ticket about the compacting issue?
>>
>> That I haven't.
>>
>>> Have you posted the code for your protocol parser? Here's mine,  
>>> maybe
>>> you can give me some tips! :.)
>>
>> I haven't done that either. We're looking at using grizzly inhouse
>> and it was in that context I wrote
>> the protocol parser. I had thoughts about writing a tutorial/blog
>> post about it but there aren't all that much time right now.
>> At work I have more pressing concerns. I
>>
>> I attach my protocol parser to this mail so you can look at it. It's
>> not tested with 1.7.2, though.
>>
>> Basically it uses a class XFlowMessage that contains the parser to
>> parse the byte buffer into xflow messages.
>>
>>> I will add your buffer compacting tip!
>>
>> I'll look at your code and see if I can come up with something. Just
>> the thing the wife wants me to do on a friday evening :-)
>>
>> cheers
>>
>> /Erik
>>
>>
>>
>>>
>>> Simon
>>>
>>> -----Original Message-----
>>> From: Erik Svensson [mailto:[hidden email]]
>>> Sent: March-07-08 10:56 AM
>>> To: [hidden email]
>>> Subject: Re: [Q] Testing ProtocolFilter
>>>
>>> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
>>> wrote:
>>>
>>>> I would like to know, how can I test my protocol filter. I would
>>>> like
>>> to
>>>> simulate, using unit tests, a truncated message.
>>>>
>>>>
>>>>
>>>> I try invoking the connectorHandler's send() method twice (part1  
>>>> and
>>>> part2 of my message) with a Thread.sleep in between but the  
>>>> protocol
>>>> parsers doesn't think tokens are missing, it sees the two different
>>>> operations as two different message. Does this has something with
>>> TCP/IP
>>>> where the stack is configured to "know" the packet received has  
>>>> been
>>>> truncated or do I need to investigate my ProtocolFilter further.  
>>>> How
>>>> should I test this?
>>>>
>>>>
>>>>
>>>>             client.connect(new
>>>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>>>
>>>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>>>> String("\r\nPA").getBytes());
>>>>
>>>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>>>> String("SS\r\n\0").getBytes());
>>>>
>>>>             client.send(outputByteBuffer1);
>>>>
>>>>             Thread.sleep(25);
>>>>
>>>>             client.send(outputByteBuffer2);
>>>>
>>>
>>> I haven't looked at your previously posted code but I've written a
>>> ProtocolParser. tcp knows nothing about messages nor does it know
>>> anything
>>> about the semantics of your message.
>>> Your ProtocolParser needs to be able to determine if it has
>>> received a
>>> complete message and then return true when the 'I need more data'-
>>> method
>>> is
>>> called.
>>>
>>> I also found that I had to do a compact() on my incoming bytebuffer
>>> when
>>> the
>>> ReleaseBuffer() is called.
>>> Otherwise, if you can't read a complete message the bytebuffer will
>>> just
>>> fill up until there's no more space and you don't get any more
>>> messages.
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>> <MyProtocolParser.java>---------------------------------------------
>>> -
>
>>> -
>>
>>> ----------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Re: [Q] Testing ProtocolFilter

CigarMan
Your logic makes sense. Those lines were an almost cut and paste from
ProtocolParserTest.java line 130-133. I will remove those lines.

By the way, when I parse a complete message, the releaseBuffer() method
gets called twice... which is a bit weird. I am having a bit of trouble
tracing this since it is multithreaded... Have you ever encountered
something like this?

Thanks,

Simon

-----Original Message-----
From: Erik Svensson [mailto:[hidden email]]
Sent: March-07-08 3:11 PM
To: [hidden email]
Subject: Re: [Q] Testing ProtocolFilter

To keep up the flowing commentary... :-)

This is turning out to be something of a flow-of-consciousness thing!

On lines 65 - 70:

  65         ByteBuffer roDuplicateBuffer =  
byteBuffer.asReadOnlyBuffer();
  66
  67         if (byteBuffer.position() == 0){
  68             isExpectingMoreData = true;
  69             return false;
  70         }

I'm wondering a bit about this. If the incoming buffers position is  
'0' (before flip()) means that the buffer is empty
(ie the eq to hasRemaining() == false). 'isExpectingMoreData' means  
that you need more data to fill out a complete message.
I cannot see how you can determine that from just looking at the  
position of the byte buffer.
I might be misunderstanding somehting from your program logic.

cheers
/Erik

On Mar 7, 2008, at 20:51 , Simon Trudeau wrote:

> You are most certainly right, a curly brace is missing on line 94.
> Thanks,
>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 2:49 PM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
>>
>
> Simon,
>
> I found something a bit odd but that might be because you've edited
> the code before posting it.
> In the hasNextMessage() method there's this snippet of code:
>
>   86 }
>   87         /*
>   88          * Validates that the buffer contains at least 2 bytes
> so we can proceed to input validation
>   89          */
>   90         if(bufferContent.length < 2)
>   91         {
>   92         isExpectingMoreData = true;
>   93         return !isExpectingMoreData;
>   94
>   95         // 1.1)
>   96         if(CR == bufferContent[0])
>   97         {
>   98         if(LF == bufferContent[1])
>   99         {
> 100         // 1.1.1)
> 101
> if(bufferContentString.contains(PASS.toString()))
> 102     {
> 103     if(LOG.isTraceEnabled())
> 104     {
> 105     LOG.trace("Found:
> PASS");
> 106     }
> 107     isExpectingMoreData = false;
> 108                     message = PASS.toString();
> 109                     position = PASS.toBytes().length;
> 110     }
>
> Note on line 93 the return without any closing curly bracket.
> However, this shouldn't compile
> as the code below is unreachable. Am I correct in assuming that there
> should be
> a closing bracket on line 94?
> (the indention is a bit off due to being cut'n'pasted into my mail
> client).
>
> cheers
> Erik
>
>
>
>
>
>> Thanks!
>>
>> Simon
>>
>> -----Original Message-----
>> From: Erik Svensson [mailto:[hidden email]]
>> Sent: March-07-08 1:35 PM
>> To: [hidden email]
>> Subject: Re: [Q] Testing ProtocolFilter
>>
>>
>> On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:
>>
>>> Have you filled up a ticket about the compacting issue?
>>
>> That I haven't.
>>
>>> Have you posted the code for your protocol parser? Here's mine,  
>>> maybe
>>> you can give me some tips! :.)
>>
>> I haven't done that either. We're looking at using grizzly inhouse
>> and it was in that context I wrote
>> the protocol parser. I had thoughts about writing a tutorial/blog
>> post about it but there aren't all that much time right now.
>> At work I have more pressing concerns. I
>>
>> I attach my protocol parser to this mail so you can look at it. It's
>> not tested with 1.7.2, though.
>>
>> Basically it uses a class XFlowMessage that contains the parser to
>> parse the byte buffer into xflow messages.
>>
>>> I will add your buffer compacting tip!
>>
>> I'll look at your code and see if I can come up with something. Just
>> the thing the wife wants me to do on a friday evening :-)
>>
>> cheers
>>
>> /Erik
>>
>>
>>
>>>
>>> Simon
>>>
>>> -----Original Message-----
>>> From: Erik Svensson [mailto:[hidden email]]
>>> Sent: March-07-08 10:56 AM
>>> To: [hidden email]
>>> Subject: Re: [Q] Testing ProtocolFilter
>>>
>>> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
>>> wrote:
>>>
>>>> I would like to know, how can I test my protocol filter. I would
>>>> like
>>> to
>>>> simulate, using unit tests, a truncated message.
>>>>
>>>>
>>>>
>>>> I try invoking the connectorHandler's send() method twice (part1  
>>>> and
>>>> part2 of my message) with a Thread.sleep in between but the  
>>>> protocol
>>>> parsers doesn't think tokens are missing, it sees the two different
>>>> operations as two different message. Does this has something with
>>> TCP/IP
>>>> where the stack is configured to "know" the packet received has  
>>>> been
>>>> truncated or do I need to investigate my ProtocolFilter further.  
>>>> How
>>>> should I test this?
>>>>
>>>>
>>>>
>>>>             client.connect(new
>>>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>>>
>>>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>>>> String("\r\nPA").getBytes());
>>>>
>>>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>>>> String("SS\r\n\0").getBytes());
>>>>
>>>>             client.send(outputByteBuffer1);
>>>>
>>>>             Thread.sleep(25);
>>>>
>>>>             client.send(outputByteBuffer2);
>>>>
>>>
>>> I haven't looked at your previously posted code but I've written a
>>> ProtocolParser. tcp knows nothing about messages nor does it know
>>> anything
>>> about the semantics of your message.
>>> Your ProtocolParser needs to be able to determine if it has
>>> received a
>>> complete message and then return true when the 'I need more data'-
>>> method
>>> is
>>> called.
>>>
>>> I also found that I had to do a compact() on my incoming bytebuffer
>>> when
>>> the
>>> ReleaseBuffer() is called.
>>> Otherwise, if you can't read a complete message the bytebuffer will
>>> just
>>> fill up until there's no more space and you don't get any more
>>> messages.
>>>
>>>
>>> --------------------------------------------------------------------

>>> -
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>> <MyProtocolParser.java>---------------------------------------------

>>> -
>
>>> -
>>
>>> ----------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [Q] Testing ProtocolFilter

Erik Svensson-3
And talking of releaseBuffer():

I notice something here.

157     public boolean releaseBuffer() {
162     if(!hasMoreBytesToParse())
163         {
168         this.byteBuffer.clear();
169         this.position = 0;
170         this.byteBufferLimit = 0;
171         }
172         else
173         {
174         byteBuffer.position(position);
175         }
176    
177         return hasMoreBytesToParse();
178     }

(I've snipped the LOG lines for brevity)

hasMoreBytesToParse() means 'there are bytes in this buffer that I  
haven't looked at and i don't know if it contains part of a message,  
one complete message or  more than one complete message'. At this  
point this should not be true. Here you should return 'needs more  
data' (if the buffer still contains bytes that  can't be parsed to a  
complete message).

Here's what I think  is happening:

hasNextMessage() is called and we end up here:

111     else
112     {
117     isExpectingMoreData = true;
118     position = bufferContentString.getBytes().length;
119     }      
120         }

saving the position == byteBuffer.limit() and setting  
isExpectingMoredata == true and returning false.Then releaseBuffer is  
called. There we start (see at the top) by calling hasMorebytes():

  20     public boolean hasMoreBytesToParse() {
  21     boolean hasMoreBytesToParse = position < byteBufferLimit;
  28         return hasMoreBytesToParse;
  29     }

Note line 21!  We check that the position is less than the limit().  
Since we've set position to limit this returns false.
which leads to that we inadvertently clear the buffer when we shouldn't.
if you change hasMoreBytesToParse to isExpectingMoreData() I think  
this might work more in the lines of how you were thinking.

hope this helps

and for your question: I can't remember :-) But it shouldn't be  
called more than once since if you do compact twice in the same  
buffer the buffer will be overwritten in subsquent puts.

cheers

/Erik


On Mar 7, 2008, at 21:35 , Simon Trudeau wrote:

> Your logic makes sense. Those lines were an almost cut and paste from
> ProtocolParserTest.java line 130-133. I will remove those lines.
>
> By the way, when I parse a complete message, the releaseBuffer()  
> method
> gets called twice... which is a bit weird. I am having a bit of  
> trouble
> tracing this since it is multithreaded... Have you ever encountered
> something like this?
>
> Thanks,
>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 3:11 PM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
> To keep up the flowing commentary... :-)
>
> This is turning out to be something of a flow-of-consciousness thing!
>
> On lines 65 - 70:
>
>   65         ByteBuffer roDuplicateBuffer =
> byteBuffer.asReadOnlyBuffer();
>   66
>   67         if (byteBuffer.position() == 0){
>   68             isExpectingMoreData = true;
>   69             return false;
>   70         }
>
> I'm wondering a bit about this. If the incoming buffers position is
> '0' (before flip()) means that the buffer is empty
> (ie the eq to hasRemaining() == false). 'isExpectingMoreData' means
> that you need more data to fill out a complete message.
> I cannot see how you can determine that from just looking at the
> position of the byte buffer.
> I might be misunderstanding somehting from your program logic.
>
> cheers
> /Erik
>
> On Mar 7, 2008, at 20:51 , Simon Trudeau wrote:
>
>> You are most certainly right, a curly brace is missing on line 94.
>> Thanks,
>>
>> Simon
>>
>> -----Original Message-----
>> From: Erik Svensson [mailto:[hidden email]]
>> Sent: March-07-08 2:49 PM
>> To: [hidden email]
>> Subject: Re: [Q] Testing ProtocolFilter
>>
>>>
>>
>> Simon,
>>
>> I found something a bit odd but that might be because you've edited
>> the code before posting it.
>> In the hasNextMessage() method there's this snippet of code:
>>
>>   86 }
>>   87         /*
>>   88          * Validates that the buffer contains at least 2 bytes
>> so we can proceed to input validation
>>   89          */
>>   90         if(bufferContent.length < 2)
>>   91         {
>>   92         isExpectingMoreData = true;
>>   93         return !isExpectingMoreData;
>>   94
>>   95         // 1.1)
>>   96         if(CR == bufferContent[0])
>>   97         {
>>   98         if(LF == bufferContent[1])
>>   99         {
>> 100         // 1.1.1)
>> 101
>> if(bufferContentString.contains(PASS.toString()))
>> 102     {
>> 103     if(LOG.isTraceEnabled())
>> 104     {
>> 105     LOG.trace("Found:
>> PASS");
>> 106     }
>> 107     isExpectingMoreData = false;
>> 108                     message = PASS.toString();
>> 109                     position = PASS.toBytes().length;
>> 110     }
>>
>> Note on line 93 the return without any closing curly bracket.
>> However, this shouldn't compile
>> as the code below is unreachable. Am I correct in assuming that there
>> should be
>> a closing bracket on line 94?
>> (the indention is a bit off due to being cut'n'pasted into my mail
>> client).
>>
>> cheers
>> Erik
>>
>>
>>
>>
>>
>>> Thanks!
>>>
>>> Simon
>>>
>>> -----Original Message-----
>>> From: Erik Svensson [mailto:[hidden email]]
>>> Sent: March-07-08 1:35 PM
>>> To: [hidden email]
>>> Subject: Re: [Q] Testing ProtocolFilter
>>>
>>>
>>> On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:
>>>
>>>> Have you filled up a ticket about the compacting issue?
>>>
>>> That I haven't.
>>>
>>>> Have you posted the code for your protocol parser? Here's mine,
>>>> maybe
>>>> you can give me some tips! :.)
>>>
>>> I haven't done that either. We're looking at using grizzly inhouse
>>> and it was in that context I wrote
>>> the protocol parser. I had thoughts about writing a tutorial/blog
>>> post about it but there aren't all that much time right now.
>>> At work I have more pressing concerns. I
>>>
>>> I attach my protocol parser to this mail so you can look at it. It's
>>> not tested with 1.7.2, though.
>>>
>>> Basically it uses a class XFlowMessage that contains the parser to
>>> parse the byte buffer into xflow messages.
>>>
>>>> I will add your buffer compacting tip!
>>>
>>> I'll look at your code and see if I can come up with something. Just
>>> the thing the wife wants me to do on a friday evening :-)
>>>
>>> cheers
>>>
>>> /Erik
>>>
>>>
>>>
>>>>
>>>> Simon
>>>>
>>>> -----Original Message-----
>>>> From: Erik Svensson [mailto:[hidden email]]
>>>> Sent: March-07-08 10:56 AM
>>>> To: [hidden email]
>>>> Subject: Re: [Q] Testing ProtocolFilter
>>>>
>>>> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
>>>> wrote:
>>>>
>>>>> I would like to know, how can I test my protocol filter. I would
>>>>> like
>>>> to
>>>>> simulate, using unit tests, a truncated message.
>>>>>
>>>>>
>>>>>
>>>>> I try invoking the connectorHandler's send() method twice (part1
>>>>> and
>>>>> part2 of my message) with a Thread.sleep in between but the
>>>>> protocol
>>>>> parsers doesn't think tokens are missing, it sees the two  
>>>>> different
>>>>> operations as two different message. Does this has something with
>>>> TCP/IP
>>>>> where the stack is configured to "know" the packet received has
>>>>> been
>>>>> truncated or do I need to investigate my ProtocolFilter further.
>>>>> How
>>>>> should I test this?
>>>>>
>>>>>
>>>>>
>>>>>             client.connect(new
>>>>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>>>>
>>>>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>>>>> String("\r\nPA").getBytes());
>>>>>
>>>>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>>>>> String("SS\r\n\0").getBytes());
>>>>>
>>>>>             client.send(outputByteBuffer1);
>>>>>
>>>>>             Thread.sleep(25);
>>>>>
>>>>>             client.send(outputByteBuffer2);
>>>>>
>>>>
>>>> I haven't looked at your previously posted code but I've written a
>>>> ProtocolParser. tcp knows nothing about messages nor does it know
>>>> anything
>>>> about the semantics of your message.
>>>> Your ProtocolParser needs to be able to determine if it has
>>>> received a
>>>> complete message and then return true when the 'I need more data'-
>>>> method
>>>> is
>>>> called.
>>>>
>>>> I also found that I had to do a compact() on my incoming bytebuffer
>>>> when
>>>> the
>>>> ReleaseBuffer() is called.
>>>> Otherwise, if you can't read a complete message the bytebuffer will
>>>> just
>>>> fill up until there's no more space and you don't get any more
>>>> messages.
>>>>
>>>>
>>>> -------------------------------------------------------------------
>>>> -
>
>>>> -
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>> <MyProtocolParser.java>--------------------------------------------
>>>> -
>
>>>> -
>>
>>>> -
>>>
>>>> ----------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Re: [Q] Testing ProtocolFilter

CigarMan
Thanks for the info, I will look into this. I am still having trouble
testing this properly.


Simon

-----Original Message-----
From: Erik Svensson [mailto:[hidden email]]
Sent: March-07-08 4:09 PM
To: [hidden email]
Subject: Re: [Q] Testing ProtocolFilter

And talking of releaseBuffer():

I notice something here.

157     public boolean releaseBuffer() {
162     if(!hasMoreBytesToParse())
163         {
168         this.byteBuffer.clear();
169         this.position = 0;
170         this.byteBufferLimit = 0;
171         }
172         else
173         {
174         byteBuffer.position(position);
175         }
176    
177         return hasMoreBytesToParse();
178     }

(I've snipped the LOG lines for brevity)

hasMoreBytesToParse() means 'there are bytes in this buffer that I  
haven't looked at and i don't know if it contains part of a message,  
one complete message or  more than one complete message'. At this  
point this should not be true. Here you should return 'needs more  
data' (if the buffer still contains bytes that  can't be parsed to a  
complete message).

Here's what I think  is happening:

hasNextMessage() is called and we end up here:

111     else
112     {
117     isExpectingMoreData = true;
118     position =
bufferContentString.getBytes().length;
119     }      
120         }

saving the position == byteBuffer.limit() and setting  
isExpectingMoredata == true and returning false.Then releaseBuffer is  
called. There we start (see at the top) by calling hasMorebytes():

  20     public boolean hasMoreBytesToParse() {
  21     boolean hasMoreBytesToParse = position <
byteBufferLimit;
  28         return hasMoreBytesToParse;
  29     }

Note line 21!  We check that the position is less than the limit().  
Since we've set position to limit this returns false.
which leads to that we inadvertently clear the buffer when we shouldn't.
if you change hasMoreBytesToParse to isExpectingMoreData() I think  
this might work more in the lines of how you were thinking.

hope this helps

and for your question: I can't remember :-) But it shouldn't be  
called more than once since if you do compact twice in the same  
buffer the buffer will be overwritten in subsquent puts.

cheers

/Erik


On Mar 7, 2008, at 21:35 , Simon Trudeau wrote:

> Your logic makes sense. Those lines were an almost cut and paste from
> ProtocolParserTest.java line 130-133. I will remove those lines.
>
> By the way, when I parse a complete message, the releaseBuffer()  
> method
> gets called twice... which is a bit weird. I am having a bit of  
> trouble
> tracing this since it is multithreaded... Have you ever encountered
> something like this?
>
> Thanks,
>
> Simon
>
> -----Original Message-----
> From: Erik Svensson [mailto:[hidden email]]
> Sent: March-07-08 3:11 PM
> To: [hidden email]
> Subject: Re: [Q] Testing ProtocolFilter
>
> To keep up the flowing commentary... :-)
>
> This is turning out to be something of a flow-of-consciousness thing!
>
> On lines 65 - 70:
>
>   65         ByteBuffer roDuplicateBuffer =
> byteBuffer.asReadOnlyBuffer();
>   66
>   67         if (byteBuffer.position() == 0){
>   68             isExpectingMoreData = true;
>   69             return false;
>   70         }
>
> I'm wondering a bit about this. If the incoming buffers position is
> '0' (before flip()) means that the buffer is empty
> (ie the eq to hasRemaining() == false). 'isExpectingMoreData' means
> that you need more data to fill out a complete message.
> I cannot see how you can determine that from just looking at the
> position of the byte buffer.
> I might be misunderstanding somehting from your program logic.
>
> cheers
> /Erik
>
> On Mar 7, 2008, at 20:51 , Simon Trudeau wrote:
>
>> You are most certainly right, a curly brace is missing on line 94.
>> Thanks,
>>
>> Simon
>>
>> -----Original Message-----
>> From: Erik Svensson [mailto:[hidden email]]
>> Sent: March-07-08 2:49 PM
>> To: [hidden email]
>> Subject: Re: [Q] Testing ProtocolFilter
>>
>>>
>>
>> Simon,
>>
>> I found something a bit odd but that might be because you've edited
>> the code before posting it.
>> In the hasNextMessage() method there's this snippet of code:
>>
>>   86 }
>>   87         /*
>>   88          * Validates that the buffer contains at least 2 bytes
>> so we can proceed to input validation
>>   89          */
>>   90         if(bufferContent.length < 2)
>>   91         {
>>   92         isExpectingMoreData = true;
>>   93         return !isExpectingMoreData;
>>   94
>>   95         // 1.1)
>>   96         if(CR == bufferContent[0])
>>   97         {
>>   98         if(LF == bufferContent[1])
>>   99         {
>> 100         // 1.1.1)
>> 101
>> if(bufferContentString.contains(PASS.toString()))
>> 102     {
>> 103     if(LOG.isTraceEnabled())
>> 104     {
>> 105     LOG.trace("Found:
>> PASS");
>> 106     }
>> 107     isExpectingMoreData = false;
>> 108                     message = PASS.toString();
>> 109                     position = PASS.toBytes().length;
>> 110     }
>>
>> Note on line 93 the return without any closing curly bracket.
>> However, this shouldn't compile
>> as the code below is unreachable. Am I correct in assuming that there
>> should be
>> a closing bracket on line 94?
>> (the indention is a bit off due to being cut'n'pasted into my mail
>> client).
>>
>> cheers
>> Erik
>>
>>
>>
>>
>>
>>> Thanks!
>>>
>>> Simon
>>>
>>> -----Original Message-----
>>> From: Erik Svensson [mailto:[hidden email]]
>>> Sent: March-07-08 1:35 PM
>>> To: [hidden email]
>>> Subject: Re: [Q] Testing ProtocolFilter
>>>
>>>
>>> On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:
>>>
>>>> Have you filled up a ticket about the compacting issue?
>>>
>>> That I haven't.
>>>
>>>> Have you posted the code for your protocol parser? Here's mine,
>>>> maybe
>>>> you can give me some tips! :.)
>>>
>>> I haven't done that either. We're looking at using grizzly inhouse
>>> and it was in that context I wrote
>>> the protocol parser. I had thoughts about writing a tutorial/blog
>>> post about it but there aren't all that much time right now.
>>> At work I have more pressing concerns. I
>>>
>>> I attach my protocol parser to this mail so you can look at it. It's
>>> not tested with 1.7.2, though.
>>>
>>> Basically it uses a class XFlowMessage that contains the parser to
>>> parse the byte buffer into xflow messages.
>>>
>>>> I will add your buffer compacting tip!
>>>
>>> I'll look at your code and see if I can come up with something. Just
>>> the thing the wife wants me to do on a friday evening :-)
>>>
>>> cheers
>>>
>>> /Erik
>>>
>>>
>>>
>>>>
>>>> Simon
>>>>
>>>> -----Original Message-----
>>>> From: Erik Svensson [mailto:[hidden email]]
>>>> Sent: March-07-08 10:56 AM
>>>> To: [hidden email]
>>>> Subject: Re: [Q] Testing ProtocolFilter
>>>>
>>>> On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
>>>> wrote:
>>>>
>>>>> I would like to know, how can I test my protocol filter. I would
>>>>> like
>>>> to
>>>>> simulate, using unit tests, a truncated message.
>>>>>
>>>>>
>>>>>
>>>>> I try invoking the connectorHandler's send() method twice (part1
>>>>> and
>>>>> part2 of my message) with a Thread.sleep in between but the
>>>>> protocol
>>>>> parsers doesn't think tokens are missing, it sees the two  
>>>>> different
>>>>> operations as two different message. Does this has something with
>>>> TCP/IP
>>>>> where the stack is configured to "know" the packet received has
>>>>> been
>>>>> truncated or do I need to investigate my ProtocolFilter further.
>>>>> How
>>>>> should I test this?
>>>>>
>>>>>
>>>>>
>>>>>             client.connect(new
>>>>> InetSocketAddress(InetAddress.getLocalHost(), serverPort));
>>>>>
>>>>>             ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
>>>>> String("\r\nPA").getBytes());
>>>>>
>>>>>             ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
>>>>> String("SS\r\n\0").getBytes());
>>>>>
>>>>>             client.send(outputByteBuffer1);
>>>>>
>>>>>             Thread.sleep(25);
>>>>>
>>>>>             client.send(outputByteBuffer2);
>>>>>
>>>>
>>>> I haven't looked at your previously posted code but I've written a
>>>> ProtocolParser. tcp knows nothing about messages nor does it know
>>>> anything
>>>> about the semantics of your message.
>>>> Your ProtocolParser needs to be able to determine if it has
>>>> received a
>>>> complete message and then return true when the 'I need more data'-
>>>> method
>>>> is
>>>> called.
>>>>
>>>> I also found that I had to do a compact() on my incoming bytebuffer
>>>> when
>>>> the
>>>> ReleaseBuffer() is called.
>>>> Otherwise, if you can't read a complete message the bytebuffer will
>>>> just
>>>> fill up until there's no more space and you don't get any more
>>>> messages.
>>>>
>>>>
>>>> -------------------------------------------------------------------

>>>> -
>
>>>> -
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>> <MyProtocolParser.java>--------------------------------------------

>>>> -
>
>>>> -
>>
>>>> -
>>>
>>>> ----------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>
>>>
>>> --------------------------------------------------------------------

>>> -
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [Q] Testing ProtocolFilter

Erik Svensson-4
Thanks for the info, I will look into this. I am still having trouble
testing this properly.

No guarantees tho' right?! :-)

And now it's time for that friday drink!

Great weekend on ya'll!

/Erik



Simon

-----Original Message-----
From: Erik Svensson [[hidden email]] 
Sent: March-07-08 4:09 PM
Subject: Re: [Q] Testing ProtocolFilter

And talking of releaseBuffer():

I notice something here.

157     public boolean releaseBuffer() {
162     if(!hasMoreBytesToParse())
163         {
168         this.byteBuffer.clear();
169         this.position = 0;
170         this.byteBufferLimit = 0;
171         }
172         else
173         {
174         byteBuffer.position(position);
175         }
176     
177         return hasMoreBytesToParse();
178     }

(I've snipped the LOG lines for brevity)

hasMoreBytesToParse() means 'there are bytes in this buffer that I  
haven't looked at and i don't know if it contains part of a message,   
one complete message or  more than one complete message'. At this  
point this should not be true. Here you should return 'needs more  
data' (if the buffer still contains bytes that  can't be parsed to a  
complete message).

Here's what I think  is happening:

hasNextMessage() is called and we end up here:

111     else
112     {
117     isExpectingMoreData = true;
118     position =
bufferContentString.getBytes().length;
119     }      
120         }

saving the position == byteBuffer.limit() and setting  
isExpectingMoredata == true and returning false.Then releaseBuffer is  
called. There we start (see at the top) by calling hasMorebytes():

  20     public boolean hasMoreBytesToParse() {
  21     boolean hasMoreBytesToParse = position <
byteBufferLimit;
  28         return hasMoreBytesToParse;
  29     }

Note line 21!  We check that the position is less than the limit().  
Since we've set position to limit this returns false.
which leads to that we inadvertently clear the buffer when we shouldn't.
if you change hasMoreBytesToParse to isExpectingMoreData() I think  
this might work more in the lines of how you were thinking.

hope this helps

and for your question: I can't remember :-) But it shouldn't be  
called more than once since if you do compact twice in the same  
buffer the buffer will be overwritten in subsquent puts.

cheers

/Erik


On Mar 7, 2008, at 21:35 , Simon Trudeau wrote:

Your logic makes sense. Those lines were an almost cut and paste from
ProtocolParserTest.java line 130-133. I will remove those lines.

By the way, when I parse a complete message, the releaseBuffer()  
method
gets called twice... which is a bit weird. I am having a bit of  
trouble
tracing this since it is multithreaded... Have you ever encountered
something like this?

Thanks,

Simon

-----Original Message-----
From: Erik Svensson [[hidden email]]
Sent: March-07-08 3:11 PM
Subject: Re: [Q] Testing ProtocolFilter

To keep up the flowing commentary... :-)

This is turning out to be something of a flow-of-consciousness thing!

On lines 65 - 70:

  65         ByteBuffer roDuplicateBuffer =
byteBuffer.asReadOnlyBuffer();
  66
  67         if (byteBuffer.position() == 0){
  68             isExpectingMoreData = true;
  69             return false;
  70         }

I'm wondering a bit about this. If the incoming buffers position is
'0' (before flip()) means that the buffer is empty
(ie the eq to hasRemaining() == false). 'isExpectingMoreData' means
that you need more data to fill out a complete message.
I cannot see how you can determine that from just looking at the
position of the byte buffer.
I might be misunderstanding somehting from your program logic.

cheers
/Erik

On Mar 7, 2008, at 20:51 , Simon Trudeau wrote:

You are most certainly right, a curly brace is missing on line 94.
Thanks,

Simon

-----Original Message-----
From: Erik Svensson [[hidden email]]
Sent: March-07-08 2:49 PM
Subject: Re: [Q] Testing ProtocolFilter



Simon,

I found something a bit odd but that might be because you've edited
the code before posting it.
In the hasNextMessage() method there's this snippet of code:

  86 }
  87         /*
  88          * Validates that the buffer contains at least 2 bytes
so we can proceed to input validation
  89          */
  90         if(bufferContent.length < 2)
  91         {
  92         isExpectingMoreData = true;
  93         return !isExpectingMoreData;
  94
  95         // 1.1)
  96         if(CR == bufferContent[0])
  97         {
  98         if(LF == bufferContent[1])
  99         {
100         // 1.1.1)
101
if(bufferContentString.contains(PASS.toString()))
102     {
103     if(LOG.isTraceEnabled())
104     {
105     LOG.trace("Found:
PASS");
106     }
107     isExpectingMoreData = false;
108                     message = PASS.toString();
109                     position = PASS.toBytes().length;
110     }

Note on line 93 the return without any closing curly bracket.
However, this shouldn't compile
as the code below is unreachable. Am I correct in assuming that there
should be
a closing bracket on line 94?
(the indention is a bit off due to being cut'n'pasted into my mail
client).

cheers
Erik





Thanks!

Simon

-----Original Message-----
From: Erik Svensson [[hidden email]]
Sent: March-07-08 1:35 PM
Subject: Re: [Q] Testing ProtocolFilter


On Mar 7, 2008, at 17:03 , Simon Trudeau wrote:

Have you filled up a ticket about the compacting issue?

That I haven't.

Have you posted the code for your protocol parser? Here's mine,
maybe
you can give me some tips! :.)

I haven't done that either. We're looking at using grizzly inhouse
and it was in that context I wrote
the protocol parser. I had thoughts about writing a tutorial/blog
post about it but there aren't all that much time right now.
At work I have more pressing concerns. I

I attach my protocol parser to this mail so you can look at it. It's
not tested with 1.7.2, though.

Basically it uses a class XFlowMessage that contains the parser to
parse the byte buffer into xflow messages.

I will add your buffer compacting tip!

I'll look at your code and see if I can come up with something. Just
the thing the wife wants me to do on a friday evening :-)

cheers

/Erik




Simon

-----Original Message-----
From: Erik Svensson [[hidden email]]
Sent: March-07-08 10:56 AM
Subject: Re: [Q] Testing ProtocolFilter

On 3/7/08 4:40 PM, "Simon Trudeau" <[hidden email]>
wrote:

I would like to know, how can I test my protocol filter. I would
like
to
simulate, using unit tests, a truncated message.



I try invoking the connectorHandler's send() method twice (part1
and
part2 of my message) with a Thread.sleep in between but the
protocol
parsers doesn't think tokens are missing, it sees the two  
different
operations as two different message. Does this has something with
TCP/IP
where the stack is configured to "know" the packet received has
been
truncated or do I need to investigate my ProtocolFilter further.
How
should I test this?



            client.connect(new
InetSocketAddress(InetAddress.getLocalHost(), serverPort));

            ByteBuffer outputByteBuffer1 = ByteBuffer.wrap(new
String("\r\nPA").getBytes());

            ByteBuffer outputByteBuffer2 = ByteBuffer.wrap(new
String("SS\r\n\0").getBytes());

            client.send(outputByteBuffer1);

            Thread.sleep(25);

            client.send(outputByteBuffer2);


I haven't looked at your previously posted code but I've written a
ProtocolParser. tcp knows nothing about messages nor does it know
anything
about the semantics of your message.
Your ProtocolParser needs to be able to determine if it has
received a
complete message and then return true when the 'I need more data'-
method
is
called.

I also found that I had to do a compact() on my incoming bytebuffer
when
the
ReleaseBuffer() is called.
Otherwise, if you can't read a complete message the bytebuffer will
just
fill up until there's no more space and you don't get any more
messages.


-------------------------------------------------------------------

-

-
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

<MyProtocolParser.java>--------------------------------------------

-

-

-

----------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


--------------------------------------------------------------------

-
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


'Grammar am for people who can't think for myself, understanded me?'
-- Bukcy Katt on the english language