exception using ByteBufferManager

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

exception using ByteBufferManager

jbrisbin
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com


Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

oleksiys
Administrator
Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com



Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

jbrisbin
I'll give this a shot.

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com




Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

jbrisbin
In reply to this post by oleksiys
Is there a snapshot repo that I can use from my gradle project? Or will I just need to install the jars locally?


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com




Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

jbrisbin
In reply to this post by oleksiys
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com




Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

oleksiys
Administrator
In reply to this post by jbrisbin
You can clone the git repo, build the project locally and the just refer 2.1.2-SNAPSHOT.
We can publish 2.1.2 release candidate today/tomorrow.

Thanks.

WBR,
Alexey.

On 07/28/2011 04:11 PM, Jon Brisbin wrote:
Is there a snapshot repo that I can use from my gradle project? Or will I just need to install the jars locally?


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com





Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

Ryan Lubke-2
I just pushed a 2.1.2 snapshot to the java.net repository [1].

Please give that a shot.

Thanks,
-rl

[1]
<repository>
 <id>maven2-repository.java.net</id>
 <name>Java.net Repository for Maven</name>
 <url>http://download.java.net/maven/2/</url>
 <layout>default</layout>
</repository>


On 7/28/11 8:12 AM, Oleksiy Stashok wrote:
You can clone the git repo, build the project locally and the just refer 2.1.2-SNAPSHOT.
We can publish 2.1.2 release candidate today/tomorrow.

Thanks.

WBR,
Alexey.

On 07/28/2011 04:11 PM, Jon Brisbin wrote:
Is there a snapshot repo that I can use from my gradle project? Or will I just need to install the jars locally?


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com






Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

oleksiys
Administrator
In reply to this post by jbrisbin
Hi Jon,

with the new code (see Ryan's email),
please use inputStream.readBuffer();

this way you won't need to call InputStream.skip(...) etc, pls. see the sample I sent recently.

Thanks.

Alexey.


On 07/28/2011 05:11 PM, Jon Brisbin wrote:
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com





Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

jbrisbin
Subtle difference that I missed. Sorry.

Looks like it works well. But I am now seeing random errors that I didn't see before:

Exception in thread "Thread-4" java.lang.IllegalArgumentException
    at java.nio.Buffer.position(Buffer.java:236)
    at sun.nio.ch.IOUtil.write(IOUtil.java:63)
    at sun.nio.ch.SimpleAsynchronousFileChannelImpl$3.run(SimpleAsynchronousFileChannelImpl.java:375)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

I can't tell exactly what's causing it but this only started happening after updating to the latest.

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Thursday, July 28, 2011 10:47:37 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

with the new code (see Ryan's email),
please use inputStream.readBuffer();

this way you won't need to call InputStream.skip(...) etc, pls. see the sample I sent recently.

Thanks.

Alexey.


On 07/28/2011 05:11 PM, Jon Brisbin wrote:
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com






Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

jbrisbin
In reply to this post by oleksiys
I think this way is a little faster than what I was doing before. Under load it performs very well and is, more importantly, more consistent in CPU and memory usage than previously.

Thanks for the help!

Is this using a DirectByteBuffer underneath? A traceback I once got from AsynchronousFileChannel made think it wasn't...

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Thursday, July 28, 2011 10:47:37 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

with the new code (see Ryan's email),
please use inputStream.readBuffer();

this way you won't need to call InputStream.skip(...) etc, pls. see the sample I sent recently.

Thanks.

Alexey.


On 07/28/2011 05:11 PM, Jon Brisbin wrote:
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com






Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

oleksiys
Administrator
On 07/28/2011 06:32 PM, Jon Brisbin wrote:
I think this way is a little faster than what I was doing before. Under load it performs very well and is, more importantly, more consistent in CPU and memory usage than previously.

Thanks for the help!

Is this using a DirectByteBuffer underneath? A traceback I once got from AsynchronousFileChannel made think it wasn't...
If you haven't changed the MemoryManager - then it doesn't use direct buffers. You can check the sample I referred recently to see how you can switch to direct buffers.

What about

Exception in thread "Thread-4" java.lang.IllegalArgumentException
    at java.nio.Buffer.position(Buffer.java:236)
    at sun.nio.ch.IOUtil.write(IOUtil.java:63)
    at sun.nio.ch.SimpleAsynchronousFileChannelImpl$3.run(SimpleAsynchronousFileChannelImpl.java:375)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

Is it still the case?

Thanks.

WBR,
Alexey.


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 10:47:37 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

with the new code (see Ryan's email),
please use inputStream.readBuffer();

this way you won't need to call InputStream.skip(...) etc, pls. see the sample I sent recently.

Thanks.

Alexey.


On 07/28/2011 05:11 PM, Jon Brisbin wrote:
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com







Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

jbrisbin
I looked at the example but it wasn't obvious to me how to change the memory manager or where it should be set.

Could you possibly be more specific?

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Monday, August 1, 2011 4:40:03 AM
Subject: Re: exception using ByteBufferManager

On 07/28/2011 06:32 PM, Jon Brisbin wrote:
I think this way is a little faster than what I was doing before. Under load it performs very well and is, more importantly, more consistent in CPU and memory usage than previously.

Thanks for the help!

Is this using a DirectByteBuffer underneath? A traceback I once got from AsynchronousFileChannel made think it wasn't...
If you haven't changed the MemoryManager - then it doesn't use direct buffers. You can check the sample I referred recently to see how you can switch to direct buffers.

What about

Exception in thread "Thread-4" java.lang.IllegalArgumentException
    at java.nio.Buffer.position(Buffer.java:236)
    at sun.nio.ch.IOUtil.write(IOUtil.java:63)
    at sun.nio.ch.SimpleAsynchronousFileChannelImpl$3.run(SimpleAsynchronousFileChannelImpl.java:375)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

Is it still the case?

Thanks.

WBR,
Alexey.


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 10:47:37 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

with the new code (see Ryan's email),
please use inputStream.readBuffer();

this way you won't need to call InputStream.skip(...) etc, pls. see the sample I sent recently.

Thanks.

Alexey.


On 07/28/2011 05:11 PM, Jon Brisbin wrote:
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com








Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

oleksiys
Administrator
uncomment this:

078.
//        final TCPNIOTransport transport = server.getListeners().iterator().next().getTransport();
079. 
080.//        If we want to try direct byte buffers?
081.//        final ByteBufferManager mm = new ByteBufferManager(true, 128 * 1024,
082.//                ByteBufferManager.DEFAULT_SMALL_BUFFER_SIZE);
083.//       
084.//        transport.setMemoryManager(mm);


WBR,
Alexey.

On 08/02/2011 02:18 PM, Jon Brisbin wrote:
I looked at the example but it wasn't obvious to me how to change the memory manager or where it should be set.

Could you possibly be more specific?

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Monday, August 1, 2011 4:40:03 AM
Subject: Re: exception using ByteBufferManager

On 07/28/2011 06:32 PM, Jon Brisbin wrote:
I think this way is a little faster than what I was doing before. Under load it performs very well and is, more importantly, more consistent in CPU and memory usage than previously.

Thanks for the help!

Is this using a DirectByteBuffer underneath? A traceback I once got from AsynchronousFileChannel made think it wasn't...
If you haven't changed the MemoryManager - then it doesn't use direct buffers. You can check the sample I referred recently to see how you can switch to direct buffers.

What about

Exception in thread "Thread-4" java.lang.IllegalArgumentException
    at java.nio.Buffer.position(Buffer.java:236)
    at sun.nio.ch.IOUtil.write(IOUtil.java:63)
    at sun.nio.ch.SimpleAsynchronousFileChannelImpl$3.run(SimpleAsynchronousFileChannelImpl.java:375)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

Is it still the case?

Thanks.

WBR,
Alexey.


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 10:47:37 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

with the new code (see Ryan's email),
please use inputStream.readBuffer();

this way you won't need to call InputStream.skip(...) etc, pls. see the sample I sent recently.

Thanks.

Alexey.


On 07/28/2011 05:11 PM, Jon Brisbin wrote:
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com









Reply | Threaded
Open this post in threaded view
|

Re: exception using ByteBufferManager

jbrisbin
Got it.

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Tuesday, August 2, 2011 7:22:44 AM
Subject: Re: exception using ByteBufferManager

uncomment this:

078.
//        final TCPNIOTransport transport = server.getListeners().iterator().next().getTransport();
079. 
080.//        If we want to try direct byte buffers?
081.//        final ByteBufferManager mm = new ByteBufferManager(true, 128 * 1024,
082.//                ByteBufferManager.DEFAULT_SMALL_BUFFER_SIZE);
083.//       
084.//        transport.setMemoryManager(mm);


WBR,
Alexey.

On 08/02/2011 02:18 PM, Jon Brisbin wrote:
I looked at the example but it wasn't obvious to me how to change the memory manager or where it should be set.

Could you possibly be more specific?

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Monday, August 1, 2011 4:40:03 AM
Subject: Re: exception using ByteBufferManager

On 07/28/2011 06:32 PM, Jon Brisbin wrote:
I think this way is a little faster than what I was doing before. Under load it performs very well and is, more importantly, more consistent in CPU and memory usage than previously.

Thanks for the help!

Is this using a DirectByteBuffer underneath? A traceback I once got from AsynchronousFileChannel made think it wasn't...
If you haven't changed the MemoryManager - then it doesn't use direct buffers. You can check the sample I referred recently to see how you can switch to direct buffers.

What about

Exception in thread "Thread-4" java.lang.IllegalArgumentException
    at java.nio.Buffer.position(Buffer.java:236)
    at sun.nio.ch.IOUtil.write(IOUtil.java:63)
    at sun.nio.ch.SimpleAsynchronousFileChannelImpl$3.run(SimpleAsynchronousFileChannelImpl.java:375)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:722)

Is it still the case?

Thanks.

WBR,
Alexey.


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 10:47:37 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

with the new code (see Ryan's email),
please use inputStream.readBuffer();

this way you won't need to call InputStream.skip(...) etc, pls. see the sample I sent recently.

Thanks.

Alexey.


On 07/28/2011 05:11 PM, Jon Brisbin wrote:
I tried this in my shootout example and got this:

java.lang.IllegalStateException: CompositeBuffer has already been disposed
    at org.glassfish.grizzly.memory.BuffersBuffer.checkDispose(BuffersBuffer.java:1614)
    at org.glassfish.grizzly.memory.BuffersBuffer.remaining(BuffersBuffer.java:348)
    at org.glassfish.grizzly.http.server.io.InputBuffer.available(InputBuffer.java:357)
    at org.glassfish.grizzly.http.server.io.InputBuffer.readyData(InputBuffer.java:349)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:731)
    at org.glassfish.grizzly.http.server.io.InputBuffer.notifyAvailable(InputBuffer.java:691)
    at org.glassfish.grizzly.http.server.NIOInputStreamImpl.notifyAvailable(NIOInputStreamImpl.java:139)
    at org.glassfish.grizzly.http.server.io.NIOInputSource$notifyAvailable.call(Unknown Source)

here's my handler code:

                    def readAvailable() {
                        def len = inputStream.readyData()
                        def inputStreamBuffer = inputStream.getBuffer()
                        def writeBuff = inputStreamBuffer.toByteBuffer()
                        try {
                            channel.write(writeBuff, bytes.get(), writeBuff, handler)
                            bytes.addAndGet(len)
                        } finally {
                            inputStreamBuffer.tryDispose()
                        }
                    }


Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" [hidden email]
To: [hidden email]
Sent: Thursday, July 28, 2011 6:22:00 AM
Subject: Re: exception using ByteBufferManager

Hi Jon,

I've updated the sample [1], so you can try to uncomment couple of lines to try direct buffers.
Only one thing - you need to use latest Grizzly trunk to run it.

Thanks.

WBR,
Alexey.

[1] http://java.net/projects/grizzly/sources/git/content/samples/http-server-samples/src/main/java/org/glassfish/grizzly/samples/httpserver/nonblockinghandler/UploadHttpHandlerSample.java

On 07/27/2011 10:58 PM, Jon Brisbin wrote:
I'm trying to use the ByteBufferManager in my application because I'm hitting some issues with garbage collecting all these byte arrays I'm allocating for writing uploads. Whenever I try and allocate an object from it, though, I get this exception:

java.lang.ClassCastException: java.nio.DirectByteBuffer cannot be cast to org.glassfish.grizzly.memory.HeapBuffer
    at org.glassfish.grizzly.memory.HeapMemoryManager$HeapBufferThreadLocalPool.reset(HeapMemoryManager.java:307)
    at org.glassfish.grizzly.memory.ByteBufferManager.reallocatePoolBuffer(ByteBufferManager.java:371)
    at org.glassfish.grizzly.memory.ByteBufferManager.allocateByteBuffer(ByteBufferManager.java:238)
    at org.glassfish.grizzly.memory.ByteBufferAware$allocateByteBuffer.call(Unknown Source)

Are there example of proper use of this somewhere?

Thanks!

Jon Brisbin
http//jbrisbin.com