Eliminating buffer copy in upload with NIOInputStream

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Eliminating buffer copy in upload with NIOInputStream

jbrisbin
I'm trying to eliminate a buffer copy in my file upload handling code. I'm currently doing:

                        def len = inputStream.readyData()
                        def b = new byte[len]
                        inputStream.read(b)
                        channel.write(ByteBuffer.wrap(b), channel.size(), b, handler)

using Java 7's AsynchronousFileChannel. I'd like to be able to shove a ByteBuffer directly into the channel.write() with no copying or anything.

I couldn't find a way to do this that would not incur extra steps nor involve additional allocations of buffers. Is there some low-level slicing I can do or something to eliminate this?


Thanks!

Jon Brisbin
http//jbrisbin.com


Reply | Threaded
Open this post in threaded view
|

Re: Eliminating buffer copy in upload with NIOInputStream

oleksiys
Administrator
Hi Jon,


                        def len = inputStream.readyData()
                        def b = new byte[len]
                        inputStream.read(b)
                        channel.write(ByteBuffer.wrap(b), channel.size(), b, handler)
This is one "byte[] copy" worse than getBuffer() approach in the sample I referred in prev. emails.
Wait a bit, currently I'm working on sample, which is going to use direct buffers, which avoids any buffer copy.

WBR,
Alexey.


using Java 7's AsynchronousFileChannel. I'd like to be able to shove a ByteBuffer directly into the channel.write() with no copying or anything.

I couldn't find a way to do this that would not incur extra steps nor involve additional allocations of buffers. Is there some low-level slicing I can do or something to eliminate this?


Thanks!

Jon Brisbin
http//jbrisbin.com



Reply | Threaded
Open this post in threaded view
|

Re: Eliminating buffer copy in upload with NIOInputStream

jbrisbin
That would be awesome, thanks.

FWIW- I tried doing that myself but the MD5 sum of the finished file didn't match that of the one I uploaded. I suppose the data had changed behind the buffer before the FileChannel had a chance to write the data out.

I'm getting ready to push some code to my GitHub that shows a Groovy/Grizzly application that performs equally to a Node.js application.

Thanks!

Jon Brisbin
http//jbrisbin.com



From: "Oleksiy Stashok" <[hidden email]>
To: [hidden email]
Sent: Tuesday, July 26, 2011 11:26:20 AM
Subject: Re: Eliminating buffer copy in upload with NIOInputStream

Hi Jon,


                        def len = inputStream.readyData()
                        def b = new byte[len]
                        inputStream.read(b)
                        channel.write(ByteBuffer.wrap(b), channel.size(), b, handler)
This is one "byte[] copy" worse than getBuffer() approach in the sample I referred in prev. emails.
Wait a bit, currently I'm working on sample, which is going to use direct buffers, which avoids any buffer copy.

WBR,
Alexey.


using Java 7's AsynchronousFileChannel. I'd like to be able to shove a ByteBuffer directly into the channel.write() with no copying or anything.

I couldn't find a way to do this that would not incur extra steps nor involve additional allocations of buffers. Is there some low-level slicing I can do or something to eliminate this?


Thanks!

Jon Brisbin
http//jbrisbin.com