DirectByteBufferRecord and HeapMemoryManager?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

DirectByteBufferRecord and HeapMemoryManager?

Daniel Feist
Hi,

I'm deliberatly using HeapMemoryManager with Grizzly (for now) to
avoid any potential memory issues via the use of off-heap memory.

That said, when running some performance tests I'm seeing OutOfMemory
errors due to DirectByteBuffer usage in
org.glassfish.grizzly.nio.transport.TCPNIOUtils.allocateAndReadBuffer.
(the connection returns a read buffer size of 12M, so with some
concurrency this seems becomes a big deal).

I see there is a system property I can use to limit maximum size of
these direct bufffers and thus avoid the OutOfMemoryExceptions, but
I'm wondering why the MemoryManager is explicitlu being bypassed here
rather than simply being used?  This also means there are two
allocations and reads per request and not just one.  Can anyone shed
some light?

thanks,
Dan
Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

oleksiys
Administrator
Hi Daniel,


On 08.12.14 09:32, Daniel Feist wrote:

> I see there is a system property I can use to limit maximum size of
> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
> I'm wondering why the MemoryManager is explicitlu being bypassed here
> rather than simply being used?  This also means there are two
> allocations and reads per request and not just one.  Can anyone shed
> some light?
Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
underneath - allocate (or take pooled) direct ByteBuffer and use it for
reading.
So we basically do the same in our code and passing direct ByteBuffer to
a SocketChannel, so SocketChannel itself will not allocate direct
ByteBuffer.

This approach gives us one advantage - once we read to the direct
ByteBuffer - we know the exact amount of bytes we need to allocate from
the MemoryManager (no guessing).

Hope it will help.

WBR,
Alexey.

PS: Pls. give a shot to PooledMemoryManager, it can work with direct and
heap buffers and it performed well on our tests.


Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

Daniel Feist
Hi,

Sorry, I'm not understanding...  Let's talk about code, it's easier :-)

What I'm wondering is why the following exist:

1) TCPNIOUtils.java Line 230-246.
(https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)

Because if a non-direct memoryManager has been chosen I'm not sure why
that choice needs to be overridden and a direct buffer used anyway as
an intermediate step.

2) DirectByteBufferRecord
(https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)

This is allocating direct buffers, and also caching them per-thread,
yet it's not a MemoryManager implementation, it's something different.
Is this just old/legacy?

Dan

On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
<[hidden email]> wrote:

> Hi Daniel,
>
>
> On 08.12.14 09:32, Daniel Feist wrote:
>
>> I see there is a system property I can use to limit maximum size of
>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>> I'm wondering why the MemoryManager is explicitlu being bypassed here
>> rather than simply being used?  This also means there are two
>> allocations and reads per request and not just one.  Can anyone shed
>> some light?
>
> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
> underneath - allocate (or take pooled) direct ByteBuffer and use it for
> reading.
> So we basically do the same in our code and passing direct ByteBuffer to a
> SocketChannel, so SocketChannel itself will not allocate direct ByteBuffer.
>
> This approach gives us one advantage - once we read to the direct ByteBuffer
> - we know the exact amount of bytes we need to allocate from the
> MemoryManager (no guessing).
>
> Hope it will help.
>
> WBR,
> Alexey.
>
> PS: Pls. give a shot to PooledMemoryManager, it can work with direct and
> heap buffers and it performed well on our tests.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

oleksiys
Administrator
Hi,

On 08.12.14 10:37, Daniel Feist wrote:
> What I'm wondering is why the following exist:
>
> 1) TCPNIOUtils.java Line 230-246.
> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>
> Because if a non-direct memoryManager has been chosen I'm not sure why
> that choice needs to be overridden and a direct buffer used anyway as
> an intermediate step.
Pls. take a look at the JDK code here [1] (line 195)

if the passed ByteBuffer is not direct ByteBuffer - JDK will do the same
"intermediate step" - allocate direct ByteBuffer, use it for reading,
copy data to our heap ByteBuffer.

We could've used that, but in that case we have to guess the read size
and do something like this:

1. memoryManager.allocate(large_chunk);
2. read to the allocated heap ByteBuffer
2.1. JDK allocates direct ByteBuffer of size large_chunk
2.2 read data to the direct ByteBuffer
2.3 copy direct ByteBuffer data to our heap ByteBuffer
3. release unused part of ByteBuffer back to MemoryManager (if any)

Instead of that, we use large enough direct ByteBuffer, read data
directly to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in
that case). After we read to the direct ByteBuffer we know exactly how
many bytes we need to allocate from the MemoryManager.
So we just reshuffled the steps sequence above and have this:

1. allocate direct ByteBuffer of size large_chunk
2. read to the allocated direct ByteBuffer (in this case JDK doesn't do
intermediate allocation step)
3. memoryManager.allocate(read_bytes_count) // we know how many bytes we
read
4. copy direct ByteBuffer to allocated heap ByteBuffer

So, by reshuffling the direct ByteBuffer allocation we're able to
optimize read path.

Makes sense?

Thanks.

WBR,
Alexey.

[1]
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29

>
> 2) DirectByteBufferRecord
> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>
> This is allocating direct buffers, and also caching them per-thread,
> yet it's not a MemoryManager implementation, it's something different.
> Is this just old/legacy?
>
> Dan
>
> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
> <[hidden email]> wrote:
>> Hi Daniel,
>>
>>
>> On 08.12.14 09:32, Daniel Feist wrote:
>>
>>> I see there is a system property I can use to limit maximum size of
>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>> I'm wondering why the MemoryManager is explicitlu being bypassed here
>>> rather than simply being used?  This also means there are two
>>> allocations and reads per request and not just one.  Can anyone shed
>>> some light?
>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
>> underneath - allocate (or take pooled) direct ByteBuffer and use it for
>> reading.
>> So we basically do the same in our code and passing direct ByteBuffer to a
>> SocketChannel, so SocketChannel itself will not allocate direct ByteBuffer.
>>
>> This approach gives us one advantage - once we read to the direct ByteBuffer
>> - we know the exact amount of bytes we need to allocate from the
>> MemoryManager (no guessing).
>>
>> Hope it will help.
>>
>> WBR,
>> Alexey.
>>
>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct and
>> heap buffers and it performed well on our tests.
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

Daniel Feist
Wow, thanks Alexey for the detailed explanation.  This also explains
the existence of DirectByteBufferRecord which reuses DirectBuffer
per-thread rather than creating a new one each time which would happer
otherwise I assume?

A related question, if you have a moment:  On my test enviroment the
connection object returns a recieveBufferSize of 12Mb so if I test
with high concurrency and I'm using the WorkerThreadIOStrategy with a
thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
memory will need to be allocated or am I jumping to conlcusions about
this relationship?

I'll try the PooledMemoryManager for sure, thanks for the tip.

Dan



On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
<[hidden email]> wrote:

> Hi,
>
> On 08.12.14 10:37, Daniel Feist wrote:
>>
>> What I'm wondering is why the following exist:
>>
>> 1) TCPNIOUtils.java Line 230-246.
>>
>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>
>> Because if a non-direct memoryManager has been chosen I'm not sure why
>> that choice needs to be overridden and a direct buffer used anyway as
>> an intermediate step.
>
> Pls. take a look at the JDK code here [1] (line 195)
>
> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the same
> "intermediate step" - allocate direct ByteBuffer, use it for reading, copy
> data to our heap ByteBuffer.
>
> We could've used that, but in that case we have to guess the read size and
> do something like this:
>
> 1. memoryManager.allocate(large_chunk);
> 2. read to the allocated heap ByteBuffer
> 2.1. JDK allocates direct ByteBuffer of size large_chunk
> 2.2 read data to the direct ByteBuffer
> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>
> Instead of that, we use large enough direct ByteBuffer, read data directly
> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that case).
> After we read to the direct ByteBuffer we know exactly how many bytes we
> need to allocate from the MemoryManager.
> So we just reshuffled the steps sequence above and have this:
>
> 1. allocate direct ByteBuffer of size large_chunk
> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't do
> intermediate allocation step)
> 3. memoryManager.allocate(read_bytes_count) // we know how many bytes we
> read
> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>
> So, by reshuffling the direct ByteBuffer allocation we're able to optimize
> read path.
>
> Makes sense?
>
> Thanks.
>
> WBR,
> Alexey.
>
> [1]
> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>
>>
>> 2) DirectByteBufferRecord
>>
>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>
>> This is allocating direct buffers, and also caching them per-thread,
>> yet it's not a MemoryManager implementation, it's something different.
>> Is this just old/legacy?
>>
>> Dan
>>
>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>> <[hidden email]> wrote:
>>>
>>> Hi Daniel,
>>>
>>>
>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>
>>>> I see there is a system property I can use to limit maximum size of
>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>>> I'm wondering why the MemoryManager is explicitlu being bypassed here
>>>> rather than simply being used?  This also means there are two
>>>> allocations and reads per request and not just one.  Can anyone shed
>>>> some light?
>>>
>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
>>> underneath - allocate (or take pooled) direct ByteBuffer and use it for
>>> reading.
>>> So we basically do the same in our code and passing direct ByteBuffer to
>>> a
>>> SocketChannel, so SocketChannel itself will not allocate direct
>>> ByteBuffer.
>>>
>>> This approach gives us one advantage - once we read to the direct
>>> ByteBuffer
>>> - we know the exact amount of bytes we need to allocate from the
>>> MemoryManager (no guessing).
>>>
>>> Hope it will help.
>>>
>>> WBR,
>>> Alexey.
>>>
>>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct and
>>> heap buffers and it performed well on our tests.
>>>
>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

oleksiys
Administrator
Hi Dan,

> Wow, thanks Alexey for the detailed explanation.  This also explains
> the existence of DirectByteBufferRecord which reuses DirectBuffer
> per-thread rather than creating a new one each time which would happer
> otherwise I assume?
If you mean JDK, it also uses thread-local cache for temporary direct
ByteBuffer.
In general caching direct ByteBuffers (when it's really needed) is much
better than allocating them all the time.

> A related question, if you have a moment:  On my test enviroment the
> connection object returns a recieveBufferSize of 12Mb so if I test
> with high concurrency and I'm using the WorkerThreadIOStrategy with a
> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
> memory will need to be allocated or am I jumping to conlcusions about
> this relationship?
well, it's possible if all threads read at the same time - you'll need
2G memory, that's why it's better to limit the receiveBufferSize either
explicitly for each Connection or using system property (I believe you
know which one).
Another possible change is to reduce the number of threads or use
SameThreadIOStrategy, if tasks you run are not blocking.


Thanks.

WBR,
Alexey.

>
> I'll try the PooledMemoryManager for sure, thanks for the tip.
>
> Dan
>
>
>
> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
> <[hidden email]> wrote:
>> Hi,
>>
>> On 08.12.14 10:37, Daniel Feist wrote:
>>> What I'm wondering is why the following exist:
>>>
>>> 1) TCPNIOUtils.java Line 230-246.
>>>
>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>
>>> Because if a non-direct memoryManager has been chosen I'm not sure why
>>> that choice needs to be overridden and a direct buffer used anyway as
>>> an intermediate step.
>> Pls. take a look at the JDK code here [1] (line 195)
>>
>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the same
>> "intermediate step" - allocate direct ByteBuffer, use it for reading, copy
>> data to our heap ByteBuffer.
>>
>> We could've used that, but in that case we have to guess the read size and
>> do something like this:
>>
>> 1. memoryManager.allocate(large_chunk);
>> 2. read to the allocated heap ByteBuffer
>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>> 2.2 read data to the direct ByteBuffer
>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>
>> Instead of that, we use large enough direct ByteBuffer, read data directly
>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that case).
>> After we read to the direct ByteBuffer we know exactly how many bytes we
>> need to allocate from the MemoryManager.
>> So we just reshuffled the steps sequence above and have this:
>>
>> 1. allocate direct ByteBuffer of size large_chunk
>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't do
>> intermediate allocation step)
>> 3. memoryManager.allocate(read_bytes_count) // we know how many bytes we
>> read
>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>
>> So, by reshuffling the direct ByteBuffer allocation we're able to optimize
>> read path.
>>
>> Makes sense?
>>
>> Thanks.
>>
>> WBR,
>> Alexey.
>>
>> [1]
>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>
>>> 2) DirectByteBufferRecord
>>>
>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>
>>> This is allocating direct buffers, and also caching them per-thread,
>>> yet it's not a MemoryManager implementation, it's something different.
>>> Is this just old/legacy?
>>>
>>> Dan
>>>
>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>> <[hidden email]> wrote:
>>>> Hi Daniel,
>>>>
>>>>
>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>
>>>>> I see there is a system property I can use to limit maximum size of
>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed here
>>>>> rather than simply being used?  This also means there are two
>>>>> allocations and reads per request and not just one.  Can anyone shed
>>>>> some light?
>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
>>>> underneath - allocate (or take pooled) direct ByteBuffer and use it for
>>>> reading.
>>>> So we basically do the same in our code and passing direct ByteBuffer to
>>>> a
>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>> ByteBuffer.
>>>>
>>>> This approach gives us one advantage - once we read to the direct
>>>> ByteBuffer
>>>> - we know the exact amount of bytes we need to allocate from the
>>>> MemoryManager (no guessing).
>>>>
>>>> Hope it will help.
>>>>
>>>> WBR,
>>>> Alexey.
>>>>
>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct and
>>>> heap buffers and it performed well on our tests.
>>>>
>>>>

Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

Daniel Feist
>> A related question, if you have a moment:  On my test enviroment the
>> connection object returns a recieveBufferSize of 12Mb so if I test
>> with high concurrency and I'm using the WorkerThreadIOStrategy with a
>> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
>> memory will need to be allocated or am I jumping to conlcusions about
>> this relationship?
>
> well, it's possible if all threads read at the same time - you'll need 2G
> memory, that's why it's better to limit the receiveBufferSize either
> explicitly for each Connection or using system property (I believe you know
> which one).
> Another possible change is to reduce the number of threads or use
> SameThreadIOStrategy, if tasks you run are not blocking.

Yes, this is what I assumed, just confirming my assumptions. :-)

thanks!


>
>
> Thanks.
>
> WBR,
> Alexey.
>
>
>>
>> I'll try the PooledMemoryManager for sure, thanks for the tip.
>>
>> Dan
>>
>>
>>
>> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
>> <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> On 08.12.14 10:37, Daniel Feist wrote:
>>>>
>>>> What I'm wondering is why the following exist:
>>>>
>>>> 1) TCPNIOUtils.java Line 230-246.
>>>>
>>>>
>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>>
>>>> Because if a non-direct memoryManager has been chosen I'm not sure why
>>>> that choice needs to be overridden and a direct buffer used anyway as
>>>> an intermediate step.
>>>
>>> Pls. take a look at the JDK code here [1] (line 195)
>>>
>>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the same
>>> "intermediate step" - allocate direct ByteBuffer, use it for reading,
>>> copy
>>> data to our heap ByteBuffer.
>>>
>>> We could've used that, but in that case we have to guess the read size
>>> and
>>> do something like this:
>>>
>>> 1. memoryManager.allocate(large_chunk);
>>> 2. read to the allocated heap ByteBuffer
>>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>>> 2.2 read data to the direct ByteBuffer
>>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>>
>>> Instead of that, we use large enough direct ByteBuffer, read data
>>> directly
>>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that
>>> case).
>>> After we read to the direct ByteBuffer we know exactly how many bytes we
>>> need to allocate from the MemoryManager.
>>> So we just reshuffled the steps sequence above and have this:
>>>
>>> 1. allocate direct ByteBuffer of size large_chunk
>>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't do
>>> intermediate allocation step)
>>> 3. memoryManager.allocate(read_bytes_count) // we know how many bytes we
>>> read
>>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>>
>>> So, by reshuffling the direct ByteBuffer allocation we're able to
>>> optimize
>>> read path.
>>>
>>> Makes sense?
>>>
>>> Thanks.
>>>
>>> WBR,
>>> Alexey.
>>>
>>> [1]
>>>
>>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>>
>>>> 2) DirectByteBufferRecord
>>>>
>>>>
>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>>
>>>> This is allocating direct buffers, and also caching them per-thread,
>>>> yet it's not a MemoryManager implementation, it's something different.
>>>> Is this just old/legacy?
>>>>
>>>> Dan
>>>>
>>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>>> <[hidden email]> wrote:
>>>>>
>>>>> Hi Daniel,
>>>>>
>>>>>
>>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>>
>>>>>> I see there is a system property I can use to limit maximum size of
>>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed here
>>>>>> rather than simply being used?  This also means there are two
>>>>>> allocations and reads per request and not just one.  Can anyone shed
>>>>>> some light?
>>>>>
>>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
>>>>> underneath - allocate (or take pooled) direct ByteBuffer and use it for
>>>>> reading.
>>>>> So we basically do the same in our code and passing direct ByteBuffer
>>>>> to
>>>>> a
>>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>>> ByteBuffer.
>>>>>
>>>>> This approach gives us one advantage - once we read to the direct
>>>>> ByteBuffer
>>>>> - we know the exact amount of bytes we need to allocate from the
>>>>> MemoryManager (no guessing).
>>>>>
>>>>> Hope it will help.
>>>>>
>>>>> WBR,
>>>>> Alexey.
>>>>>
>>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct
>>>>> and
>>>>> heap buffers and it performed well on our tests.
>>>>>
>>>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

Daniel Feist
Hi again,

Just a small follow up on this one:

In the end I am both i) increasing amount of direct memory and ii)
limiting the recieve buffer size to 1MB to avoid 12MB being used.

One thing I noticed though, is while there is a system property to
limit the receive buffer size, if I happen to send payloads of 8MB or
more a direct buffer of 12MB will always be allocated per thread and
there is no way to limit this.

This isn't an immediate isssue for me because the kernal/selector
threads do the sending (worker threads perform the recieve) and there
are therefore less of them.. but it's something to be aware of..

Dan

On Tue, Dec 9, 2014 at 6:21 PM, Daniel Feist <[hidden email]> wrote:

>>> A related question, if you have a moment:  On my test enviroment the
>>> connection object returns a recieveBufferSize of 12Mb so if I test
>>> with high concurrency and I'm using the WorkerThreadIOStrategy with a
>>> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
>>> memory will need to be allocated or am I jumping to conlcusions about
>>> this relationship?
>>
>> well, it's possible if all threads read at the same time - you'll need 2G
>> memory, that's why it's better to limit the receiveBufferSize either
>> explicitly for each Connection or using system property (I believe you know
>> which one).
>> Another possible change is to reduce the number of threads or use
>> SameThreadIOStrategy, if tasks you run are not blocking.
>
> Yes, this is what I assumed, just confirming my assumptions. :-)
>
> thanks!
>
>
>>
>>
>> Thanks.
>>
>> WBR,
>> Alexey.
>>
>>
>>>
>>> I'll try the PooledMemoryManager for sure, thanks for the tip.
>>>
>>> Dan
>>>
>>>
>>>
>>> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
>>> <[hidden email]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> On 08.12.14 10:37, Daniel Feist wrote:
>>>>>
>>>>> What I'm wondering is why the following exist:
>>>>>
>>>>> 1) TCPNIOUtils.java Line 230-246.
>>>>>
>>>>>
>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>>>
>>>>> Because if a non-direct memoryManager has been chosen I'm not sure why
>>>>> that choice needs to be overridden and a direct buffer used anyway as
>>>>> an intermediate step.
>>>>
>>>> Pls. take a look at the JDK code here [1] (line 195)
>>>>
>>>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the same
>>>> "intermediate step" - allocate direct ByteBuffer, use it for reading,
>>>> copy
>>>> data to our heap ByteBuffer.
>>>>
>>>> We could've used that, but in that case we have to guess the read size
>>>> and
>>>> do something like this:
>>>>
>>>> 1. memoryManager.allocate(large_chunk);
>>>> 2. read to the allocated heap ByteBuffer
>>>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>>>> 2.2 read data to the direct ByteBuffer
>>>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>>>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>>>
>>>> Instead of that, we use large enough direct ByteBuffer, read data
>>>> directly
>>>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that
>>>> case).
>>>> After we read to the direct ByteBuffer we know exactly how many bytes we
>>>> need to allocate from the MemoryManager.
>>>> So we just reshuffled the steps sequence above and have this:
>>>>
>>>> 1. allocate direct ByteBuffer of size large_chunk
>>>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't do
>>>> intermediate allocation step)
>>>> 3. memoryManager.allocate(read_bytes_count) // we know how many bytes we
>>>> read
>>>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>>>
>>>> So, by reshuffling the direct ByteBuffer allocation we're able to
>>>> optimize
>>>> read path.
>>>>
>>>> Makes sense?
>>>>
>>>> Thanks.
>>>>
>>>> WBR,
>>>> Alexey.
>>>>
>>>> [1]
>>>>
>>>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>>>
>>>>> 2) DirectByteBufferRecord
>>>>>
>>>>>
>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>>>
>>>>> This is allocating direct buffers, and also caching them per-thread,
>>>>> yet it's not a MemoryManager implementation, it's something different.
>>>>> Is this just old/legacy?
>>>>>
>>>>> Dan
>>>>>
>>>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>>>> <[hidden email]> wrote:
>>>>>>
>>>>>> Hi Daniel,
>>>>>>
>>>>>>
>>>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>>>
>>>>>>> I see there is a system property I can use to limit maximum size of
>>>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed here
>>>>>>> rather than simply being used?  This also means there are two
>>>>>>> allocations and reads per request and not just one.  Can anyone shed
>>>>>>> some light?
>>>>>>
>>>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
>>>>>> underneath - allocate (or take pooled) direct ByteBuffer and use it for
>>>>>> reading.
>>>>>> So we basically do the same in our code and passing direct ByteBuffer
>>>>>> to
>>>>>> a
>>>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>>>> ByteBuffer.
>>>>>>
>>>>>> This approach gives us one advantage - once we read to the direct
>>>>>> ByteBuffer
>>>>>> - we know the exact amount of bytes we need to allocate from the
>>>>>> MemoryManager (no guessing).
>>>>>>
>>>>>> Hope it will help.
>>>>>>
>>>>>> WBR,
>>>>>> Alexey.
>>>>>>
>>>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct
>>>>>> and
>>>>>> heap buffers and it performed well on our tests.
>>>>>>
>>>>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

oleksiys
Administrator
Hi,

you mean 12M buffer will be allocated when you *send* a huge packet, right?

WBR,
Alexey.

On 29.12.14 11:18, Daniel Feist wrote:

> Hi again,
>
> Just a small follow up on this one:
>
> In the end I am both i) increasing amount of direct memory and ii)
> limiting the recieve buffer size to 1MB to avoid 12MB being used.
>
> One thing I noticed though, is while there is a system property to
> limit the receive buffer size, if I happen to send payloads of 8MB or
> more a direct buffer of 12MB will always be allocated per thread and
> there is no way to limit this.
>
> This isn't an immediate isssue for me because the kernal/selector
> threads do the sending (worker threads perform the recieve) and there
> are therefore less of them.. but it's something to be aware of..
>
> Dan
>
> On Tue, Dec 9, 2014 at 6:21 PM, Daniel Feist <[hidden email]> wrote:
>>>> A related question, if you have a moment:  On my test enviroment the
>>>> connection object returns a recieveBufferSize of 12Mb so if I test
>>>> with high concurrency and I'm using the WorkerThreadIOStrategy with a
>>>> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
>>>> memory will need to be allocated or am I jumping to conlcusions about
>>>> this relationship?
>>> well, it's possible if all threads read at the same time - you'll need 2G
>>> memory, that's why it's better to limit the receiveBufferSize either
>>> explicitly for each Connection or using system property (I believe you know
>>> which one).
>>> Another possible change is to reduce the number of threads or use
>>> SameThreadIOStrategy, if tasks you run are not blocking.
>> Yes, this is what I assumed, just confirming my assumptions. :-)
>>
>> thanks!
>>
>>
>>>
>>> Thanks.
>>>
>>> WBR,
>>> Alexey.
>>>
>>>
>>>> I'll try the PooledMemoryManager for sure, thanks for the tip.
>>>>
>>>> Dan
>>>>
>>>>
>>>>
>>>> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
>>>> <[hidden email]> wrote:
>>>>> Hi,
>>>>>
>>>>> On 08.12.14 10:37, Daniel Feist wrote:
>>>>>> What I'm wondering is why the following exist:
>>>>>>
>>>>>> 1) TCPNIOUtils.java Line 230-246.
>>>>>>
>>>>>>
>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>>>>
>>>>>> Because if a non-direct memoryManager has been chosen I'm not sure why
>>>>>> that choice needs to be overridden and a direct buffer used anyway as
>>>>>> an intermediate step.
>>>>> Pls. take a look at the JDK code here [1] (line 195)
>>>>>
>>>>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the same
>>>>> "intermediate step" - allocate direct ByteBuffer, use it for reading,
>>>>> copy
>>>>> data to our heap ByteBuffer.
>>>>>
>>>>> We could've used that, but in that case we have to guess the read size
>>>>> and
>>>>> do something like this:
>>>>>
>>>>> 1. memoryManager.allocate(large_chunk);
>>>>> 2. read to the allocated heap ByteBuffer
>>>>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>>>>> 2.2 read data to the direct ByteBuffer
>>>>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>>>>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>>>>
>>>>> Instead of that, we use large enough direct ByteBuffer, read data
>>>>> directly
>>>>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that
>>>>> case).
>>>>> After we read to the direct ByteBuffer we know exactly how many bytes we
>>>>> need to allocate from the MemoryManager.
>>>>> So we just reshuffled the steps sequence above and have this:
>>>>>
>>>>> 1. allocate direct ByteBuffer of size large_chunk
>>>>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't do
>>>>> intermediate allocation step)
>>>>> 3. memoryManager.allocate(read_bytes_count) // we know how many bytes we
>>>>> read
>>>>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>>>>
>>>>> So, by reshuffling the direct ByteBuffer allocation we're able to
>>>>> optimize
>>>>> read path.
>>>>>
>>>>> Makes sense?
>>>>>
>>>>> Thanks.
>>>>>
>>>>> WBR,
>>>>> Alexey.
>>>>>
>>>>> [1]
>>>>>
>>>>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>>>>
>>>>>> 2) DirectByteBufferRecord
>>>>>>
>>>>>>
>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>>>>
>>>>>> This is allocating direct buffers, and also caching them per-thread,
>>>>>> yet it's not a MemoryManager implementation, it's something different.
>>>>>> Is this just old/legacy?
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>>>>> <[hidden email]> wrote:
>>>>>>> Hi Daniel,
>>>>>>>
>>>>>>>
>>>>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>>>>
>>>>>>>> I see there is a system property I can use to limit maximum size of
>>>>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>>>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed here
>>>>>>>> rather than simply being used?  This also means there are two
>>>>>>>> allocations and reads per request and not just one.  Can anyone shed
>>>>>>>> some light?
>>>>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the same
>>>>>>> underneath - allocate (or take pooled) direct ByteBuffer and use it for
>>>>>>> reading.
>>>>>>> So we basically do the same in our code and passing direct ByteBuffer
>>>>>>> to
>>>>>>> a
>>>>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>>>>> ByteBuffer.
>>>>>>>
>>>>>>> This approach gives us one advantage - once we read to the direct
>>>>>>> ByteBuffer
>>>>>>> - we know the exact amount of bytes we need to allocate from the
>>>>>>> MemoryManager (no guessing).
>>>>>>>
>>>>>>> Hope it will help.
>>>>>>>
>>>>>>> WBR,
>>>>>>> Alexey.
>>>>>>>
>>>>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct
>>>>>>> and
>>>>>>> heap buffers and it performed well on our tests.
>>>>>>>
>>>>>>>

Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

Daniel Feist
Yes, the relevant code lines are:

Write (send):  org.glassfish.grizzly.nio.transport.TCPNIOUtils#calcWriteBufferSize
line 212
Recieve:
org.glassfish.grizzly.nio.transport.TCPNIOUtils#allocateAndReadBuffer
line 227

In the case where the http body is large (and chunking isn't used),
and SocketChannel.getSendBufferSize() returns a large size (e.g. 12MB
in my case) there is no way to limit the amount of direct memory used
per thread.  On the other hand, a maximum can be defined for the
recieve buffer size.

So in my case, I'm limiting the recieve buffer to 1MB, but without
modifications to the TCP/IP stack of the host OS, a send buffer of
12MB will be used per thread if packet is 8MB+.  Also the buffer used
will always be greater than 1MB if packet is 670Kb+. There is no way
to limit send buffer to 1MB also.

Dan




On Mon, Dec 29, 2014 at 4:29 PM, Oleksiy Stashok
<[hidden email]> wrote:

> Hi,
>
> you mean 12M buffer will be allocated when you *send* a huge packet, right?
>
> WBR,
> Alexey.
>
>
> On 29.12.14 11:18, Daniel Feist wrote:
>>
>> Hi again,
>>
>> Just a small follow up on this one:
>>
>> In the end I am both i) increasing amount of direct memory and ii)
>> limiting the recieve buffer size to 1MB to avoid 12MB being used.
>>
>> One thing I noticed though, is while there is a system property to
>> limit the receive buffer size, if I happen to send payloads of 8MB or
>> more a direct buffer of 12MB will always be allocated per thread and
>> there is no way to limit this.
>>
>> This isn't an immediate isssue for me because the kernal/selector
>> threads do the sending (worker threads perform the recieve) and there
>> are therefore less of them.. but it's something to be aware of..
>>
>> Dan
>>
>> On Tue, Dec 9, 2014 at 6:21 PM, Daniel Feist <[hidden email]> wrote:
>>>>>
>>>>> A related question, if you have a moment:  On my test enviroment the
>>>>> connection object returns a recieveBufferSize of 12Mb so if I test
>>>>> with high concurrency and I'm using the WorkerThreadIOStrategy with a
>>>>> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
>>>>> memory will need to be allocated or am I jumping to conlcusions about
>>>>> this relationship?
>>>>
>>>> well, it's possible if all threads read at the same time - you'll need
>>>> 2G
>>>> memory, that's why it's better to limit the receiveBufferSize either
>>>> explicitly for each Connection or using system property (I believe you
>>>> know
>>>> which one).
>>>> Another possible change is to reduce the number of threads or use
>>>> SameThreadIOStrategy, if tasks you run are not blocking.
>>>
>>> Yes, this is what I assumed, just confirming my assumptions. :-)
>>>
>>> thanks!
>>>
>>>
>>>>
>>>> Thanks.
>>>>
>>>> WBR,
>>>> Alexey.
>>>>
>>>>
>>>>> I'll try the PooledMemoryManager for sure, thanks for the tip.
>>>>>
>>>>> Dan
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
>>>>> <[hidden email]> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On 08.12.14 10:37, Daniel Feist wrote:
>>>>>>>
>>>>>>> What I'm wondering is why the following exist:
>>>>>>>
>>>>>>> 1) TCPNIOUtils.java Line 230-246.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>>>>>
>>>>>>> Because if a non-direct memoryManager has been chosen I'm not sure
>>>>>>> why
>>>>>>> that choice needs to be overridden and a direct buffer used anyway as
>>>>>>> an intermediate step.
>>>>>>
>>>>>> Pls. take a look at the JDK code here [1] (line 195)
>>>>>>
>>>>>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the
>>>>>> same
>>>>>> "intermediate step" - allocate direct ByteBuffer, use it for reading,
>>>>>> copy
>>>>>> data to our heap ByteBuffer.
>>>>>>
>>>>>> We could've used that, but in that case we have to guess the read size
>>>>>> and
>>>>>> do something like this:
>>>>>>
>>>>>> 1. memoryManager.allocate(large_chunk);
>>>>>> 2. read to the allocated heap ByteBuffer
>>>>>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>>>>>> 2.2 read data to the direct ByteBuffer
>>>>>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>>>>>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>>>>>
>>>>>> Instead of that, we use large enough direct ByteBuffer, read data
>>>>>> directly
>>>>>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that
>>>>>> case).
>>>>>> After we read to the direct ByteBuffer we know exactly how many bytes
>>>>>> we
>>>>>> need to allocate from the MemoryManager.
>>>>>> So we just reshuffled the steps sequence above and have this:
>>>>>>
>>>>>> 1. allocate direct ByteBuffer of size large_chunk
>>>>>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't
>>>>>> do
>>>>>> intermediate allocation step)
>>>>>> 3. memoryManager.allocate(read_bytes_count) // we know how many bytes
>>>>>> we
>>>>>> read
>>>>>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>>>>>
>>>>>> So, by reshuffling the direct ByteBuffer allocation we're able to
>>>>>> optimize
>>>>>> read path.
>>>>>>
>>>>>> Makes sense?
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> WBR,
>>>>>> Alexey.
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>>
>>>>>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>>>>>
>>>>>>> 2) DirectByteBufferRecord
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>>>>>
>>>>>>> This is allocating direct buffers, and also caching them per-thread,
>>>>>>> yet it's not a MemoryManager implementation, it's something
>>>>>>> different.
>>>>>>> Is this just old/legacy?
>>>>>>>
>>>>>>> Dan
>>>>>>>
>>>>>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>>>>>> <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> Hi Daniel,
>>>>>>>>
>>>>>>>>
>>>>>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>>>>>
>>>>>>>>> I see there is a system property I can use to limit maximum size of
>>>>>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>>>>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed
>>>>>>>>> here
>>>>>>>>> rather than simply being used?  This also means there are two
>>>>>>>>> allocations and reads per request and not just one.  Can anyone
>>>>>>>>> shed
>>>>>>>>> some light?
>>>>>>>>
>>>>>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the
>>>>>>>> same
>>>>>>>> underneath - allocate (or take pooled) direct ByteBuffer and use it
>>>>>>>> for
>>>>>>>> reading.
>>>>>>>> So we basically do the same in our code and passing direct
>>>>>>>> ByteBuffer
>>>>>>>> to
>>>>>>>> a
>>>>>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>>>>>> ByteBuffer.
>>>>>>>>
>>>>>>>> This approach gives us one advantage - once we read to the direct
>>>>>>>> ByteBuffer
>>>>>>>> - we know the exact amount of bytes we need to allocate from the
>>>>>>>> MemoryManager (no guessing).
>>>>>>>>
>>>>>>>> Hope it will help.
>>>>>>>>
>>>>>>>> WBR,
>>>>>>>> Alexey.
>>>>>>>>
>>>>>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct
>>>>>>>> and
>>>>>>>> heap buffers and it performed well on our tests.
>>>>>>>>
>>>>>>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

oleksiys
Administrator
Can you pls. file the issue, so we don't forget about this?

Thank you.

WBR,
Alexey.

On 29.12.14 11:44, Daniel Feist wrote:

> Yes, the relevant code lines are:
>
> Write (send):  org.glassfish.grizzly.nio.transport.TCPNIOUtils#calcWriteBufferSize
> line 212
> Recieve:
> org.glassfish.grizzly.nio.transport.TCPNIOUtils#allocateAndReadBuffer
> line 227
>
> In the case where the http body is large (and chunking isn't used),
> and SocketChannel.getSendBufferSize() returns a large size (e.g. 12MB
> in my case) there is no way to limit the amount of direct memory used
> per thread.  On the other hand, a maximum can be defined for the
> recieve buffer size.
>
> So in my case, I'm limiting the recieve buffer to 1MB, but without
> modifications to the TCP/IP stack of the host OS, a send buffer of
> 12MB will be used per thread if packet is 8MB+.  Also the buffer used
> will always be greater than 1MB if packet is 670Kb+. There is no way
> to limit send buffer to 1MB also.
>
> Dan
>
>
>
>
> On Mon, Dec 29, 2014 at 4:29 PM, Oleksiy Stashok
> <[hidden email]> wrote:
>> Hi,
>>
>> you mean 12M buffer will be allocated when you *send* a huge packet, right?
>>
>> WBR,
>> Alexey.
>>
>>
>> On 29.12.14 11:18, Daniel Feist wrote:
>>> Hi again,
>>>
>>> Just a small follow up on this one:
>>>
>>> In the end I am both i) increasing amount of direct memory and ii)
>>> limiting the recieve buffer size to 1MB to avoid 12MB being used.
>>>
>>> One thing I noticed though, is while there is a system property to
>>> limit the receive buffer size, if I happen to send payloads of 8MB or
>>> more a direct buffer of 12MB will always be allocated per thread and
>>> there is no way to limit this.
>>>
>>> This isn't an immediate isssue for me because the kernal/selector
>>> threads do the sending (worker threads perform the recieve) and there
>>> are therefore less of them.. but it's something to be aware of..
>>>
>>> Dan
>>>
>>> On Tue, Dec 9, 2014 at 6:21 PM, Daniel Feist <[hidden email]> wrote:
>>>>>> A related question, if you have a moment:  On my test enviroment the
>>>>>> connection object returns a recieveBufferSize of 12Mb so if I test
>>>>>> with high concurrency and I'm using the WorkerThreadIOStrategy with a
>>>>>> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
>>>>>> memory will need to be allocated or am I jumping to conlcusions about
>>>>>> this relationship?
>>>>> well, it's possible if all threads read at the same time - you'll need
>>>>> 2G
>>>>> memory, that's why it's better to limit the receiveBufferSize either
>>>>> explicitly for each Connection or using system property (I believe you
>>>>> know
>>>>> which one).
>>>>> Another possible change is to reduce the number of threads or use
>>>>> SameThreadIOStrategy, if tasks you run are not blocking.
>>>> Yes, this is what I assumed, just confirming my assumptions. :-)
>>>>
>>>> thanks!
>>>>
>>>>
>>>>> Thanks.
>>>>>
>>>>> WBR,
>>>>> Alexey.
>>>>>
>>>>>
>>>>>> I'll try the PooledMemoryManager for sure, thanks for the tip.
>>>>>>
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
>>>>>> <[hidden email]> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On 08.12.14 10:37, Daniel Feist wrote:
>>>>>>>> What I'm wondering is why the following exist:
>>>>>>>>
>>>>>>>> 1) TCPNIOUtils.java Line 230-246.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>>>>>>
>>>>>>>> Because if a non-direct memoryManager has been chosen I'm not sure
>>>>>>>> why
>>>>>>>> that choice needs to be overridden and a direct buffer used anyway as
>>>>>>>> an intermediate step.
>>>>>>> Pls. take a look at the JDK code here [1] (line 195)
>>>>>>>
>>>>>>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the
>>>>>>> same
>>>>>>> "intermediate step" - allocate direct ByteBuffer, use it for reading,
>>>>>>> copy
>>>>>>> data to our heap ByteBuffer.
>>>>>>>
>>>>>>> We could've used that, but in that case we have to guess the read size
>>>>>>> and
>>>>>>> do something like this:
>>>>>>>
>>>>>>> 1. memoryManager.allocate(large_chunk);
>>>>>>> 2. read to the allocated heap ByteBuffer
>>>>>>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>>>>>>> 2.2 read data to the direct ByteBuffer
>>>>>>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>>>>>>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>>>>>>
>>>>>>> Instead of that, we use large enough direct ByteBuffer, read data
>>>>>>> directly
>>>>>>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that
>>>>>>> case).
>>>>>>> After we read to the direct ByteBuffer we know exactly how many bytes
>>>>>>> we
>>>>>>> need to allocate from the MemoryManager.
>>>>>>> So we just reshuffled the steps sequence above and have this:
>>>>>>>
>>>>>>> 1. allocate direct ByteBuffer of size large_chunk
>>>>>>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't
>>>>>>> do
>>>>>>> intermediate allocation step)
>>>>>>> 3. memoryManager.allocate(read_bytes_count) // we know how many bytes
>>>>>>> we
>>>>>>> read
>>>>>>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>>>>>>
>>>>>>> So, by reshuffling the direct ByteBuffer allocation we're able to
>>>>>>> optimize
>>>>>>> read path.
>>>>>>>
>>>>>>> Makes sense?
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> WBR,
>>>>>>> Alexey.
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>>
>>>>>>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>>>>>>
>>>>>>>> 2) DirectByteBufferRecord
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>>>>>>
>>>>>>>> This is allocating direct buffers, and also caching them per-thread,
>>>>>>>> yet it's not a MemoryManager implementation, it's something
>>>>>>>> different.
>>>>>>>> Is this just old/legacy?
>>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>>>>>>> <[hidden email]> wrote:
>>>>>>>>> Hi Daniel,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>>>>>>
>>>>>>>>>> I see there is a system property I can use to limit maximum size of
>>>>>>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions, but
>>>>>>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed
>>>>>>>>>> here
>>>>>>>>>> rather than simply being used?  This also means there are two
>>>>>>>>>> allocations and reads per request and not just one.  Can anyone
>>>>>>>>>> shed
>>>>>>>>>> some light?
>>>>>>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the
>>>>>>>>> same
>>>>>>>>> underneath - allocate (or take pooled) direct ByteBuffer and use it
>>>>>>>>> for
>>>>>>>>> reading.
>>>>>>>>> So we basically do the same in our code and passing direct
>>>>>>>>> ByteBuffer
>>>>>>>>> to
>>>>>>>>> a
>>>>>>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>>>>>>> ByteBuffer.
>>>>>>>>>
>>>>>>>>> This approach gives us one advantage - once we read to the direct
>>>>>>>>> ByteBuffer
>>>>>>>>> - we know the exact amount of bytes we need to allocate from the
>>>>>>>>> MemoryManager (no guessing).
>>>>>>>>>
>>>>>>>>> Hope it will help.
>>>>>>>>>
>>>>>>>>> WBR,
>>>>>>>>> Alexey.
>>>>>>>>>
>>>>>>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with direct
>>>>>>>>> and
>>>>>>>>> heap buffers and it performed well on our tests.
>>>>>>>>>
>>>>>>>>>

Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

Daniel Feist
Done.

https://java.net/jira/browse/GRIZZLY-1730

Feel free to edit, if issue is not as clear as it could be..

Dan

On Mon, Dec 29, 2014 at 5:31 PM, Oleksiy Stashok
<[hidden email]> wrote:

> Can you pls. file the issue, so we don't forget about this?
>
> Thank you.
>
> WBR,
> Alexey.
>
>
> On 29.12.14 11:44, Daniel Feist wrote:
>>
>> Yes, the relevant code lines are:
>>
>> Write (send):
>> org.glassfish.grizzly.nio.transport.TCPNIOUtils#calcWriteBufferSize
>> line 212
>> Recieve:
>> org.glassfish.grizzly.nio.transport.TCPNIOUtils#allocateAndReadBuffer
>> line 227
>>
>> In the case where the http body is large (and chunking isn't used),
>> and SocketChannel.getSendBufferSize() returns a large size (e.g. 12MB
>> in my case) there is no way to limit the amount of direct memory used
>> per thread.  On the other hand, a maximum can be defined for the
>> recieve buffer size.
>>
>> So in my case, I'm limiting the recieve buffer to 1MB, but without
>> modifications to the TCP/IP stack of the host OS, a send buffer of
>> 12MB will be used per thread if packet is 8MB+.  Also the buffer used
>> will always be greater than 1MB if packet is 670Kb+. There is no way
>> to limit send buffer to 1MB also.
>>
>> Dan
>>
>>
>>
>>
>> On Mon, Dec 29, 2014 at 4:29 PM, Oleksiy Stashok
>> <[hidden email]> wrote:
>>>
>>> Hi,
>>>
>>> you mean 12M buffer will be allocated when you *send* a huge packet,
>>> right?
>>>
>>> WBR,
>>> Alexey.
>>>
>>>
>>> On 29.12.14 11:18, Daniel Feist wrote:
>>>>
>>>> Hi again,
>>>>
>>>> Just a small follow up on this one:
>>>>
>>>> In the end I am both i) increasing amount of direct memory and ii)
>>>> limiting the recieve buffer size to 1MB to avoid 12MB being used.
>>>>
>>>> One thing I noticed though, is while there is a system property to
>>>> limit the receive buffer size, if I happen to send payloads of 8MB or
>>>> more a direct buffer of 12MB will always be allocated per thread and
>>>> there is no way to limit this.
>>>>
>>>> This isn't an immediate isssue for me because the kernal/selector
>>>> threads do the sending (worker threads perform the recieve) and there
>>>> are therefore less of them.. but it's something to be aware of..
>>>>
>>>> Dan
>>>>
>>>> On Tue, Dec 9, 2014 at 6:21 PM, Daniel Feist <[hidden email]> wrote:
>>>>>>>
>>>>>>> A related question, if you have a moment:  On my test enviroment the
>>>>>>> connection object returns a recieveBufferSize of 12Mb so if I test
>>>>>>> with high concurrency and I'm using the WorkerThreadIOStrategy with a
>>>>>>> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
>>>>>>> memory will need to be allocated or am I jumping to conlcusions about
>>>>>>> this relationship?
>>>>>>
>>>>>> well, it's possible if all threads read at the same time - you'll need
>>>>>> 2G
>>>>>> memory, that's why it's better to limit the receiveBufferSize either
>>>>>> explicitly for each Connection or using system property (I believe you
>>>>>> know
>>>>>> which one).
>>>>>> Another possible change is to reduce the number of threads or use
>>>>>> SameThreadIOStrategy, if tasks you run are not blocking.
>>>>>
>>>>> Yes, this is what I assumed, just confirming my assumptions. :-)
>>>>>
>>>>> thanks!
>>>>>
>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>> WBR,
>>>>>> Alexey.
>>>>>>
>>>>>>
>>>>>>> I'll try the PooledMemoryManager for sure, thanks for the tip.
>>>>>>>
>>>>>>> Dan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
>>>>>>> <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On 08.12.14 10:37, Daniel Feist wrote:
>>>>>>>>>
>>>>>>>>> What I'm wondering is why the following exist:
>>>>>>>>>
>>>>>>>>> 1) TCPNIOUtils.java Line 230-246.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>>>>>>>
>>>>>>>>> Because if a non-direct memoryManager has been chosen I'm not sure
>>>>>>>>> why
>>>>>>>>> that choice needs to be overridden and a direct buffer used anyway
>>>>>>>>> as
>>>>>>>>> an intermediate step.
>>>>>>>>
>>>>>>>> Pls. take a look at the JDK code here [1] (line 195)
>>>>>>>>
>>>>>>>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the
>>>>>>>> same
>>>>>>>> "intermediate step" - allocate direct ByteBuffer, use it for
>>>>>>>> reading,
>>>>>>>> copy
>>>>>>>> data to our heap ByteBuffer.
>>>>>>>>
>>>>>>>> We could've used that, but in that case we have to guess the read
>>>>>>>> size
>>>>>>>> and
>>>>>>>> do something like this:
>>>>>>>>
>>>>>>>> 1. memoryManager.allocate(large_chunk);
>>>>>>>> 2. read to the allocated heap ByteBuffer
>>>>>>>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>>>>>>>> 2.2 read data to the direct ByteBuffer
>>>>>>>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>>>>>>>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>>>>>>>
>>>>>>>> Instead of that, we use large enough direct ByteBuffer, read data
>>>>>>>> directly
>>>>>>>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that
>>>>>>>> case).
>>>>>>>> After we read to the direct ByteBuffer we know exactly how many
>>>>>>>> bytes
>>>>>>>> we
>>>>>>>> need to allocate from the MemoryManager.
>>>>>>>> So we just reshuffled the steps sequence above and have this:
>>>>>>>>
>>>>>>>> 1. allocate direct ByteBuffer of size large_chunk
>>>>>>>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't
>>>>>>>> do
>>>>>>>> intermediate allocation step)
>>>>>>>> 3. memoryManager.allocate(read_bytes_count) // we know how many
>>>>>>>> bytes
>>>>>>>> we
>>>>>>>> read
>>>>>>>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>>>>>>>
>>>>>>>> So, by reshuffling the direct ByteBuffer allocation we're able to
>>>>>>>> optimize
>>>>>>>> read path.
>>>>>>>>
>>>>>>>> Makes sense?
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> WBR,
>>>>>>>> Alexey.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>>>>>>>
>>>>>>>>> 2) DirectByteBufferRecord
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>>>>>>>
>>>>>>>>> This is allocating direct buffers, and also caching them
>>>>>>>>> per-thread,
>>>>>>>>> yet it's not a MemoryManager implementation, it's something
>>>>>>>>> different.
>>>>>>>>> Is this just old/legacy?
>>>>>>>>>
>>>>>>>>> Dan
>>>>>>>>>
>>>>>>>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>>>>>>>> <[hidden email]> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Daniel,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>>>>>>>
>>>>>>>>>>> I see there is a system property I can use to limit maximum size
>>>>>>>>>>> of
>>>>>>>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions,
>>>>>>>>>>> but
>>>>>>>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed
>>>>>>>>>>> here
>>>>>>>>>>> rather than simply being used?  This also means there are two
>>>>>>>>>>> allocations and reads per request and not just one.  Can anyone
>>>>>>>>>>> shed
>>>>>>>>>>> some light?
>>>>>>>>>>
>>>>>>>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the
>>>>>>>>>> same
>>>>>>>>>> underneath - allocate (or take pooled) direct ByteBuffer and use
>>>>>>>>>> it
>>>>>>>>>> for
>>>>>>>>>> reading.
>>>>>>>>>> So we basically do the same in our code and passing direct
>>>>>>>>>> ByteBuffer
>>>>>>>>>> to
>>>>>>>>>> a
>>>>>>>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>>>>>>>> ByteBuffer.
>>>>>>>>>>
>>>>>>>>>> This approach gives us one advantage - once we read to the direct
>>>>>>>>>> ByteBuffer
>>>>>>>>>> - we know the exact amount of bytes we need to allocate from the
>>>>>>>>>> MemoryManager (no guessing).
>>>>>>>>>>
>>>>>>>>>> Hope it will help.
>>>>>>>>>>
>>>>>>>>>> WBR,
>>>>>>>>>> Alexey.
>>>>>>>>>>
>>>>>>>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with
>>>>>>>>>> direct
>>>>>>>>>> and
>>>>>>>>>> heap buffers and it performed well on our tests.
>>>>>>>>>>
>>>>>>>>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: DirectByteBufferRecord and HeapMemoryManager?

oleksiys
Administrator
Fixed.

Thank you.


On 29.12.14 13:26, Daniel Feist wrote:

> Done.
>
> https://java.net/jira/browse/GRIZZLY-1730
>
> Feel free to edit, if issue is not as clear as it could be..
>
> Dan
>
> On Mon, Dec 29, 2014 at 5:31 PM, Oleksiy Stashok
> <[hidden email]> wrote:
>> Can you pls. file the issue, so we don't forget about this?
>>
>> Thank you.
>>
>> WBR,
>> Alexey.
>>
>>
>> On 29.12.14 11:44, Daniel Feist wrote:
>>> Yes, the relevant code lines are:
>>>
>>> Write (send):
>>> org.glassfish.grizzly.nio.transport.TCPNIOUtils#calcWriteBufferSize
>>> line 212
>>> Recieve:
>>> org.glassfish.grizzly.nio.transport.TCPNIOUtils#allocateAndReadBuffer
>>> line 227
>>>
>>> In the case where the http body is large (and chunking isn't used),
>>> and SocketChannel.getSendBufferSize() returns a large size (e.g. 12MB
>>> in my case) there is no way to limit the amount of direct memory used
>>> per thread.  On the other hand, a maximum can be defined for the
>>> recieve buffer size.
>>>
>>> So in my case, I'm limiting the recieve buffer to 1MB, but without
>>> modifications to the TCP/IP stack of the host OS, a send buffer of
>>> 12MB will be used per thread if packet is 8MB+.  Also the buffer used
>>> will always be greater than 1MB if packet is 670Kb+. There is no way
>>> to limit send buffer to 1MB also.
>>>
>>> Dan
>>>
>>>
>>>
>>>
>>> On Mon, Dec 29, 2014 at 4:29 PM, Oleksiy Stashok
>>> <[hidden email]> wrote:
>>>> Hi,
>>>>
>>>> you mean 12M buffer will be allocated when you *send* a huge packet,
>>>> right?
>>>>
>>>> WBR,
>>>> Alexey.
>>>>
>>>>
>>>> On 29.12.14 11:18, Daniel Feist wrote:
>>>>> Hi again,
>>>>>
>>>>> Just a small follow up on this one:
>>>>>
>>>>> In the end I am both i) increasing amount of direct memory and ii)
>>>>> limiting the recieve buffer size to 1MB to avoid 12MB being used.
>>>>>
>>>>> One thing I noticed though, is while there is a system property to
>>>>> limit the receive buffer size, if I happen to send payloads of 8MB or
>>>>> more a direct buffer of 12MB will always be allocated per thread and
>>>>> there is no way to limit this.
>>>>>
>>>>> This isn't an immediate isssue for me because the kernal/selector
>>>>> threads do the sending (worker threads perform the recieve) and there
>>>>> are therefore less of them.. but it's something to be aware of..
>>>>>
>>>>> Dan
>>>>>
>>>>> On Tue, Dec 9, 2014 at 6:21 PM, Daniel Feist <[hidden email]> wrote:
>>>>>>>> A related question, if you have a moment:  On my test enviroment the
>>>>>>>> connection object returns a recieveBufferSize of 12Mb so if I test
>>>>>>>> with high concurrency and I'm using the WorkerThreadIOStrategy with a
>>>>>>>> thread pool of 200 threads,  does that mean that up tp 2.3Gb off-heap
>>>>>>>> memory will need to be allocated or am I jumping to conlcusions about
>>>>>>>> this relationship?
>>>>>>> well, it's possible if all threads read at the same time - you'll need
>>>>>>> 2G
>>>>>>> memory, that's why it's better to limit the receiveBufferSize either
>>>>>>> explicitly for each Connection or using system property (I believe you
>>>>>>> know
>>>>>>> which one).
>>>>>>> Another possible change is to reduce the number of threads or use
>>>>>>> SameThreadIOStrategy, if tasks you run are not blocking.
>>>>>> Yes, this is what I assumed, just confirming my assumptions. :-)
>>>>>>
>>>>>> thanks!
>>>>>>
>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> WBR,
>>>>>>> Alexey.
>>>>>>>
>>>>>>>
>>>>>>>> I'll try the PooledMemoryManager for sure, thanks for the tip.
>>>>>>>>
>>>>>>>> Dan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Dec 8, 2014 at 7:04 PM, Oleksiy Stashok
>>>>>>>> <[hidden email]> wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On 08.12.14 10:37, Daniel Feist wrote:
>>>>>>>>>> What I'm wondering is why the following exist:
>>>>>>>>>>
>>>>>>>>>> 1) TCPNIOUtils.java Line 230-246.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/transport/TCPNIOUtils.java#L230)
>>>>>>>>>>
>>>>>>>>>> Because if a non-direct memoryManager has been chosen I'm not sure
>>>>>>>>>> why
>>>>>>>>>> that choice needs to be overridden and a direct buffer used anyway
>>>>>>>>>> as
>>>>>>>>>> an intermediate step.
>>>>>>>>> Pls. take a look at the JDK code here [1] (line 195)
>>>>>>>>>
>>>>>>>>> if the passed ByteBuffer is not direct ByteBuffer - JDK will do the
>>>>>>>>> same
>>>>>>>>> "intermediate step" - allocate direct ByteBuffer, use it for
>>>>>>>>> reading,
>>>>>>>>> copy
>>>>>>>>> data to our heap ByteBuffer.
>>>>>>>>>
>>>>>>>>> We could've used that, but in that case we have to guess the read
>>>>>>>>> size
>>>>>>>>> and
>>>>>>>>> do something like this:
>>>>>>>>>
>>>>>>>>> 1. memoryManager.allocate(large_chunk);
>>>>>>>>> 2. read to the allocated heap ByteBuffer
>>>>>>>>> 2.1. JDK allocates direct ByteBuffer of size large_chunk
>>>>>>>>> 2.2 read data to the direct ByteBuffer
>>>>>>>>> 2.3 copy direct ByteBuffer data to our heap ByteBuffer
>>>>>>>>> 3. release unused part of ByteBuffer back to MemoryManager (if any)
>>>>>>>>>
>>>>>>>>> Instead of that, we use large enough direct ByteBuffer, read data
>>>>>>>>> directly
>>>>>>>>> to this ByteBuffer (JDK doesn't use intermediate ByteBuffer in that
>>>>>>>>> case).
>>>>>>>>> After we read to the direct ByteBuffer we know exactly how many
>>>>>>>>> bytes
>>>>>>>>> we
>>>>>>>>> need to allocate from the MemoryManager.
>>>>>>>>> So we just reshuffled the steps sequence above and have this:
>>>>>>>>>
>>>>>>>>> 1. allocate direct ByteBuffer of size large_chunk
>>>>>>>>> 2. read to the allocated direct ByteBuffer (in this case JDK doesn't
>>>>>>>>> do
>>>>>>>>> intermediate allocation step)
>>>>>>>>> 3. memoryManager.allocate(read_bytes_count) // we know how many
>>>>>>>>> bytes
>>>>>>>>> we
>>>>>>>>> read
>>>>>>>>> 4. copy direct ByteBuffer to allocated heap ByteBuffer
>>>>>>>>>
>>>>>>>>> So, by reshuffling the direct ByteBuffer allocation we're able to
>>>>>>>>> optimize
>>>>>>>>> read path.
>>>>>>>>>
>>>>>>>>> Makes sense?
>>>>>>>>>
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> WBR,
>>>>>>>>> Alexey.
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/sun/nio/ch/IOUtil.java#IOUtil.read%28java.io.FileDescriptor%2Cjava.nio.ByteBuffer%2Clong%2Csun.nio.ch.NativeDispatcher%29
>>>>>>>>>
>>>>>>>>>> 2) DirectByteBufferRecord
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (https://github.com/GrizzlyNIO/grizzly-mirror/blob/2.3.x/modules/grizzly/src/main/java/org/glassfish/grizzly/nio/DirectByteBufferRecord.java#L54)
>>>>>>>>>>
>>>>>>>>>> This is allocating direct buffers, and also caching them
>>>>>>>>>> per-thread,
>>>>>>>>>> yet it's not a MemoryManager implementation, it's something
>>>>>>>>>> different.
>>>>>>>>>> Is this just old/legacy?
>>>>>>>>>>
>>>>>>>>>> Dan
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 8, 2014 at 6:03 PM, Oleksiy Stashok
>>>>>>>>>> <[hidden email]> wrote:
>>>>>>>>>>> Hi Daniel,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 08.12.14 09:32, Daniel Feist wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I see there is a system property I can use to limit maximum size
>>>>>>>>>>>> of
>>>>>>>>>>>> these direct bufffers and thus avoid the OutOfMemoryExceptions,
>>>>>>>>>>>> but
>>>>>>>>>>>> I'm wondering why the MemoryManager is explicitlu being bypassed
>>>>>>>>>>>> here
>>>>>>>>>>>> rather than simply being used?  This also means there are two
>>>>>>>>>>>> allocations and reads per request and not just one.  Can anyone
>>>>>>>>>>>> shed
>>>>>>>>>>>> some light?
>>>>>>>>>>> Well, if you pass HeapByteBuffer to a SocketChannel - it'll do the
>>>>>>>>>>> same
>>>>>>>>>>> underneath - allocate (or take pooled) direct ByteBuffer and use
>>>>>>>>>>> it
>>>>>>>>>>> for
>>>>>>>>>>> reading.
>>>>>>>>>>> So we basically do the same in our code and passing direct
>>>>>>>>>>> ByteBuffer
>>>>>>>>>>> to
>>>>>>>>>>> a
>>>>>>>>>>> SocketChannel, so SocketChannel itself will not allocate direct
>>>>>>>>>>> ByteBuffer.
>>>>>>>>>>>
>>>>>>>>>>> This approach gives us one advantage - once we read to the direct
>>>>>>>>>>> ByteBuffer
>>>>>>>>>>> - we know the exact amount of bytes we need to allocate from the
>>>>>>>>>>> MemoryManager (no guessing).
>>>>>>>>>>>
>>>>>>>>>>> Hope it will help.
>>>>>>>>>>>
>>>>>>>>>>> WBR,
>>>>>>>>>>> Alexey.
>>>>>>>>>>>
>>>>>>>>>>> PS: Pls. give a shot to PooledMemoryManager, it can work with
>>>>>>>>>>> direct
>>>>>>>>>>> and
>>>>>>>>>>> heap buffers and it performed well on our tests.
>>>>>>>>>>>
>>>>>>>>>>>