Grizzly fileupload

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Grizzly fileupload

gagansnt
I am working on a server application using Grizzly which can handle uploading of very large files. I am looking for a sample example which shows how to parse a POST request and save file on server side.

If anybody had written such example then I would appreciate if you could share it.

Thanks in advance.
Gagan
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

oleksiys
Administrator
Hi Gagan,

I just realized that you might use the Nabble mailing list and didn't get response notification [1].
Until we're working on fixing this, you can check for the response on the java.net Grizzly mailing list directly [2].

WBR,
Alexey.

[1] http://old.nabble.com/Grizzly-Nabble-mailing-list-is-temporary-unavailable-tt31152872.html
[2] http://java.net/projects/grizzly/lists/users/archive/2011-03/message/8


gagansnt wrote
I am working on a server application using Grizzly which can handle uploading of very large files. I am looking for a sample example which shows how to parse a POST request and save file on server side.

If anybody had written such example then I would appreciate if you could share it.

Thanks in advance.
Gagan
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

gagansnt
Thanks for the reply Alexey.

Not sure how to respond to your answer on grizzly mailing list. Because when I try replying on the grizzly forum it says "You are not allowed to do that (send lists)" So I am reply here only.

Your sample code may work for text or doc files but it won't work for binary files. We may need NIOInputStream in that case. I was stuck in a problem where I have to parse POST fileupload request. If I use NIOReader then binary files got corrupted because of Char enconding problem and if I use NIOInputStream then I had no way of getting the POST request boundary, content-length info and know from which position actual file contents start. Also there is no readLine function available in NIOReader or NIOInputStream classes. I am putting all this just to help developers of the API to know.

What I did is...
1. I used Grizzly's NIOInputStream,
2. read initial chunk in a byte buffer,
3. Converted that buffer to a String
4. Split that String based on new line character and stored in a String array
5. Read the first 4 files and parsed them further to extract relevant info, stored boundary info separately so that I could know where file ends.
6. Sum the length of first 4 lines then added 4 in it to make for new line character that I used to split the lines
7. Then Substring the original string from (length of first 4 lines+4, String length)
8. Converted String to bytes again.
9. Read next chunks do the same and look for the boundary in them
10. When I got the boundary, I removed it from the contents and closed the output file buffer and saved it.

One very important thing while doing byte to String and String to byte conversion pls make sure you use Char encoding ISO8859_1 for example
String data  = new String(buf,0,len,"ISO8859_1");
And data.getBytes("ISO8859_1")


Not sure how far I can go in it because my application is now very specific and work on POST file uploads. I may be doing something wrong please let me know if thr is any other easier and more elegant way.
 
A readline function would be handy if you people can add it.

oleksiys wrote
Hi Gagan,

I just realized that you might use the Nabble mailing list and didn't get response notification [1].
Until we're working on fixing this, you can check for the response on the java.net Grizzly mailing list directly [2].

WBR,
Alexey.

[1] http://old.nabble.com/Grizzly-Nabble-mailing-list-is-temporary-unavailable-tt31152872.html
[2] http://java.net/projects/grizzly/lists/users/archive/2011-03/message/8


gagansnt wrote
I am working on a server application using Grizzly which can handle uploading of very large files. I am looking for a sample example which shows how to parse a POST request and save file on server side.

If anybody had written such example then I would appreciate if you could share it.

Thanks in advance.
Gagan
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

oleksiys
Administrator
Hi Gagan,

yes, you're absolutely right, you need to use NIOInputStream, rather than Reader.
Actually multipart content sounds like a great contribution to Grizzly, so we'd really like to help you with that.

The steps you're doing look right, though I'm not sure we have to convert byte[] -> String and vise versa, IMO it's not very optimal, it's probably better to look for boundary string directly in the byte[] or Grizzly Buffer.

If it's possible can you pls. share the code and we can try to help you make it better and more optimal. And it would be great if then you will be able to contribute the code back to Grizzly, so other users may use it :)

Thanks.

WBR,
Alexey.
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

gagansnt
We need to change bytes to String so that we could get the first 4 lines. Though this can be done using byte array manipulation by looking for new line character. I have attached my code. It is still half baked and have lot of adhoc things..like checking for new line...I am working on windows so I m checking for "\r\n" character. In order to make it more generic we may need to check for "\n" as well.

There is one more issue I m facing with this current code, sometimes the call to  in.ready()  in GrizzlyServer class gives false at the start of the file upload and because of which my code to remove the header gives error.

Also one more thing, Is NIOInputStream has a limit to the number of input connection. Is there some default setting and if yes then how can I change it.

I will be happy to contribute in any possible way.

Thanks,
Gagan

GrizzlyWebserver.rar
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

gagansnt
Hi Alexey

I have one more question. There are number of project listed under the heading Projects using Grizzly on main page of Grizzly website. Does any of these projects solves the simultaneously big file upload problem. And if Grizzly does not have multi-part fileupload support yet then it means none of the project listed there have support for it.

Thanks,
Gagan
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

oleksiys
Administrator
Hi Gagan,

I have one more question. There are number of project listed under the
heading Projects using Grizzly on main page of Grizzly website. Does any of
these projects solves the simultaneously big file upload problem. And if
Grizzly does not have multi-part fileupload support yet then it means none
of the project listed there have support for it.
Currently we don't have multi-part message parser for non-blocking InputStream.

But still you're able to use any existing multi-part parser with regular blocking streams.
For example the one from O'Reilly [1], which you can use with InputStream:

final InputStream in = request.getInputStream(*true*);



Thanks.

WBR,
Alexey.

[1] http://www.koders.com/java/fid09524E23152581E993DB11E966C08182D9A9B26C.aspx?s=cdef%3aMultipartParser+oreilly

Thanks, 
Gagan


--
View this message in context: http://grizzly.1045725.n5.nabble.com/Grizzly-fileupload-tp3732246p3847647.html
Sent from the Grizzly - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

gagansnt
Hi Alexey,

When do you think non-blocking multipart file upload will be available in Grizzly or any of the other dependent projects. Do we have any set timelines for it.

Thanks,
Gagan
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

oleksiys
Administrator
Hi Gagan,

it's on my list,
should be ready for testing approx. in a week.

WBR,
Alexey.

On 03/22/2011 11:21 PM, gagansnt wrote:

> Hi Alexey,
>
> When do you think non-blocking multipart file upload will be available in
> Grizzly or any of the other dependent projects. Do we have any set timelines
> for it.
>
> Thanks,
> Gagan
>
> --
> View this message in context: http://grizzly.1045725.n5.nabble.com/Grizzly-fileupload-tp3732246p4258260.html
> Sent from the Grizzly - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

gagansnt
gr8! I will be eagerly waiting for it.

Thanks,
Gagan
Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

oleksiys
Administrator
Hi Gagan,

here is the sample how to upload files using HTML forms (multipart-form/data) in Grizzly 2.0 in async. and non-blocking way.
If you have any ideas what might be improved or any feedback - let me know.

I'm going to add correspondent section into our documentation.

Thanks.

WBR,
Alexey.

On 03/25/2011 06:44 AM, gagansnt wrote:
gr8! I will be eagerly waiting for it. 

Thanks, 
Gagan 

--
View this message in context: http://grizzly.1045725.n5.nabble.com/Grizzly-fileupload-tp3732246p4263562.html
Sent from the Grizzly - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Grizzly fileupload

oleksiys
Administrator
Here :)

http://java.net/projects/grizzly/sources/git/show/samples/http-multipart-samples


On 04/18/2011 02:33 PM, Oleksiy Stashok wrote:
Hi Gagan,

here is the sample how to upload files using HTML forms (multipart-form/data) in Grizzly 2.0 in async. and non-blocking way.
If you have any ideas what might be improved or any feedback - let me know.

I'm going to add correspondent section into our documentation.

Thanks.

WBR,
Alexey.

On 03/25/2011 06:44 AM, gagansnt wrote:
gr8! I will be eagerly waiting for it. 

Thanks, 
Gagan 

--
View this message in context: http://grizzly.1045725.n5.nabble.com/Grizzly-fileupload-tp3732246p4263562.html
Sent from the Grizzly - Users mailing list archive at Nabble.com.