Chunk File Upload

A while back I worked on an AIR client which had to be capable uploading very large files. In this scenario you have to upload files in little chunks. The research on how to solve this took me quite a while so I thought it would make sense to post my findings.

More or less I have taken the NirvanixBinaryUploader as a base and then extended (I should say forked) it to fit my needs.

The class UploadFile takes a file and upload URL as constructor arguments. You can also define a separate resumeUrl if it differs from the upload URL. Further you can set the flag doCalcMD5 to true which is kind of experimental. It works using the as3corelib MD5Stream class but calculating the MD5 hash for large chunks like 1MB took about 20-30 secs on my machine.

I have also implement getters for bytesLoaded and bytesTotal so I can use an instance as a data source for the Flex ProgressBar in polled mode which comes in very handy.

<mx:ProgressBar source="{uploadFile}" mode="polled" label="Uploading" />

If you want to use my code I guess you want to subclass my class because it contains example code and you probably want to override at least the createBody method. However, if you have a similar scenario like me, the code might be a good start.

Another interesting read on this topic was A long journey through chunked transfer and file uploading.

Here now the code:

8 comments to Chunk File Upload

  • Hi Sönke,

    Your post doesn’t seem to be complete, and the download link to GitHub for your UploadFile class doesn’t work. I also can’t seem to find this class in any of your GitHub repo’s.
    Could you please tell me where I can find the code?

    Thanks!

    Sven

  • Must have been a hickup in my browser Sönke, the page comes through fine now, sorry!

  • jo

    Hi Sönke,
    what do you use on server side? A servlet? Maybe you can post some code?
    Would be great!
    Thanks! Jo

  • Thanks for the feedback! Unfortunately I can’t share the server side code but in this case it was Perl.

  • jo

    Hi Sönke, but how do you separate the parameters and the values from the “raw” file data?
    I’ve monitored an upload request with Charles (Chunk size 20B, request 2 of 5). The file content comes together with the filename..? Shouldn’t there be an extra content-disposition chapter? Or how did you handle the requests in your case? Thanks, Jo

    POST /hello/uploadhandler HTTP/1.1
    Referer: app:/Uploader.swf
    Accept: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9, text/plain;q=0.8, text/css, image/png, image/jpeg, image/gif;q=0.8, application/x-shockwave-flash, video/mp4;q=0.9, flv-application/octet-stream;q=0.8, video/x-flv;q=0.7, audio/mp4, application/futuresplash, */*;q=0.5
    x-flash-version: 10,0,42,34
    Content-Type: multipart/form-data; boundary=———————13286573512345675675865
    Content-Length: 396
    User-Agent: Mozilla/5.0 (Windows; U; de-DE) AppleWebKit/526.9+ (KHTML, like Gecko) AdobeAIR/1.5.3
    Host: 192.168.77.24:8080
    Pragma: no-cache

    ———————–13286573512345675675865
    content-disposition: form-data; name=”foo”

    bar
    ———————–13286573512345675675865
    content-disposition: form-data; name=”fileid”

    123
    ———————–13286573512345675675865
    content-disposition: form-data; name=”filename”

    abc.txt
    eee
    fff
    ggg
    hhh

    ———————–13286573512345675675865–

  • Each variable has its own content-disposition. Have a look at the method createVariable.

  • jo

    Hi Sönke, do you think that there’s a chance to avoid the ui freeze while
    calling “loader = new URLLoader( request );”? Small chunks minimize the freeze but the
    upload takes much longer. Larger Chunks are good for upload speed but unfortunately the air app
    freezes while calling the URLLoader. A native proccess for uploading would be great, but is it
    possible to call an air app as native proccess? Or how did you solve the problem?

  • No idea, sorry. Maybe the Flashplayer will handle this better in the future.