upload_progress: tracking already registered even if its not-registered and unique

[khizarsonu]

hello,

my problem is than when i upload file on nginx.. and getting upload_progress using its module i'm getting error in nginx_error.log
error is "upload_progress: tracking already registered"

i verified that upload_progress id is uniq for every form that are populated on web pages

then why i'm getting this error.

here is my virtual host configuration

server
{
listen 80;
server_name localhost;
index index.html index.htm index.php default.html default.htm default.php;
root /home/wwwroot/mobisfy;

include other.conf;
location ~ .*.(php|php5)?$
{
fastcgi_pass 127.0.0.1:90;
fastcgi_index index.php;
include fcgi.conf;
track_uploads proxied 30s;
}

location ^~ /progress {

report uploads tracked in the 'proxied' zone

report_uploads proxied;
}

location = /favicon.ico {
log_not_found off;
access_log off;
}

location = /robots.txt {
allow all;
log_not_found off;
access_log off;
}

rewrite ^/uploads/files/(.+?)/(.+?)/(.+?)/(.+?)$ /uploads/files/$1?st=$2&e=$3 last;

location ^~ /uploads/files/
{
secure_link $arg_st,$arg_e;
secure_link_md5 sonu$uri$arg_e;

if ($secure_link = 0) {
return 403;
}

if ($secure_link = "")
{
return 500;
}

if ($secure_link != 1)
{
return 500;
}

if everything is ok secure_link is 1

}

location ~ /. {
deny all;
access_log off;
log_not_found off;
}

location ~ /(protected|framework|nbproject|make) {
deny all;
access_log off;
log_not_found off;
}

location ~ /themes/\w+/views {
deny all;
access_log off;
log_not_found off;
}

location /images/r
{
location ~* .jpg$
{
try_files $uri $uri/ /index.php$is_args$args;

return 500;

}
}
location / {
try_files $uri $uri/ /index.php$is_args$args;
}

location ~* .(js|css|png|jpg|jpeg|gif|ico)$ {
expires 24h;
log_not_found off;
}

location ~ .*.(gif|jpg|jpeg|png|bmp|swf)$
{
expires 30d;
}

location ~ .*.(js|css)?$
{
expires 12h;
}

access_log off;
}

[pulse00]

@khizarsonu we're running in a similar issue. have you solved this somehow?

[masterzen]

If you can send me a full debug log I can check what is wrong (you need to build nginx with --with-debug). Most of the time the issue is that the same id is used for a subsequent upload.

[pulse00]

@masterzen i've already turned on debug logging. What i see in the logs is upload_progress: tracking already registered id: upload_536cff9453c7b0.12009139.

I am very sure that the problem is not caused by duplicate progress ids, as we are generating an unique ID for each client. Furthermore, we can reproduce this behavior which is restricted to one specific server (ubuntu 12.04, nginx-extras 1.6.0). The really weird issue is the following:

• If the client is connected via WIFI, the upload works just fine
• If the client is connected via LAN, the upload freezes with the above mentioned warning.

There are no other log messages on the server. Also, we made sure that there's only one single upload happening on the server when we test it (it's our development machine).

Here's what happens on the client:

• We POST a form with a large file (~ 200 MB) via AJAX using a jquery form plugin
• We call the /progress URL via ajax once a second to check the progress and display a progress bar
• After 1 or 2 seconds, the ajax call to nginx_upload_progress keeps sending the same received value for each request
• After 3 to 4 seconds, the ajax call returns error: 500 status, at this time we see the tracking already registered id error in the debug logs.

The original ajax POST request in the browser console at this point is still in state Pending, and will continue to be in this state until you close the window. No further logs on the server.

Here's our nginx.conf

Are there any more debug infos i could provide?

[masterzen]

@pulse00, please send me privately the full debug.log (you can anonymize it) so that I can have a look. Restart nginx before starting the upload, then stop the experiment at the moment you get the error.

The usual difference between wifi and ethernet is the speed and latency. It might be possible this triggers a bug somewhere.

[pulse00]

do you need the log at info or debug level?

[masterzen]

@pulse00 debug please (it might be too large to send it by e-mail, in which case if you can upload this to an http server somewhere that'd be great).

[pulse00]

@masterzen here's the gzipped log file - it's ~ 230 MB unzipped.

However, after activating debug level the error is not occuring anymore, but it's now showing different behavior: After the upload starts, all calls to /progress result in the received key to be 0. After a while, when the whole file has been transmitted to the server, the upload has succeeded.

Please let me know if there's anything else you need.

[masterzen]

@pulse00 I thrown an eye on the log file and something struck me:

2014/05/10 00:16:58 [debug] 10393#0: *8 http client request body recv -2
2014/05/10 00:16:58 [debug] 10393#0: *8 http client request body rest 224229458
2014/05/10 00:16:58 [debug] 10393#0: upload-progress: read_event_handler found node: upload_536d5084197908.68183439
2014/05/10 00:16:58 [debug] 10393#0: upload-progress: read_event_handler storing rest 224229458/224229458 for upload_536d5084197908.68183439

As you can see above, nginx http module says there's still 224229458 KiB of upload to be uploaded , and as you can see that's what is stored in the upload-progress.
The issue is that normally when the upload progresses, this number should decrease, in your case it stays like this until the end (I didn't check every occurence).

I think you're seeing the issue #37, which I haven't found the solution for yet.

Can you try to disable TLS/SSL in your experiment and upload again to see if that's the same issue?

[masterzen]

@pulse00 I've found the issue now. A change in nginx 1.3.9 made that the number of bytes that rest to be uploaded is not updated anymore by nginx unless nginx recycles the buffer where it stores the incoming body.

This buffer is sized with the client_body_buffer_size settings. Yours is at 1GB (which IMHO is way too large unless you have a very powerful server with 64 GB of RAM that could allow at least 64 upload of a 1GB file in parallel (or your uploads are not too heavy)).

If you reduce this to let's say 128k or even 1M and upload a large file, you'll see the upload progress working.

I will fix this issue on the side of the upload progress module later this week-end (I need to find a way to capture the missing information), but I'd appreciate if you could test with a reduced client_body_buffer_size.

[masterzen]

BTW @pulse00, what you are seeing is issue #36.