3792: Outlook message (.msg extension) upload fails in 2.0.13

raa****@gmai***** (Google Code) (Is this you? Claim this profile.)
david
david
March 13, 2015
I wanted to know if there is a quick fix is available for this.
If not how do I stop uploading such extension before hand than corrupting posts.
Is there some possibility to block at Apache configuration level?

Please follow the necessary information.

What version are you running?
ReviewBoard: 2.0.13 
mimeparser: 0.1.3
Python: 2.7.8
Apache: 2.4.10

What's the URL of the page containing the problem?
After creating the review post, during file attachment procedure.
File upload, while updating the outlook message as an attachment the RB hangs ~30min and produces the attached error in .png file. Also the admin receives the information in text attachment.


What steps will reproduce the problem?
1. Create a new post

2. Attach a message from Microsoft outlook as it is as .msg extension. (hangs for a long time)
   Then produces the error as in attachment: rb_outlook_attach_error.png

3. Try to open the post, but doesn't open (while other posts are accessible) and produces the default 500 error only for the post that contain the .msg file extension as an attachment.

As user access to the post is broken and receives Error 500, however the admin can still access the corrupted post.

Something broke! (Error 500)

It appears something broke when you tried to go here. This is either a bug in ReviewBoard or a server configuration error. Please report this to your administrator

What is the expected output? What do you see instead?
After attempting to upload the outlook_message.msg, the screen shot is attached in .png file.

What operating system are you using? What browser?
RHEL 6.5 (Santiago)

Please provide any additional information below.

The mimetype property was found in the link: <RB link root>/admin/db/attachments/fileattachment/

Before upgrade the RB version is 1.6.9 and the outlook message (Eg: outlook_message.msg) has the property of mimetype is: application/octet-stream - 1.6.9 was accepting outlook messages with this mimetype. 

After the RB upgrade to 2.0.13 the mimetype is mentioned as "CDF V2 Document, No summary info"

The end result is the post is broken and not accessible, user has to generate the new post and publish without attaching the outlook_message.msg but in a different format.

So far this behavior is observed only with .msg extension.
other extensions like .txt .c .h .rtf .xlsx .htm .jpg and .sh are accepted by new  RB version: 2.0.13

Also giving the text attachment that is received in the email back to admin with error. (trimmed path information).
Traceback (most recent call last):
  File "<install root>/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "<install root>/lib/python2.7/site-packages/ReviewBoard-2.0.13-py2.7.egg/reviewboard/accounts/decorators.py", line 21, in _check
    return login_required(view_func)(*args, **kwargs)
  File "<install root>/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/contrib/auth/decorators.py", line 22, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "<install root>/lib/python2.7/site-packages/ReviewBoard-2.0.13-py2.7.egg/reviewboard/site/decorators.py", line 35, in _check
    return view_func(request, local_site=local_site, *args, **kwargs)
  File "<install root>/lib/python2.7/site-packages/ReviewBoard-2.0.13-py2.7.egg/reviewboard/reviews/views.py", line 717, in review_detail
    RequestContext(requ
#1 raa****@gmai***** (Google Code) (Is this you? Claim this profile.)
I wanted to know if there is a quick fix is available for this.
If not how do I stop uploading such extension before hand than corrupting posts.
Is there some possibility to block at Apache configuration level?

I have tried adding .msg extension against the line "application/octet-stream"  already in httpd.conf and restarted the httpd service.

Even then the problem remains same.

Any suggestion please?

Thanks, Ravi
david
#2 david
Probably your browser is supplying a bad mimetype when sending the file. Would you be able to add some logging to find out what that mimetype is?
  • +NeedInfo
#3 raa****@gmai***** (Google Code) (Is this you? Claim this profile.)
As I have mentioned after analysis of both before and after migration is:

// 
Before upgrade the RB version is 1.6.9 and the outlook message (Eg: outlook_message.msg) has the property of mimetype is: application/octet-stream - 1.6.9 was accepting outlook messages with this mimetype. 

After the RB upgrade to 2.0.13 the mimetype is mentioned as "CDF V2 Document, No summary info"
//

However, How do I trace the information browser-->apache2-->RB information flow?
PS: I don't see any apache error log related to this behavior at the moment of file upload fails.

Please suggest.
#4 raa****@gmai***** (Google Code) (Is this you? Claim this profile.)
This behavior is true even in Chrome version: 40.0.2214.115 m 
Also in Firefox: 35.0.1 & 36.0
IEv8 doesn't even work to load dashboard - IEv8 usage completely ruled out - I guess.

Thanks, Ravi
#5 raa****@gmai***** (Google Code) (Is this you? Claim this profile.)
DEBUG log in RB was enable since the upgrade took place. The same is attached in the text file. Please let me know if more/specific information is needed.
david
#6 david
  • -NeedInfo
    +Started
  • +Project-ReviewBoard
    +Component-Attachments
  • +david
david
#7 david
Fixed in release-2.0.x (79d6e45). Thanks!
  • -Started
    +Fixed
#8 raa****@gmai***** (Google Code) (Is this you? Claim this profile.)
Hello,

Good morning.

I have tested the 2.0.14 release few minutes back.
Thank you for providing the quick fix.

Two observations from the tests:

1) Review is not broken and the same is accessible to users - Thanks for that.
2) I am still unable to attach the message taken out of "Microsoft Outlook" as an attachment 'as is'.

In the past the attachment upload is successful and breaks the post.
But now the attachment is failing to upload but doesn't break the post.

What is needed also is to enable the successful attachment upload without breaking the post.

I am attaching a sample .msg file that I have tried to attach in RB 
Also the dubug trace from the reviewboard log.

may be you can reproduce the same in your environment using 'Test message.msg" file.
If you are unable to reproduce 

Please suggest.

Thanks, Ravi
david
#9 david
  • -Fixed
    +New
#10 raa****@gmai***** (Google Code) (Is this you? Claim this profile.)
Completing the last line: 
If you are unable to reproduce or more information is needed, please let me know.

Browser screenshot of attachment screen, that appears after ~20min of uploading .msg file.
david
#11 david
Fixed the remaining issues in release-2.0.x (a5b5133). Thanks!
  • -New
    +Fixed
#12 nan****@gmai***** (Google Code) (Is this you? Claim this profile.)
Hi David,

I can see _guess_mimetype is implemented using pipe and some chunks of input file

maybe it can help for you information

file --mime-type -b  ~/Downloads/CQ_03176_Test_Record.doc
application/msword

cat  ~/Downloads/CQ_03176_Test_Record.doc |file --mime-type -b -
application/CDFV2-corrupt

looks like our problem to guess mimetype

So maybe using file --mime-type -b path could fix most use case?
#13 nan****@gmai***** (Google Code) (Is this you? Claim this profile.)
https://pypi.python.org/pypi/python-magic/ could be an alternative to "file --mime-type -b" command but another python dependency :-(