3784: Internal Error Publishing Review due to character encoding

msunde
david
david
Feb. 27, 2015
What version are you running?
2.0.12 with the patch for  bug #3716  manually applied.

What's the URL of the page containing the problem?
http://review.aurea.local/r/931/

What steps will reproduce the problem?
1. Create a new review request.
2. Enter none ASCII characters in the review subject, description, testing: é in calculée or à
3. Publish review, get internal server error.
4. I suspect this in turn causes bug #3760

What is the expected output? What do you see instead?
Review Should be properly published

What operating system are you using? What browser?
Amazon Linux, Google Chrome.

Please provide any additional information below.

The error from the review board log files is:
2015-02-24 17:22:53,478 - DEBUG -  - Cache miss for key jsi18n-djangojs-reviewboard_djblets-en-1421103165.
2015-02-24 17:23:40,182 - ERROR -  - Exception thrown for user msunde at http://review.aurea.local/api/review-requests/931/draft/

'ascii' codec can't decode byte 0xc3 in position 159: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/resources.py", line 496, in __call__
    request, method, view, api_format=api_format, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/resources.py", line 567, in call_method_view
    return view(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/resources.py", line 764, in put
    return self.update(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/webapi/decorators.py", line 110, in _check
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 138, in _checklogin
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 287, in _validate
    return view_func(*args, **new_kwargs)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/webapi/resources/review_request_draft.py", line 481, in update
    review_request.publish(user=request.user)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/reviews/models/review_request.py", line 793, in publish
    changedesc=changes)
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/dispatch/dispatcher.py", line 185, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/notifications/email.py", line 46, in review_request_published_cb
    mail_review_request(review_request, changedesc)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/notifications/email.py", line 328, in mail_review_request
    review_request.summary)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 159: ordinal not in range(128)
#1 msunde
This is making it very difficult for our French users to use reviewboard since they have to manually sanitize the text prior to submitting a review request.
david
#3 david
This is very strange, since everything should be a unicode object at this point. Can you add some debugging to mail_review_request and see what type 'review_request.summary' and 'review_request.display_id' are?
  • +NeedInfo
#4 msunde
I am not familiar with python. Let me know what code to add and the command to recompile. Thanks.

BTW, any chance that there has been fixes in this area for 2.0.13 release? I have not updated yet.
david
#5 david
Add this just above the line that sets subject =

logging.warning('display_id = %s\nsummary = %s', type(review_request.display_id, review_request.summary))

You shouldn't need to do any manual recompilation, but you may need to delete the .pyc file and reload your web server.
#6 msunde
Here is the code that I ran:
    logging.error("display_id = %s\nsummary = %s", type(review_request.display_id), type(review_request.summary))

    subject = "Review Request %d: %s" % (review_request.display_id,
                                         review_request.summary)


Here is the output:
2015-02-24 22:22:11,828 - DEBUG -  - DiffParser.parse: Beginning parse of diff, size = 108772
2015-02-24 22:22:11,845 - DEBUG -  - DiffParser.parse: Finished parsing diff.
2015-02-24 22:23:03,588 - ERROR -  - display_id = <type 'long'>
summary = <type 'str'>
2015-02-24 22:23:03,588 - ERROR -  - Exception thrown for user msunde at http://review.aurea.local/api/review-requests/936/draft/

'ascii' codec can't decode byte 0xc3 in position 101: ordinal not in range(128)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/resources.py", line 496, in __call__
    request, method, view, api_format=api_format, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/resources.py", line 567, in call_method_view
    return view(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/resources.py", line 764, in put
    return self.update(request, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/webapi/decorators.py", line 110, in _check
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 138, in _checklogin
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 117, in _call
    return view_func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/Djblets-0.8.14-py2.6.egg/djblets/webapi/decorators.py", line 287, in _validate
    return view_func(*args, **new_kwargs)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/webapi/resources/review_request_draft.py", line 481, in update
    review_request.publish(user=request.user)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/reviews/models/review_request.py", line 793, in publish
    changedesc=changes)
  File "/usr/lib/python2.6/site-packages/Django-1.6.9-py2.6.egg/django/dispatch/dispatcher.py", line 185, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/notifications/email.py", line 46, in review_request_published_cb
    mail_review_request(review_request, changedesc)
  File "/usr/lib/python2.6/site-packages/ReviewBoard-2.0.12-py2.6.egg/reviewboard/notifications/email.py", line 330, in mail_review_request
    review_request.summary)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 101: ordinal not in range(128)



david
#7 david
Thanks!
  • -NeedInfo
    +PendingReview
  • +Component-EMail
    +Project-ReviewBoard
  • +david
david
#8 david
Fixed in release-2.0.x (71a0423). Thanks!
  • -PendingReview
    +Fixed