2776: View Diff should be ignored invalid characters.

icu***@gmai***** (Google Code) (Is this you? Claim this profile.)
david
david
Nov. 18, 2012
What version are you running?
1.6.13

What's the URL of the page containing the problem?


What steps will reproduce the problem?
1. Upload diff file with invalid characters.
2. View Diff
3.

What is the expected output? What do you see instead?
View Diff should be ignored invalid characters.
But, Diff currently unavailable.
Error: Diff content couldn't be converted to UTF-8 using the following encodings: euc-kr

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/ReviewBoard-1.7beta2-py2.7.egg/reviewboard/diffviewer/views.py", line 370, in view_diff_fragment
    file = get_requested_diff_file()
  File "/usr/local/lib/python2.7/dist-packages/ReviewBoard-1.7beta2-py2.7.egg/reviewboard/diffviewer/views.py", line 296, in get_requested_diff_file
    populate_diff_chunks(files, highlighting)
  File "/usr/local/lib/python2.7/dist-packages/ReviewBoard-1.7beta2-py2.7.egg/reviewboard/diffviewer/diffutils.py", line 1191, in populate_diff_chunks
    large_data=True)
  File "/usr/local/lib/python2.7/dist-packages/Djblets-0.7.2-py2.7.egg/djblets/util/misc.py", line 156, in cache_memoize
    data = lookup_callable()
  File "/usr/local/lib/python2.7/dist-packages/ReviewBoard-1.7beta2-py2.7.egg/reviewboard/diffviewer/diffutils.py", line 1190, in <lambda>
    enable_syntax_highlighting)),
  File "/usr/local/lib/python2.7/dist-packages/ReviewBoard-1.7beta2-py2.7.egg/reviewboard/diffviewer/diffutils.py", line 604, in get_chunks
    old = convert_to_utf8(old, encoding)
  File "/usr/local/lib/python2.7/dist-packages/ReviewBoard-1.7beta2-py2.7.egg/reviewboard/diffviewer/diffutils.py", line 333, in convert_to_utf8
    "using the following encodings: %s") % enc)
Exception: Diff content couldn't be converted to UTF-8 using the following encodings: euc-kr


What operating system are you using? What browser?


Please provide any additional information below.
diff --git a/reviewboard/diffviewer/diffutils.py b/reviewboard/diffviewer/diffutils.py
index f94aead..82dfd36 100644
--- a/reviewboard/diffviewer/diffutils.py
+++ b/reviewboard/diffviewer/diffutils.py
@@ -325,7 +325,7 @@ def convert_to_utf8(s, enc):
         except UnicodeError:
             for e in enc.split(','):
                 try:
-                    u = unicode(s, e)
+                    u = unicode(s, e, errors='ignore')
                     return u.encode('utf-8')
                 except UnicodeError:
                     pass
david
#1 david
  • +PendingReview
  • +david
david
#2 david
Fixed in release-1.6.x (dcb4710). Thanks!
  • -PendingReview
    +Fixed