4042: Reviewboard unable to serialize diffset to JSON for webhooks

rayalan
brennie
 brennie 
What version are you running?


Reviewboard 2.5.2


What's the URL of the page containing the problem?


r/1813/

(when submitting an review change)


What steps will reproduce the problem?


    

  1. Make some changes to my code.
    2. 

  2. rbt post -o -u  
    3. 

  3. Enter a comment about what changed in this update (optional; doesn't really matter)
    4. 

  4. Click 'Publish Changes' to update the review.
    5. 



What is the expected output? What do you see instead?


I expect the following to happen:

1. The review to publish. 

2. Not to be e-mailed an error (because no error occurred)

3. Not to be told that the server failed (because the server actually succeeded)

4. The webhook to be called.


    

  1. A popup box occurs with the text 'HTTP 500 INTERNAL SERVER ERROR'
    2. 

  2. The review does actually update, but you have to reload the page to convince the browser of this.
    3. 

  3. The webhook doesn't trigger. 
    4. 

  4. The server sends out an error log and logs to the server logs the following traceback:
    5. 



- Exception thrown for user <username> at http://reviewboard.ces.cvnt.net/api/review-requests/1813/draft/

<DiffSet: [3675] diff r2> is not JSON serializable
Traceback (most recent call last):
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/django/core/handlers/base.py", line 112, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/django/views/decorators/cache.py", line 52, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/django/views/decorators/vary.py", line 19, in inner_func
    response = func(*args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/resources/base.py", line 195, in __call__
    request, method, view, api_format=api_format, *args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/resources/mixins/api_tokens.py", line 65, in call_method_view
    return view(request, *args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/resources/base.py", line 463, in put
    return self.update(request, *args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/decorators.py", line 120, in _call
    return view_func(*args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/reviewboard/webapi/decorators.py", line 125, in _check
    return view_func(*args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/decorators.py", line 120, in _call
    return view_func(*args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/decorators.py", line 141, in _checklogin
    return view_func(*args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/decorators.py", line 120, in _call
    return view_func(*args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/decorators.py", line 120, in _call
    return view_func(*args, **kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/decorators.py", line 301, in _validate
    return view_func(*args, **new_kwargs)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/reviewboard/webapi/resources/review_request_draft.py", line 493, in update
    review_request.publish(user=request.user, trivial=trivial)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/reviewboard/reviews/models/review_request.py", line 846, in publish
    changedesc=changes)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/django/dispatch/dispatcher.py", line 185, in send
    response = receiver(signal=self, sender=sender, **named)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/reviewboard/notifications/webhooks.py", line 240, in review_request_published_cb
    dispatch_webhook_event(request, webhook_targets, event, payload)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/reviewboard/notifications/webhooks.py", line 109, in dispatch_webhook_event
    body = adapter.encode(payload, request=request)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/encoders.py", line 112, in encode
    return super(JSONEncoderAdapter, self).encode(o)
  File "/usr/lib64/python2.6/json/encoder.py", line 367, in encode
    chunks = list(self.iterencode(o))
  File "/usr/lib64/python2.6/json/encoder.py", line 309, in _iterencode
    for chunk in self._iterencode_dict(o, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 275, in _iterencode_dict
    for chunk in self._iterencode(value, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 306, in _iterencode
    for chunk in self._iterencode_list(o, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 204, in _iterencode_list
    for chunk in self._iterencode(value, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 309, in _iterencode
    for chunk in self._iterencode_dict(o, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 275, in _iterencode_dict
    for chunk in self._iterencode(value, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 309, in _iterencode
    for chunk in self._iterencode_dict(o, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 275, in _iterencode_dict
    for chunk in self._iterencode(value, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 309, in _iterencode
    for chunk in self._iterencode_dict(o, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 275, in _iterencode_dict
    for chunk in self._iterencode(value, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 317, in _iterencode
    for chunk in self._iterencode_default(o, markers):
  File "/usr/lib64/python2.6/json/encoder.py", line 323, in _iterencode_default
    newobj = self.default(o)
  File "/home/reviewboard/rb_venv/lib/python2.6/site-packages/djblets/webapi/encoders.py", line 123, in default
    raise TypeError("%r is not JSON serializable" % (o,))
TypeError: <DiffSet: [3675] diff r2> is not JSON serializable





What operating system are you using? What browser?


Browsing on Firefox 43.0 on OS X 10.10.5. RBTools 0.7.1. Reviewboard running on Scientific Linux 6.3 (Carbon) with mysql 5.1.73-3. 


Other people in the company have encountered this as well (typically using Chrome and RBTools 0.6.x). 


Please provide any additional information below.


    

  • While I like the behavior that the review posting succeeds even if the webhook call fails for some reason, a secondary concern is that the entire publishing step appears to fail even when just the webhook failed.
    • 

  • I've effectively locked up Reviewboard trying to retrieved the diffset from the database via the admin database review. Each time I try that way, one thread seems to go into an infinite loop cycle trying to retrieve the diffset.
    • 

  • It doesn't seem to matter what version of RBTools the initial review or review updates are posted with. 
    • 

  • Given that other people haven't reported this while other webhook issues have gotten reported, I suspect it's an issue with the way the data is getting stored in the database. But I'm not sure quite what I'm investigating if to go down that route.
    •
#1 rayalan 
I dug a bit more into this. If I tap into the webhook and log the payload before it is passed to the encoder, I get this snippet (I prettified it):




   ...
   u'change': (
        {
            u'links': {
                u'self': {
                    u'href': 'http://omitted/api/review-requests/xxxx/changes/yyyy/', u'method': u'GET'
                    }
                }, 
            u'text': u'Debugging Reviewboard', 
            u'timestamp': datetime.datetime(2015, 12, 22, 3, 23, 1, 227916, tzinfo=<UTC>), 
            u'text_type': u'markdown', 
            u'fields_changed': {
                u'diff': {
                    u'added': <DiffSet: [zzzz] diff r7>
                    }
                }, 
            u'id': yyyyL
        },
        ),



Everything looks fine except that fields_changed --> diff --> added --> DiffSet pairing, which almost certainly can't be serialized by djblets.
brennie
#2 brennie
  • +brennie
brennie
#3 brennie 
A fix for this is pending at https://reviews.reviewboard.org/r/7822/
  • -New
    +PendingReview
brennie
#4 brennie 
This has been fixed on the release-2.5.x branch as commit 0b6bb9a. It will be included in the next minor release.


Thanks for reporting this!
  • -PendingReview
    +Fixed