but it can even end up with incomplete and missed data
I've been asking a few others about this, and I've been wondering if it's not a potential issue with the websocket implementation being used, as I had described in this github issue:
https://github.com/zaphoyd/websocketpp/issues/641#issuecomment-329650068
I believe this is also related to an overloaded server (at times of especially "high response latency"), but here's the gist of it:
All frames returned are in json format and no compression is being used (I believe there were issues when they tried enabling compression). Every so often, I get what appears to be a "complete" websocket frame, however, the json packet is incomplete (ie. the frame size appears to be smaller than the actual packet size). The rest of the packet still comes through (but not as another frame). After appending the remainder to the original truncated packet, all seems well again.
I have a feeling if STEEMIT and other graphene-related chains transitioned to use https://github.com/uNetworking/uWebSockets instead, it might help alleviate at least some of these issues.
Great finding! 👍
I have never had that error thrown that I can remember. But I also only tested a short while before I moved away from the default rpc nodes.
There could be many things playing a role here, but for sure, using the stream_comments function there is an enormous overhead initiating a post object for each comment.