Large InputStreams cause negative index fault

[ahantke]

The code attempts to increase the size of the buffer with a 2x multiplier until no further bytes are available. I have some concerns about this (but no concrete solution) because this in-memory approach is not a true stream. But, that aside, the following code within keepSkippedBytesThenRead(...) causes a problem:

byte[] newBuf = new byte[iter.buf.length * 2];

This leads to rapid increases in memory allocation for the buffer. When processing a large input you can easily roll to a negative integer and then you will obviously end up with an allocation error as the length is negative. A better approach, which I'll check in shortly, is to only increase the length of the byte array by the increment of the initially provided size in the JsonIter.parse(...). A more meaningful error than the negative index is also advised.

[taowen]

please provide some sample code to re-produce the problem. Large has many meanings, it is large array or large string?

[taowen]

keepSkippedBytesThenRead is only used when readAny which means skip and copy the bytes. It is not involved in normal encoding process. Keep bytes in memory is expected in this case.