json: bytes remaining in Tokenizer

[chriso]

There's no way to determine how far into the input buffer a json.Tokenizer is.

This is useful information to have. For example, you might want to measure progress as you walk through a large JSON object. You might want to mark positions in the buffer as you traverse nested objects/arrays, so that you can slice a json.RawMessage from the buffer and defer parsing until later.

I originally wanted a Tell() int or Pos() int function that returned the current position into the input buffer, but then realized that the tokenizer progressively consumes its buffer and this information is lost. Rather than track this information and increase the size of the tokenizer, I opted to create a Remaining() int function which returns the number of bytes still to consume. The position into the buffer can be derived by the caller, which presumably still has access to the whole input buffer, by taking len(buf) - tokenizer.Remaining().