Skip to content

Conversation

ssteinbach
Copy link
Collaborator

@ssteinbach ssteinbach commented Feb 20, 2021

Closes #884

  • Asserts that the only integer data type that OTIO serializes/deserializes is int64_t
  • ...which means that OTIO serializes into JSON:
    • bool
    • int64_t
    • string
    • IEEE754 double (including NaN, Inf, -Infinity support)
    • list
    • dictionary
    • things that derive from SerializableObject
    • RationalTime, TimeTransform and TimeRange
  • An exception is raised if an invalid type or value is inserted into PyAny:
    • TypeError: A value of type '<type 'set'>' is incompatible with OpenTimelineIO. OpenTimelineIO only supports the following value types in AnyDictionary containers (like the .metadata dictionary): ('int', 'float', 'str', 'bool', 'list', 'dictionary', 'opentime.RationalTime', 'opentime.TimeRange', 'opentime.TimeTransform').
    • ValueError: A value of 36893488147419103232 is outside of the range of integers that OpenTimelineIO supports, [-9223372036854775808, 9223372036854775807], which is the range of C++ int64_t.
  • Adds documentation about this behavior and a unit test that hopefully makes this behavior explicit
  • Adds an explicit test for the C++ SDK to the python unit test suite (just one function, but its a start)

TODO:

  • make the exception that is raised when an invalid type is inserted more readable, and have it explain what the invalid types are
  • update the documentation to reflect where we settled on for this behavior

@ssteinbach ssteinbach added this to the Public Beta 14 milestone Feb 20, 2021
@ssteinbach
Copy link
Collaborator Author

I'm hoping to not do that in this PR... I think that entails a bunch of other small changes to true the whole class up to int64_t, and can be done in a later PR

@reinecke
Copy link
Collaborator

Ahh, gotcha. Sounds good.

- to remove integer ambiguity around 32/64bit integers, we coerce all
  ints to int64_t.
- When a uint64_t is serialized, it is bitwise & with INT64_MAX first to
  avoid overflowing the integer.  In order to do that comparison at
  serialization time, the uint64_t needs to be stored as a uint64_t all
  the way through the python bindings into the any mapping.
- the next step is to move the & operation up into the python bindings
  and prevent people from storing invalid numbers in OTIO files.
 Please enter the commit message for your changes. Lines starting
- this is to allow parity with the values in python files
- add support in the serializer/deserializer for NaN, Inf, -Infinity etc
  from rapidjson
- block pybind11 from automatically casting numbers that don't fit into
  double or int64_t into bool
- add a unit test and test data to test the serialization and behavior
@ssteinbach ssteinbach added the bug A problem, flaw, or broken functionality. label Mar 2, 2021
- Adds specific messages that test if something is an out of range
  integer, or an invalid type
- includes a list of supported types
- The 'Writer' part of SerializableObject needed the same tightening as
  the RapidJSON dispatcher
- this is used for things that serialize and compare JSON (like cloning
  and the `is_equivalent_to` method.
- Because the ImageSequenceReference schema has `int` in it (which
  should at some point be changed to int64_t), it was tripping this
  check on certain platforms
Copy link
Collaborator

@reinecke reinecke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great clarification!

@codecov-io
Copy link

Codecov Report

Merging #892 (5cac58f) into master (5676338) will decrease coverage by 0.04%.
The diff coverage is 80.40%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #892      +/-   ##
==========================================
- Coverage   86.15%   86.10%   -0.05%     
==========================================
  Files         183      184       +1     
  Lines       17719    17850     +131     
  Branches     1970     1982      +12     
==========================================
+ Hits        15265    15369     +104     
- Misses       1956     1977      +21     
- Partials      498      504       +6     
Flag Coverage Δ
unittests 86.10% <80.40%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/opentimelineio/safely_typed_any.cpp 80.76% <0.00%> (-15.07%) ⬇️
src/opentimelineio/serializableObject.h 90.38% <ø> (+0.38%) ⬆️
src/opentimelineio/serialization.cpp 82.31% <20.00%> (-1.83%) ⬇️
...ntimelineio/opentimelineio-bindings/otio_utils.cpp 82.55% <25.00%> (-4.03%) ⬇️
tests/test_image_sequence_reference.py 97.59% <63.63%> (-2.41%) ⬇️
...-opentimelineio/opentimelineio/core/_core_utils.py 78.48% <71.42%> (-0.60%) ⬇️
tests/test_cxx_sdk_bindings.py 71.42% <71.42%> (ø)
tests/test_timeline.py 93.93% <87.50%> (-4.09%) ⬇️
src/opentimelineio/deserialization.cpp 60.48% <100.00%> (+1.12%) ⬆️
src/opentimelineio/imageSequenceReference.cpp 94.18% <100.00%> (+0.67%) ⬆️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5676338...5cac58f. Read the comment docs.

@meshula meshula merged commit 8a7bd25 into AcademySoftwareFoundation:master Mar 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A problem, flaw, or broken functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Large positive integers in OTIO JSON file misinterpreted as negative
5 participants