Schema Downgrading System (AcademySoftwareFoundation#1387)

* refactor versioning tests into their own class
* DRY cleanup in the serializer before other stuff
* DRY reduction in the json FILE serializer
* Add io_perf_test to repo
* Add a call w/ downgrade manifest to io_perf_test
* add anydictionary convenience functions
* add .cache to gitignore
* add override tags
* add perf tests for no-downgrade scenarios
* autogen version info struct
* add exceptions for overwriting up/downgrade fn
* add exception when double registering a type
* move schema version types into typeRegistry
* add version manifest plugin
* lint pass
* comment formatting for RTD
* add upgrade_downgrade_example in C++
* Add notes to environment variables markdown.
* Improve error handling and text for env var errors

- the OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL has checking to make sure
  the format is correct and that the version/label requested are
  present.
- Adds a custom exception that gets raised if there is a problem
- Adds a unit test for testing this behavior

* Performance tuning (cache the child cloning encoder across the entire serialization, use const & as much as possible)

Signed-off-by: ssteinbach <[email protected]>
Co-authored-by: meshula <[email protected]>
Signed-off-by: Michele Spina <[email protected]>

Author: 2 people

.gitignore

@@ -10,6 +10,7 @@ htmlcov
 .vscode/
 xcuserdata/
 .venv/
+.cache
 
 # Pycharm metadata
 .idea/

Makefile

@@ -182,6 +182,13 @@ doc-plugins:
 doc-plugins-update:
 	@python src/py-opentimelineio/opentimelineio/console/autogen_plugin_documentation.py -o docs/tutorials/otio-plugins.md --public-only --sanitized-paths
 
+# build the CORE_VERSION_MAP cpp file
+version-map:
+	@python src/py-opentimelineio/opentimelineio/console/autogen_version_map.py -i src/opentimelineio/CORE_VERSION_MAP.last.cpp --dryrun
+
+version-map-update:
+	@python src/py-opentimelineio/opentimelineio/console/autogen_version_map.py -i src/opentimelineio/CORE_VERSION_MAP.last.cpp -o src/opentimelineio/CORE_VERSION_MAP.cpp
+
 # generate documentation in html
 doc-html:
 	@# if you just want to build the docs yourself outside of RTD

docs/tutorials/otio-env-variables.md

@@ -5,11 +5,12 @@ various aspects of OTIO.
 
 ## Plugin Configuration
 
-These variables must be set _before_ the OpenTimelineIO python library is imported.
+These variables must be set _before_ the OpenTimelineIO python library is imported.  They only impact the python library.  The C++ library has no environment variables.
 
 - `OTIO_PLUGIN_MANIFEST_PATH`: a ":" separated string with paths to .manifest.json files that contain OTIO plugin manifests.  See: [Tutorial on how to write an adapter plugin](write-an-adapter).
 - `OTIO_DEFAULT_MEDIA_LINKER`: the name of the default media linker to use after reading a file, if "" then no media linker is automatically invoked.
 - `OTIO_DISABLE_PKG_RESOURCE_PLUGINS`: By default, OTIO will use the pkg_resource entry_points mechanism to discover plugins that have been installed into the current python environment.  pkg_resources, however, can be slow in certain cases, so for users who wish to disable this behavior, this variable can be set to 1.
+- `OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL`: if no downgrade arguments are passed to `write_to_file`/`write_to_string`, use the downgrade manifest specified by the family/label combination in the variable.  Variable is of the form FAMILY:LABEL.  Only one tuple of FAMILY:LABEL may be specified.
 
 ## Unit tests
 

docs/tutorials/otio-plugins.md

@@ -106,7 +106,7 @@ OpenTimelineIO Final Cut Pro 7 XML Adapter.
 ### otio_json
 
 ```
-This adapter lets you read and write native .otio files
+Adapter for reading and writing native .otio json files.
 ```
 
 *source*: `opentimelineio/adapters/otio_json.py`
@@ -147,14 +147,23 @@ Serializes an OpenTimelineIO object into a file
       indent (int): number of spaces for each json indentation level.
   Use -1 for no indentation or newlines.
 
+  If target_schema_versions is None and the environment variable
+  "OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL" is set, will read a map out of
+  that for downgrade target.  The variable should be of the form
+  FAMILY:LABEL, for example "MYSTUDIO:JUNE2022".
+
   Returns:
       bool: Write success
 
   Raises:
       ValueError: on write error
+      otio.exceptions.InvalidEnvironmentVariableError: if there is a problem
+      with the default environment variable
+      "OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL".
 ```
   - input_otio
   - filepath
+  - target_schema_versions
   - indent
 - write_to_string: 
 ```
@@ -165,10 +174,21 @@ Serializes an OpenTimelineIO object into a string
       indent (int): number of spaces for each json indentation level. Use
   -1 for no indentation or newlines.
 
+  If target_schema_versions is None and the environment variable
+  "OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL" is set, will read a map out of
+  that for downgrade target.  The variable should be of the form
+  FAMILY:LABEL, for example "MYSTUDIO:JUNE2022".
+
   Returns:
       str: A json serialized string representation
+
+  Raises:
+      otio.exceptions.InvalidEnvironmentVariableError: if there is a problem
+      with the default environment variable
+      "OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL".
 ```
   - input_otio
+  - target_schema_versions
   - indent
 
 

docs/tutorials/otio-serialized-schema-only-fields.md

@@ -121,6 +121,7 @@ parameters:
 - *hooks*
 - *media_linkers*
 - *schemadefs*
+- *version_manifests*
 
 ### SerializableObject.1
 

docs/tutorials/otio-serialized-schema.md

@@ -254,6 +254,7 @@ parameters:
 - *hooks*: Hooks that hooks scripts can be attached to.
 - *media_linkers*: Media Linkers this manifest describes.
 - *schemadefs*: Schemadefs this manifest describes.
+- *version_manifests*: Sets of versions to downgrade schemas to.
 
 ### SerializableObject.1