From 29b2a2b700b18c3e28287f3369775f4c5f5a27fb Mon Sep 17 00:00:00 2001 From: ssteinbach Date: Wed, 28 Sep 2022 22:45:39 -0700 Subject: [PATCH 1/9] add developing a new schema doc --- docs/index.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/index.rst b/docs/index.rst index 2a70981578..fe6e8e64f0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -47,6 +47,7 @@ Tutorials tutorials/write-a-hookscript tutorials/write-a-schemadef tutorials/spatial-coordinates + tutorials/developing-a-new-schema tutorials/versioning-schemas Use Cases From 744ea8d8fe68c3823eb57c1be70a32e79543e787 Mon Sep 17 00:00:00 2001 From: ssteinbach Date: Wed, 28 Sep 2022 23:45:51 -0700 Subject: [PATCH 2/9] add doc this time --- docs/tutorials/developing-a-new-schema.md | 126 ++++++++++++++++++++++ 1 file changed, 126 insertions(+) create mode 100644 docs/tutorials/developing-a-new-schema.md diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md new file mode 100644 index 0000000000..c2c9b690c5 --- /dev/null +++ b/docs/tutorials/developing-a-new-schema.md @@ -0,0 +1,126 @@ +# Schema Proposal and Development Workflow + +## Introduction + +This document describes a process for proposing and developing a new schema for the +[OpenTimelineIO project](https://opentimeline.io). + +The process includes several steps: + +* Proposing at a TSC meeting and gathering interested parties for feedback + * Outlining example JSON +* Implementing and iterating on a branch +* Building support into an adapter as a demonstration +* Incrementing other schemas that are impacted (Eg. Changes to `Clip` to + implement `Media Multi Reference` + +## Examples + +There have been at this point a number of schemas introduced well into +OpenTimelineIO's development. These include: + +* [ImageSequenceReference](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/602) +* [SpatialCoordinates](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/1219) +* [Multi media-reference](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/1241) + +## Core schema or Plugin? + +A reminder that OpenTimelineIO has a number of plugin mechanisms, including the +[Schemadef](write-a-schemadef). Plugin schemadefs are great for things that +won't be broadly useful to folks or are specific to a particular studio, +workflow, or practice. Example of this might be a reference to a proprietary +database or a proprietary effect. They can also be a good place to prototype a +particular schema before proposing it to the community for adoption. + +## Proposal + +A proposal can be as fleshed out as a proposed implementation or as vague as an +idea. Presenting the proposal at a Technical Steering Committee for discussion +is preferred so that interested parties can form a working group if necessary. +The goal is to find view points / impacts that might not have been considered +and advertise the development to the community at large. + +Including an example JSON blob which has the fields you think might be needed +can help. + +References that are particularly helpful are examples from existing +applications/formats, information about how (or if) the schema participates in +temporal transformations, or other citations. + +## Implementing and Iterating on a branch + +Development of schemas typically takes longer and includes more feedback and +review than normal development. To facilitate this, generally the project will +open a branch on the repository so that pull requests can be merged into the +prototype without disturbing the main branch. For example, the +[ImageSequenceReference](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/602) +branch. + +A complete implementation should have a: + +* C++ core implementation in src/opentimelineio +* python binding in src/py-opentimelineio +* unit tests + +### Unit Tests + +Unit Tests should include a C++ test for the C++ component, a python test for +the python binding, and a baseline test. + +#### C++ test + +The C++ test should directly test the C++ interface. For examples of that, see +`tests/*.cpp`. + +#### Python tests + +The Python test should test the python binding, including any extra ergonomic +conveniences unique to the python implementation (iterators, etc). We use the +`unittest` python library. For examples of this, see: `tests/test_*.py`. + +#### Baseline tests + +Baseline tests are written in python and are intended to test the serializer. + +They include: + +* a file named `tests/baselines/empty_.json`, which is the result + of calling the constructor and then immediately serializing the object: + +```python +ys = YourSchema() +otio.adapters.write_to_file(ys, "empty_your_schema.json", adapter="otio_json") +``` + +* a test in `tests/test_json_backend.py` of the form: + +```python +class TestJsonFormat(unittest.TestCase, otio_test_utils.OTIOAssertions): + ... + def test_your_schema(self): + ys = YourSchema() + self.check_against_baseline(ys, "empty_your_schema") + ... +``` + +## Demo Adapter + +Providing an adapter that supports the schema can show how the schema is +translated into another format. For example, the +[ImageSequenceReference](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/722) +used the RV adapter to demonstrate how it could be used by an adapter. + +## Incrementing Other Schemas + +Depending on where the schema fits into the overall structure, other schemas +might need to be incremented or changed. For example, the Media +multi-reference caused the clip schema to increment. Considering and +implementing this is part of the implementation. Providing up and downgrade +functions ensure backwards and forwards compatibility. + +## Conclusion + +OpenTimelineIO is designed to evolve, and through its schema versioning system +hopes to adapt to changes in the world of timelines and time math. We hope +that working with and on OpenTimelineIO can be a positive, enriching experience +for the community. Thanks for being a part of it! From c654ec9660a1d489d5f4bdbc942988f359db607a Mon Sep 17 00:00:00 2001 From: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> Date: Thu, 29 Sep 2022 11:09:18 -0700 Subject: [PATCH 3/9] Update docs/tutorials/developing-a-new-schema.md Co-authored-by: Nick Porcino Signed-off-by: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> --- docs/tutorials/developing-a-new-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md index c2c9b690c5..9990e13b88 100644 --- a/docs/tutorials/developing-a-new-schema.md +++ b/docs/tutorials/developing-a-new-schema.md @@ -16,7 +16,7 @@ The process includes several steps: ## Examples -There have been at this point a number of schemas introduced well into +A number of schemas have been proposed and introduced during OpenTimelineIO's development. These include: * [ImageSequenceReference](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/602) From 57b7efe7035fe5b90b44cc6964019dbe108affde Mon Sep 17 00:00:00 2001 From: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> Date: Thu, 29 Sep 2022 11:09:26 -0700 Subject: [PATCH 4/9] Update docs/tutorials/developing-a-new-schema.md Co-authored-by: Nick Porcino Signed-off-by: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> --- docs/tutorials/developing-a-new-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md index 9990e13b88..f4ef52ecf0 100644 --- a/docs/tutorials/developing-a-new-schema.md +++ b/docs/tutorials/developing-a-new-schema.md @@ -25,7 +25,7 @@ OpenTimelineIO's development. These include: ## Core schema or Plugin? -A reminder that OpenTimelineIO has a number of plugin mechanisms, including the +OpenTimelineIO has a number of plugin mechanisms, including the [Schemadef](write-a-schemadef). Plugin schemadefs are great for things that won't be broadly useful to folks or are specific to a particular studio, workflow, or practice. Example of this might be a reference to a proprietary From 729ceecd1bce05dbdd018b0390404e6168223f28 Mon Sep 17 00:00:00 2001 From: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> Date: Thu, 29 Sep 2022 11:09:35 -0700 Subject: [PATCH 5/9] Update docs/tutorials/developing-a-new-schema.md Co-authored-by: Nick Porcino Signed-off-by: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> --- docs/tutorials/developing-a-new-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md index f4ef52ecf0..579964d47c 100644 --- a/docs/tutorials/developing-a-new-schema.md +++ b/docs/tutorials/developing-a-new-schema.md @@ -27,7 +27,7 @@ OpenTimelineIO's development. These include: OpenTimelineIO has a number of plugin mechanisms, including the [Schemadef](write-a-schemadef). Plugin schemadefs are great for things that -won't be broadly useful to folks or are specific to a particular studio, +aren't expected to be useful to the broader community, or are specific to a particular studio, workflow, or practice. Example of this might be a reference to a proprietary database or a proprietary effect. They can also be a good place to prototype a particular schema before proposing it to the community for adoption. From 5b8a85b320d6096d7a82198adf22b8a1a0291f09 Mon Sep 17 00:00:00 2001 From: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> Date: Thu, 29 Sep 2022 11:09:47 -0700 Subject: [PATCH 6/9] Update docs/tutorials/developing-a-new-schema.md Co-authored-by: Nick Porcino Signed-off-by: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> --- docs/tutorials/developing-a-new-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md index 579964d47c..89b3b6f501 100644 --- a/docs/tutorials/developing-a-new-schema.md +++ b/docs/tutorials/developing-a-new-schema.md @@ -34,7 +34,7 @@ particular schema before proposing it to the community for adoption. ## Proposal -A proposal can be as fleshed out as a proposed implementation or as vague as an +A proposal can be as fleshed out as a proposed implementation, or as vague as an idea. Presenting the proposal at a Technical Steering Committee for discussion is preferred so that interested parties can form a working group if necessary. The goal is to find view points / impacts that might not have been considered From 3e8e2f9b32e984c6dfb4b7a2d8849172f1351a57 Mon Sep 17 00:00:00 2001 From: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> Date: Thu, 29 Sep 2022 11:09:56 -0700 Subject: [PATCH 7/9] Update docs/tutorials/developing-a-new-schema.md Co-authored-by: Nick Porcino Signed-off-by: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> --- docs/tutorials/developing-a-new-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md index 89b3b6f501..32ee3e2b8f 100644 --- a/docs/tutorials/developing-a-new-schema.md +++ b/docs/tutorials/developing-a-new-schema.md @@ -37,7 +37,7 @@ particular schema before proposing it to the community for adoption. A proposal can be as fleshed out as a proposed implementation, or as vague as an idea. Presenting the proposal at a Technical Steering Committee for discussion is preferred so that interested parties can form a working group if necessary. -The goal is to find view points / impacts that might not have been considered +The goal of a TSC presentation would be to find view points / impacts that might not have been considered and advertise the development to the community at large. Including an example JSON blob which has the fields you think might be needed From 6f581796f0d086d0aeffb46248aba67d49a60b66 Mon Sep 17 00:00:00 2001 From: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> Date: Thu, 29 Sep 2022 11:10:23 -0700 Subject: [PATCH 8/9] Apply suggestions from code review Co-authored-by: Nick Porcino Signed-off-by: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> --- docs/tutorials/developing-a-new-schema.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md index 32ee3e2b8f..433c073df7 100644 --- a/docs/tutorials/developing-a-new-schema.md +++ b/docs/tutorials/developing-a-new-schema.md @@ -45,7 +45,7 @@ can help. References that are particularly helpful are examples from existing applications/formats, information about how (or if) the schema participates in -temporal transformations, or other citations. +temporal transformations, and other relevant citations. ## Implementing and Iterating on a branch @@ -54,7 +54,7 @@ review than normal development. To facilitate this, generally the project will open a branch on the repository so that pull requests can be merged into the prototype without disturbing the main branch. For example, the [ImageSequenceReference](https://github.com/AcademySoftwareFoundation/OpenTimelineIO/pull/602) -branch. +branch demonstrates that workflow. A complete implementation should have a: From 6b4ca9f32ea353e8232851d927822cd29e86a23d Mon Sep 17 00:00:00 2001 From: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> Date: Thu, 29 Sep 2022 11:10:39 -0700 Subject: [PATCH 9/9] Apply suggestions from code review Co-authored-by: Nick Porcino Signed-off-by: Stephan Steinbach <61017+ssteinbach@users.noreply.github.com> --- docs/tutorials/developing-a-new-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tutorials/developing-a-new-schema.md b/docs/tutorials/developing-a-new-schema.md index 433c073df7..7aa5c23b2d 100644 --- a/docs/tutorials/developing-a-new-schema.md +++ b/docs/tutorials/developing-a-new-schema.md @@ -40,7 +40,7 @@ is preferred so that interested parties can form a working group if necessary. The goal of a TSC presentation would be to find view points / impacts that might not have been considered and advertise the development to the community at large. -Including an example JSON blob which has the fields you think might be needed +Including an example JSON excerpt which has the fields you think might be needed can help. References that are particularly helpful are examples from existing