time-varying data support in OpenTimelineIO (originally: "Implement Curve Schema")

[sobotka]

Implement curve schema for Bezier, linear, and others?

See Appendix C of the FCPXML document, 2004 as an example implementation.

[ssteinbach]

Huh, good link. This is something we might potentially want to add to OTIO, but is not on our current internal (pixar) roadmap. If someone wanted to do this work, however, we would love to chat with them with some thoughts about how to approach this. Thanks for linking that! Do you know how closely this matches the curve format found in AAF or in EDL (for non linear speed ramps)?

[ssteinbach]

Also see: #39 (comment)

[ssteinbach]

There is also this library: https://github.com/Autodesk/animx

[markreidvfx]

In pyaaf2 here is the code that does curve interpolation for AAF, including Bezier and Cubic
https://github.com/markreidvfx/pyaaf2/blob/master/aaf2/misc.py#L368

[sobotka]

If someone where to try their hand at this, is there a good skeleton to follow as an outline that fits in well with OTIO’s development patterns?

[ssteinbach]

Hi @sobotka, depends on how you want to extend the library. Couple options:

1. its a bit of custom metadata that you inject via an optional plugin. You can do this via the SchemaDef system:
   https://opentimelineio.readthedocs.io/en/latest/tutorials/write-a-schemadef.html
2. Its implemented as metadata to start, and when the schema solidified gets elevated into its own bit of schema. An example of this is CDL data:
   https://github.com/PixarAnimationStudios/OpenTimelineIO/pull/143/files
3. Its its own class right from the start. For an example of that, I would look at the pr which added time effects:
   https://github.com/PixarAnimationStudios/OpenTimelineIO/pull/143/files

Of these options, I think I would prefer #1 or #2. Is there a clear schema from curves that you want to use? Is this a case where we could inject a blob of USDA to encode the curve?

[ssteinbach]

I should note, if you need an example of the USD curve schema:

#usda 1.0
(
    endFrame = 1
    startFrame = 1
)

def Scope "World" (
    customData = {
        bool zUp = 0
    }
)
{

    def BasisCurves "curve_example"
    {
        uniform token basis = "bezier"
        int[] curveVertexCounts = [149]
        Vec3f[] extent = [(-8.431681632995605, 13.877711296081543, -7.41163969039917), (-4.271599769592285, 18.288991928100586, -6.473941326141357)]
        uniform token orientation = "rightHanded"
        PointFloat[] points = [(-5.798529624938965, 18.316255569458008, -7.044877052307129), (-5.764450550079346, 18.288991928100586, -7.077324390411377), (-5.712676048278809, 18.242820739746094, -7.110983848571777), (-5.652022838592529, 18.194751739501953, -7.147763252258301), (-5.58786678314209, 18.140182495117188, -7.163681507110596), (-5.516987323760986, 18.062509536743164, -7.191572666168213), (-5.428699493408203, 17.98286247253418, -7.216586112976074), (-5.350419044494629, 17.905115127563477, -7.226143836975098), (-5.259804725646973, 17.820283889770508, -7.232003688812256), (-5.166261196136475, 17.731197357177734, -7.233789443969727), (-5.098991394042969, 17.65022087097168, -7.224778652191162), (-5.015336990356445, 17.56589126586914, -7.217911720275879), (-4.937463760375977, 17.4731388092041, -7.209527015686035), (-4.8633012771606445, 17.371631622314453, -7.194432258605957), (-4.786374092102051, 17.266658782958984, -7.174495697021484), (-4.710364818572998, 17.157276153564453, -7.157707691192627), (-4.648616313934326, 17.038482666015625, -7.14022159576416), (-4.583930492401123, 16.913177490234375, -7.114993095397949), (-4.527316570281982, 16.780765533447266, -7.089240074157715), (-4.477229595184326, 16.644676208496094, -7.070681571960449), (-4.431090354919434, 16.50658416748047, -7.044270038604736), (-4.394740104675293, 16.363908767700195, -7.009704113006592), (-4.360725402832031, 16.219980239868164, -6.97139835357666), (-4.327887535095215, 16.073923110961914, -6.933936595916748), (-4.301976203918457, 15.927762985229492, -6.897108554840088), (-4.282479286193848, 15.77198600769043, -6.862356185913086), (-4.271599769592285, 15.619695663452148, -6.8242645263671875), (-4.275524616241455, 15.469982147216797, -6.784649848937988), (-4.275715351104736, 15.3218994140625, -6.751595497131348), (-4.288318157196045, 15.179079055786133, -6.713292121887207), (-4.3137922286987305, 15.043816566467285, -6.6717376708984375), (-4.338052272796631, 14.91185188293457, -6.641101837158203), (-4.368602752685547, 14.79130744934082, -6.605554103851318), (-4.405693054199219, 14.671232223510742, -6.5683674812316895), (-4.448169708251953, 14.55950927734375, -6.549091815948486), (-4.498363018035889, 14.459219932556152, -6.526265621185303), (-4.564486503601074, 14.36806869506836, -6.501640796661377), (-4.6218438148498535, 14.276843070983887, -6.492730140686035), (-4.691420555114746, 14.207422256469727, -6.487419128417969), (-4.764915466308594, 14.141447067260742, -6.473941326141357), (-4.843922138214111, 14.108104705810547, -6.49385929107666), (-4.917010307312012, 14.066675186157227, -6.4994611740112305), (-5.001415252685547, 14.049372673034668, -6.510268211364746), (-5.079850673675537, 14.038363456726074, -6.5438232421875), (-5.160880088806152, 14.039912223815918, -6.569972038269043), (-5.24770450592041, 14.040372848510742, -6.590716361999512), (-5.331051826477051, 14.057589530944824, -6.632673740386963), (-5.417025566101074, 14.100809097290039, -6.673892974853516), (-5.505289554595947, 14.131881713867188, -6.70423698425293), (-5.5867462158203125, 14.176652908325195, -6.752299785614014), (-5.669767379760742, 14.232455253601074, -6.7941813468933105), (-5.750812530517578, 14.287765502929688, -6.8327178955078125), (-5.8219499588012695, 14.356086730957031, -6.8774518966674805), (-5.891955375671387, 14.434263229370117, -6.915879726409912), (-5.963547229766846, 14.511224746704102, -6.953146934509277), (-6.026339530944824, 14.599044799804688, -6.99616813659668), (-6.081738471984863, 14.699056625366211, -7.028326988220215), (-6.139471054077148, 14.791899681091309, -7.061502933502197), (-6.19234561920166, 14.888758659362793, -7.091449737548828), (-6.233818054199219, 14.997491836547852, -7.121301174163818), (-6.266703128814697, 15.107736587524414, -7.140946388244629), (-6.29489803314209, 15.222074508666992, -7.161044120788574), (-6.3153605461120605, 15.33570384979248, -7.171276092529297), (-6.332770347595215, 15.444573402404785, -7.177068710327148), (-6.34885835647583, 15.548019409179688, -7.190596580505371), (-6.353478908538818, 15.645086288452148, -7.1864471435546875), (-6.3533525466918945, 15.730472564697266, -7.177324295043945), (-6.353339672088623, 15.807703971862793, -7.169253826141357), (-6.345412731170654, 15.869796752929688, -7.155083656311035), (-6.330376148223877, 15.919514656066895, -7.1399078369140625), (-6.315944194793701, 15.962608337402344, -7.12258243560791), (-6.289356231689453, 15.99725341796875, -7.099160671234131), (-6.248839378356934, 16.020732879638672, -7.077181339263916), (-6.215677261352539, 16.047447204589844, -7.051718235015869), (-6.162703514099121, 16.05813217163086, -7.018344402313232), (-6.099419116973877, 16.052967071533203, -6.989802360534668), (-6.045535087585449, 16.040786743164062, -6.9509196281433105), (-5.9889631271362305, 16.009767532348633, -6.912732124328613), (-5.921966552734375, 15.975187301635742, -6.880302429199219), (-5.86612606048584, 15.92074203491211, -6.847167015075684), (-5.803770542144775, 15.861483573913574, -6.80978536605835), (-5.7387237548828125, 15.765299797058105, -6.777604103088379), (-5.691912651062012, 15.66439151763916, -6.753253936767578), (-5.644923210144043, 15.547589302062988, -6.72102689743042), (-5.599220275878906, 15.446615219116211, -6.692538738250732), (-5.56266975402832, 15.345821380615234, -6.658262252807617), (-5.538607597351074, 15.237594604492188, -6.631525039672852), (-5.527679920196533, 15.123348236083984, -6.60994815826416), (-5.521845817565918, 15.003228187561035, -6.588702201843262), (-5.530472755432129, 14.884973526000977, -6.576107025146484), (-5.548778057098389, 14.769630432128906, -6.563443183898926), (-5.5670318603515625, 14.655667304992676, -6.552760601043701), (-5.588052749633789, 14.546268463134766, -6.544062614440918), (-5.625372886657715, 14.4450101852417, -6.533968448638916), (-5.655047416687012, 14.342473983764648, -6.5308332443237305), (-5.708406448364258, 14.257610321044922, -6.538503170013428), (-5.761215686798096, 14.171981811523438, -6.539337158203125), (-5.823328971862793, 14.09787368774414, -6.554062366485596), (-5.895819187164307, 14.036781311035156, -6.573716163635254), (-5.971530914306641, 13.975406646728516, -6.586936950683594), (-6.051448822021484, 13.928291320800781, -6.6139421463012695), (-6.1302571296691895, 13.898723602294922, -6.638946056365967), (-6.21148681640625, 13.885042190551758, -6.659483909606934), (-6.284393310546875, 13.878507614135742, -6.692380905151367), (-6.3613128662109375, 13.877711296081543, -6.724212169647217), (-6.453092575073242, 13.901104927062988, -6.761641025543213), (-6.548768997192383, 13.9378080368042, -6.815985679626465), (-6.6557159423828125, 13.999044418334961, -6.8743391036987305), (-6.771184921264648, 14.078888893127441, -6.930717468261719), (-6.87617301940918, 14.154025077819824, -6.993041038513184), (-6.985936164855957, 14.264006614685059, -7.054795742034912), (-7.094746112823486, 14.381155967712402, -7.103140354156494), (-7.198413848876953, 14.491877555847168, -7.159719467163086), (-7.29835319519043, 14.63565444946289, -7.205453395843506), (-7.399726390838623, 14.763840675354004, -7.247619152069092), (-7.4946112632751465, 14.908594131469727, -7.288734436035156), (-7.588034629821777, 15.062997817993164, -7.32442569732666), (-7.683770656585693, 15.21764850616455, -7.358137130737305), (-7.777655601501465, 15.387819290161133, -7.37272834777832), (-7.865678787231445, 15.566238403320312, -7.390521049499512), (-7.9474711418151855, 15.739744186401367, -7.41163969039917), (-8.023111343383789, 15.924019813537598, -7.4112677574157715), (-8.085203170776367, 16.11362075805664, -7.398886680603027), (-8.14303207397461, 16.290241241455078, -7.395522117614746), (-8.197183609008789, 16.472640991210938, -7.380401611328125), (-8.24987506866455, 16.646629333496094, -7.362552642822266), (-8.282423973083496, 16.81459617614746, -7.337915897369385), (-8.31879997253418, 16.978260040283203, -7.300570011138916), (-8.356868743896484, 17.13711166381836, -7.268261909484863), (-8.377009391784668, 17.278955459594727, -7.236685752868652), (-8.40914535522461, 17.423961639404297, -7.200990200042725), (-8.418909072875977, 17.534587860107422, -7.15449333190918), (-8.426562309265137, 17.650711059570312, -7.120608806610107), (-8.431681632995605, 17.74968910217285, -7.086295127868652), (-8.422812461853027, 17.83391571044922, -7.051823616027832), (-8.413572311401367, 17.91779899597168, -7.024791717529297), (-8.401973724365234, 17.97516632080078, -6.997857093811035), (-8.392462730407715, 18.038198471069336, -6.968812465667725), (-8.378514289855957, 18.0794677734375, -6.95194149017334), (-8.372298240661621, 18.126873016357422, -6.932058334350586), (-8.357425689697266, 18.154993057250977, -6.911208152770996), (-8.327571868896484, 18.197233200073242, -6.8864946365356445), (-8.294103622436523, 18.205894470214844, -6.865139007568359), (-8.247763633728027, 18.192359924316406, -6.850216865539551), (-8.218963623046875, 18.15266227722168, -6.8443803787231445), (-8.194067001342773, 18.113746643066406, -6.850166320800781), (-8.158916473388672, 18.05900001525879, -6.848714351654053), (-8.134465217590332, 18.0087947845459, -6.837794303894043), (-8.134465217590332, 18.0087947845459, -6.837794303894043)]
        ColorFloat[] primvars:displayColor = [(0.19999998807907104, 0.19999998807907104, 0.9019607901573181)]
        uniform token type = "cubic"
        float[] widths = [0.112499967217]
        custom Matrix4d xformOp:transform = ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 0, 0, 1) )
        uniform token[] xformOpOrder = ["xformOp:transform"]
    }
}

You could put that blob into a string attribute and then in an adapter use the USD API to evaluate values on that curve.

[sobotka]

Is it currently possible to have a unique "in house" schema for curves for OTIO, while at the same time, defining some glue to get in and out of other schemas? For example, the in-houseCurve <-> USDcurve?

[ssteinbach]

I believe so. If you build a schemadef plugin, that is a complete python class, which could have methods like .to_usda_curve() or something.

[jminor]

Of the strategies that @ssteinbach listed, I recommend using the SchemaDef plugin mechanism. That will give you the ability to define the schema, a class that represents it, a versioning mechanism, and whatever methods you want on it. It also gives us flexibility to merge it into the official schema easily later.

[ssteinbach]

Where are you planning on getting the curve data? Is it from one of the existing adapters?