Move benchmarks folder outside of flow

[eugenevinitsky]

It's sort of confusing that examples is outside the flow folder but benchmarks is inside? Maybe we move both inside or both outside? @AboudyKreidieh

[AboudyKreidieh]

we can't move the benchmarks outside the flow main folder because we can then no longer call them by saying

from flow.benchmarks import ...

I don't know what the examples folder, however. Any idea what standard software practices on this are?

[eugenevinitsky]

Mmmm, I see. I'm not sure; it's just a matter of making sure the benchmarks are very visible.

[cathywu]

Perhaps we can reference the OpenAI gym structure for this.

[AboudyKreidieh]

I just checked. OpenAI gym places there examples in a folder outside of gym, and their environments are located in gym/envs. If we wanted to follow their structure, I don't think we would have to change anything.

[cathywu]

Thanks for checking, Aboudy! Eugene, what do you think? One perspective: Only "devs" will want to change the benchmarks, and they will know to look inside the flow directory. Other users who are just using the benchmarks will likely benefit from the intuitive "flow.benchmarks" import statement.

[AboudyKreidieh]

@eugenevinitsky what are your thoughts on this? Personally, I'm happy sticking with our current setup

[AboudyKreidieh]

This issue is stale, so closing. I think we're happy with things as is for now. If anyone disagrees, they can reopen it.