Currently the mean over the $R^2$ on validation split of folds of the CV is used to pick the best hyperparams. However, a more robust score might be preferable such as: mean - one standard error (e.g. Elements of Statistical Learning, pg. 244, "one-standard error" rule).

An implementation is being tested as part of #621.