oc cluster up fails on latest master

[smarterclayton]

$ oc cluster up
Starting OpenShift using openshift/origin:v3.7.0-alpha.1 ...
-- Checking OpenShift client ... OK
-- Checking Docker client ... OK
-- Checking Docker version ... OK
-- Checking for existing OpenShift container ...
   Deleted existing OpenShift container
-- Checking for openshift/origin:v3.7.0-alpha.1 image ... OK
-- Checking Docker daemon configuration ... OK
-- Checking for available ports ... OK
-- Checking type of volume mount ...
   Using nsenter mounter for OpenShift volumes
-- Creating host directories ... OK
-- Finding server IP ...
   Using 10.1.2.2 as the server IP
-- Starting OpenShift container ...
   Creating initial OpenShift configuration
   Starting OpenShift using container 'origin'
FAIL
   Error: could not start OpenShift container "origin"
   Details:
     No log available from "origin" container

$ docker logs origin
W1021 15:03:53.620263  122349 start_master.go:290] Warning: assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container logs in the console, master start will continue.
W1021 15:03:53.620409  122349 start_master.go:290] Warning: assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in the console, master start will continue.
E1021 15:03:53.634384  122349 controllers.go:116] Server isn't healthy yet. Waiting a little while.
panic: An admission plugin wants the service resolver, but it was not provided.
goroutine 1 [running]:
github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/kubeapiserver/admission.(*PluginInitializer).Initialize(0xc421a7e300, 0xef3a020, 0xc420284fc0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/kubeapiserver/admission/initializer.go:178 +0x418
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission.PluginInitializers.Initialize(0xc421a5cd80, 0x3, 0x3, 0xef3a020, 0xc420284fc0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission/plugins.go:180 +0x55
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission.(*PluginInitializers).Initialize(0xc421a9c020, 0xef3a020, 0xc420284fc0)
	<autogenerated>:1 +0x62
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission.(*Plugins).InitPlugin(0x10e1d070, 0x58e5c56, 0x17, 0xeefd420, 0xc420284c40, 0xef28ca0, 0xc421a9c020, 0x0, 0x0, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/admission/plugins.go:155 +0x20c
github.com/openshift/origin/pkg/cmd/server/origin/admission.newAdmissionChain(0xeeec2e0, 0x24, 0x24, 0x0, 0x0, 0xc421a5cea0, 0x0, 0x0, 0x0, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/admission/chain_builder.go:299 +0x4ae
github.com/openshift/origin/pkg/cmd/server/origin/admission.NewAdmissionChains(0x0, 0x0, 0x0, 0x0, 0xc4204b3400, 0xc, 0xc4204b340c, 0x4, 0xc420e68d40, 0x3f, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/admission/chain_builder.go:212 +0x40c
github.com/openshift/origin/pkg/cmd/server/origin.BuildMasterConfig(0x0, 0x0, 0x0, 0x0, 0xc4204b3400, 0xc, 0xc4204b340c, 0x4, 0xc420e68d40, 0x3f, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/origin/master_config.go:155 +0x85b
github.com/openshift/origin/pkg/cmd/server/start.(*Master).Start(0xc4202f4680, 0xc4202f4680, 0xc4214d97e0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:517 +0x448
github.com/openshift/origin/pkg/cmd/server/start.MasterOptions.RunMaster(0xc420bfc400, 0x1, 0x2da, 0x721, 0x7ffebbf0ee48, 0x40, 0xef21d20, 0xc42000e018, 0x0, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_master.go:306 +0x2ef
github.com/openshift/origin/pkg/cmd/server/start.AllInOneOptions.StartAllInOne(0xc42109a780, 0xc4207ac9a0, 0x2da, 0x721, 0x0, 0x58e2ee6, 0x16, 0x7ffebbf0ee97, 0x46, 0x0, ...)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:309 +0x1e7
github.com/openshift/origin/pkg/cmd/server/start.NewCommandStartAllInOne.func1(0xc4203f5680, 0xc420e65420, 0x0, 0x2)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/server/start/start_allinone.go:90 +0x125
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc4203f5680, 0xc420e652a0, 0x2, 0x2, 0xc4203f5680, 0xc420e652a0)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:603 +0x234
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc4203f5440, 0xc42000e020, 0xc4203f5440, 0x6)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:689 +0x2fe
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc4203f5440, 0x9, 0xc4203f5440)
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:648 +0x2b
main.main()
	/data/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift/openshift.go:36 +0x24b

[smarterclayton]

@openshift/sig-developer-experience

[smarterclayton]

@openshift/sig-master related to refactoring of admission plugins?

[xiaods]

use upstream ,not found the bug. are you sure?

[bparees]

@xiaods I was able to recreate it.

Presumably this means the master config file being generated by the latest origin code is not valid for a 3.7-alpha1 master/cluster?

Oddly it works fine with a 3.6 cluster, so did we just have a temporary regression in 3.7-alpha1? (in which case presumably we can close this since we shouldn't be making any compatibility assumptions about an alpha release).

[mfojtik]

@deads2k ptal

[deads2k]

I'm betting its an old images problem. Admission webhooks wiring was messed up and I fixed it in master.

[smarterclayton]

But it worked before, so it has to be config related (some new config we added / took away).

…

On Mon, Oct 23, 2017 at 3:19 PM, David Eads ***@***.***> wrote: I'm betting its an old images problem. Admission webhooks wiring was messed up and I fixed it in master. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#16995 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG_p2Oegr_Jkz9gPWP4lisvkO6sYA-5ks5svJJPgaJpZM4QCKUq> .

[deads2k]

This #16861 enabled admission webhooks in oc cluster up and included the apiserver changes to make them not panic. If you don't have images at least that recent, but do have an oc that recent, you can get the error you're seeing.

[jim-minter]

Is there any mileage in improving the process around releasing openshift/origin:vx.y.z-alpha.* images so that this kind of thing can be avoided / can be fixed quicker when it happens?

[smarterclayton]

Unless you're volunteering, not right now. It's mostly blocked on the other more critical CI automation.

…

On Mon, Oct 23, 2017 at 5:08 PM, Jim Minter ***@***.***> wrote: Is there any mileage in improving the process around releasing openshift/origin:vx.y.z-alpha.* images so that this kind of thing can be avoided / can be fixed quicker when it happens? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#16995 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG_pxm4Td3JnDVZnxEtmqMHk0w1utq1ks5svKwAgaJpZM4QCKUq> .