NO_PROXY settings

What would be the right way to bypass https proxy? Traditional -Dhttps.nonProxyHosts=localhost|127.0.0.1|kubernetes.default.svc and it is still going to https proxy host and of course fails since kubernetes.default.svc is an internal DNS.

Exporting https://github.com/fabric8io/kubernetes-client/blob/master/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/Config.java#L107 does not help as well.

[daxgames]

Is there any response to this?

[ljnelson]

Just another user lurking, but a few seconds of banging around in the codebase led me here:

kubernetes-client/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/utils/HttpClientUtils.java

Lines 173 to 197 in 9e850ad

	private static URL getProxyUrl(Config config) throws MalformedURLException {
	URL master = new URL(config.getMasterUrl());
	String host = master.getHost();
	if (config.getNoProxy() != null) {
	for (String noProxy : config.getNoProxy()) {
	if (isIpAddress(noProxy)) {
	if (new IpAddressMatcher(noProxy).matches(host)) {
	return null;
	}
	} else {
	if (host.contains(noProxy)) {
	return null;
	}
	}
	}
	}
	String proxy = config.getHttpsProxy();
	if (master.getProtocol().equals("http")) {
	proxy = config.getHttpProxy();
	}
	if (proxy != null) {
	return new URL(proxy);
	}
	return null;
	}

Maybe you can figure out what the issue is from there.

[carlossg]

try -Dno.proxy=localhost,127.0.0.1,kubernetes.default.svc

kubernetes-client/kubernetes-client/src/main/java/io/fabric8/kubernetes/client/Config.java

Line 109 in 9e850ad

public static final String KUBERNETES_NO_PROXY = "no.proxy";

[carlossg]

although this should probably use the Java default -Dhttp.nonProxyHosts="localhost|host.example.com"

https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html

[daxgames]

@carlossg Thanks, I will try that today and let you know, as I said before I am not a java expert. Just poking around trying to help fix issues and learn something new.

[daxgames]

I am now running Jenkins using the following java properties:

-Dhttp.proxy=http://x.x.x.x:xxxx -Dhttps.proxy=https://x.x.x.x:xxxx -Dno.Proxy=xxx -Dhttp.proxyHost=x.x.x.x -Dhttp.proxyPort=xxxx -Dhttps.proxyHost=x.x.x.x -Dhttps.proxyPort=xxxx -Dhttp.nonProxyHosts=xxx

Testing the connection from the Jenkins server running in Kubernetes through the Kubernetes plugin using the 'test connection' results in:

Error testing connection https://kubernetes.default: java.io.IOException: Unexpected response code for CONNECT: 502

and the following log entry on the proxy server:

1557513166.881     67 192.168.1.73 TCP_TUNNEL/502 0 CONNECT kubernetes.default.svc:443 - FIRSTUP_PARENT/192.168.1.100 -

Testing the connection from the Jenkins server running in Kubernetes through the 'Manage Jenkins->Manage Plugins->Advanced' tab using the 'validate proxy' button with the test URL of 'https://kubernetes.default.svc' results in the following expected failure since we are not authenticated:

Failed to connect to https://kubernetes.default.svc.
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
	at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:450)
	at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:257)
	at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:115)
	at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:156)
	at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:714)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:394)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:178)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:404)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:330)

From the command line of the same Jenkins server that has the exact same proxy settings defined in the java properties above set in environment variables. I can issue the following and retrieve a json object containing all pods in the namespace:

curl $APISERVER/api/v1/namespaces/pd-dgames-test/pods/ --header "Authorization: Bearer $TOKEN" --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

Kubernetes plugin spinning up a slave repeatedly gives me this:

May 10, 2019 6:24:53 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Excess workload after pending Kubernetes agents: 1
May 10, 2019 6:24:53 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Template for label pd-slave: Kubernetes Pod Template
May 10, 2019 6:24:53 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
WARNING: Failed to count the # of live instances on Kubernetes
java.io.IOException: Unexpected response code for CONNECT: 502
        at okhttp3.internal.connection.RealConnection.createTunnel(RealConnection.java:389)
        at okhttp3.internal.connection.RealConnection.connectTunnel(RealConnection.java:211)
        at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:152)
        at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256)
        at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134)
        at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113)
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at io.fabric8.kubernetes.client.utils.HttpClientUtils$2.intercept(HttpClientUtils.java:107)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
        at okhttp3.RealCall.execute(RealCall.java:77)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:328)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:193)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:618)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:68)
        at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.addProvisionedSlave(KubernetesCloud.java:505)
        at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.provision(KubernetesCloud.java:458)
        at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:715)
        at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:320)
        at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:61)
        at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:809)
        at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
        at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

[maczikasz]

I just spent quite a lot time on this, so please use standard at least fall back to it, I guess keep the no.proxy to support CIDR

[stale]

This issue has been automatically marked as stale because it has not had any activity since 90 days. It will be closed if no further activity occurs within 7 days. Thank you for your contributions!

[tjindilar]

I am now running Jenkins using the following java properties:

-Dhttp.proxy=http://x.x.x.x:xxxx -Dhttps.proxy=https://x.x.x.x:xxxx -Dno.Proxy=xxx -Dhttp.proxyHost=x.x.x.x -Dhttp.proxyPort=xxxx -Dhttps.proxyHost=x.x.x.x -Dhttps.proxyPort=xxxx -Dhttp.nonProxyHosts=xxx

Testing the connection from the Jenkins server running in Kubernetes through the Kubernetes plugin using the 'test connection' results in:

Error testing connection https://kubernetes.default: java.io.IOException: Unexpected response code for CONNECT: 502

and the following log entry on the proxy server:

1557513166.881     67 192.168.1.73 TCP_TUNNEL/502 0 CONNECT kubernetes.default.svc:443 - FIRSTUP_PARENT/192.168.1.100 -

Testing the connection from the Jenkins server running in Kubernetes through the 'Manage Jenkins->Manage Plugins->Advanced' tab using the 'validate proxy' button with the test URL of 'https://kubernetes.default.svc' results in the following expected failure since we are not authenticated:

Failed to connect to https://kubernetes.default.svc.
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
	at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:450)
	at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:257)
	at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:115)
	at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:156)
	at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:714)
	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:394)
	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:178)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:404)
	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:330)

From the command line of the same Jenkins server that has the exact same proxy settings defined in the java properties above set in environment variables. I can issue the following and retrieve a json object containing all pods in the namespace:

curl $APISERVER/api/v1/namespaces/pd-dgames-test/pods/ --header "Authorization: Bearer $TOKEN" --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

Kubernetes plugin spinning up a slave repeatedly gives me this:

May 10, 2019 6:24:53 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Excess workload after pending Kubernetes agents: 1
May 10, 2019 6:24:53 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
INFO: Template for label pd-slave: Kubernetes Pod Template
May 10, 2019 6:24:53 PM org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud provision
WARNING: Failed to count the # of live instances on Kubernetes
java.io.IOException: Unexpected response code for CONNECT: 502
        at okhttp3.internal.connection.RealConnection.createTunnel(RealConnection.java:389)
        at okhttp3.internal.connection.RealConnection.connectTunnel(RealConnection.java:211)
        at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:152)
        at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:256)
        at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:134)
        at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:113)
        at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:125)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at io.fabric8.kubernetes.client.utils.HttpClientUtils$2.intercept(HttpClientUtils.java:107)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
        at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:200)
        at okhttp3.RealCall.execute(RealCall.java:77)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:379)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:328)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:193)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:618)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:68)
        at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.addProvisionedSlave(KubernetesCloud.java:505)
        at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.provision(KubernetesCloud.java:458)
        at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:715)
        at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:320)
        at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:61)
        at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:809)
        at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
        at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Hi @daxgames
I am facing the same issue.
Have you resolved yours ?

[tjindilar]

I just spent quite a lot time on this, so please use standard at least fall back to it, I guess keep the no.proxy to support CIDR

HI @maczikasz
How could I configure that and where ?
I was just trying to add new cloud configuration in Jenkins. And I am facing the similar if not same issue.

[shawkins]

I just spent quite a lot time on this, so please use standard at least fall back to it, I guess keep the no.proxy to support CIDR

This was observed as well with #4938

@tjindilar please reopen or capture a new issue

We are forcing NO_PROXY if it's not configured on the config https://github.com/fabric8io/kubernetes-client/blob/master/httpclient-okhttp/src/main/java/io/fabric8/kubernetes/client/okhttp/OkHttpClientBuilderImpl.java#L71

For httpclients that support the standard java properties for configuration - jdk, okhttp? - if we instead allow the httpclient to use the default, the full range of system settings would be available (except for jdk socks support).

However I don't believe that jetty or vertx will use the system properties.

[manusa]

We might want to provide some means to easily disable this

kubernetes-client/httpclient-okhttp/src/main/java/io/fabric8/kubernetes/client/okhttp/OkHttpClientBuilderImpl.java

Lines 70 to 72 in fe579f0

	if (proxyAddress == null) {
	builder.proxy(Proxy.NO_PROXY);
	} else {

as a quick workaround. Then maybe provide full support for SOCKS proxy across client implementations* (see how to overcome this https://bugs.openjdk.org/browse/JDK-8214516).

[shawkins]

as a quick workaround

Unfortunately there isn't a great one via the HttpClient Factory with the existing way code is structured. We'll probably need a code change.

Then maybe provide full support for SOCKS proxy across client implementations* (see how to overcome this https://bugs.openjdk.org/browse/JDK-8214516).

Socks support is captured as a separate issue #5086