How to connect to Zeppelin Server with a remote proxy
Answered
Hello,
There is no public IP for my EMR cluster and so I create a tunnel to a bastion and then connect to Zeppelin using a Firefox plugin like FoxyProxy. The tunnel is created using:
ssh -o ProxyCommand=ssh -W %h:%p bastion -i my-emr.pem -ND 8157 me@ip-my-ip.ec2.internal
Now using FoxyProxy I can set a proxy on localhost 8157 using SOCKS5 and connect to Zeppelin Notebooks as shown below:.
I try the same in case of IntelliJ but can't connect.
I set the URL to http://ip-my-ip-ec2.internal:8890/#/ and proxy to 'SOCKS' , host to localhost and proxy port to 8157 as shown below:
What can I do different?
Please sign in to leave a comment.
Hi. Could you please provide idea.log (Help | Show log...) after reproducing the problem?
I can't share with you the entire idea.log because of security concerns: But here is the exception in the log. The hostname has been changed. At the same time if I enter the URL http://ip-my-ip.ec2.internal:8890 in the firefox browser with FoxyProxy set as described above it works.
2020-05-07 10:48:48,577 [1639120] INFO - connections.ZeppelinConnection - Zeppelin WS ERROR Url: http://ip-my-ip.ec2.internal:8890, id: 1337628216, thread: com.intellij.bigdatatools.zeppelin.components.connections.ZeppelinConnection$webSocketListener$1@4fba9638
com.intellij.bigdatatools.zeppelin.api.remote.ZeppelinRestConnectionException: Cannot execute Rest Request to Zeppelin.
Check the connection status.
Original exception: com.jetbrains.bigdatatools.connection.RestPerformException: Cannot perform Get version request.Origin: java.net.UnknownHostException: ip-my-ip.ec2.internal: nodename nor servname provided, or not known
at com.intellij.bigdatatools.zeppelin.api.remote.ZeppelinGlobalApi.tryGetZeppelinInfo(ZeppelinGlobalApi.kt:258)
at com.intellij.bigdatatools.zeppelin.api.remote.ZeppelinGlobalApi.connect(ZeppelinGlobalApi.kt:44)
at com.intellij.bigdatatools.zeppelin.components.connections.ZeppelinConnection.connect(ZeppelinConnection.kt:195)
at com.intellij.bigdatatools.zeppelin.components.connections.ZeppelinConnection.doRefreshConnection(ZeppelinConnection.kt:177)
at com.intellij.bigdatatools.zeppelin.components.connections.ZeppelinConnection.access$doRefreshConnection(ZeppelinConnection.kt:28)
at com.intellij.bigdatatools.zeppelin.components.connections.ZeppelinConnection$refreshConnectionAsync$$inlined$synchronized$lambda$1.invoke(ZeppelinConnection.kt:154)
at com.intellij.bigdatatools.zeppelin.components.connections.ZeppelinConnection$refreshConnectionAsync$$inlined$synchronized$lambda$1.invoke(ZeppelinConnection.kt:28)
at com.jetbrains.bigdatatools.util.ThreadUtilsKt$executeOnPooledThread$1.run(ThreadUtils.kt:17)
at com.intellij.util.RunnableCallable.call(RunnableCallable.java:20)
at com.intellij.util.RunnableCallable.call(RunnableCallable.java:11)
at com.intellij.openapi.application.impl.ApplicationImpl$1.call(ApplicationImpl.java:255)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.jetbrains.bigdatatools.connection.RestPerformException: Cannot perform Get version request.Origin: java.net.UnknownHostException: ip-my-ip.ec2.internal: nodename nor servname provided, or not known
at com.jetbrains.bigdatatools.connection.RestClient.executeAndCheckRaw(RestClient.kt:120)
at com.jetbrains.bigdatatools.connection.RestClient.executeAndCheck(RestClient.kt:95)
at com.jetbrains.bigdatatools.connection.RestClient.performGetRequest(RestClient.kt:51)
at com.intellij.bigdatatools.zeppelin.api.remote.rest.ZeppelinRestClient.performGetRequest(ZeppelinRestClient.kt:38)
at com.intellij.bigdatatools.zeppelin.api.remote.rest.ZeppelinRestApi.getZeppelinInfo(ZeppelinRestApi.kt:147)
at com.intellij.bigdatatools.zeppelin.api.remote.ZeppelinGlobalApi.serverInfo(ZeppelinGlobalApi.kt:79)
at com.intellij.bigdatatools.zeppelin.api.remote.ZeppelinGlobalApi.tryGetZeppelinInfo(ZeppelinGlobalApi.kt:252)
... 14 more
Caused by: java.net.UnknownHostException: ip-my-ip.ec2.internal: nodename nor servname provided, or not known
at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:929)
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1515)
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:848)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1505)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1364)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1298)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at com.jetbrains.bigdatatools.connection.RestClient.executeAndCheckRaw(RestClient.kt:117)
Hi, Salil! Thank you for the question.
I think you do everything Ok, but it is a bug in Java.
The similar problem is described on StackOverflow: https://stackoverflow.com/questions/44369862/java-socks-proxy-uses-local-dns
As a workaround, you can try to enter the direct IP address of your host on S3 if it's possible.
We created a task in our YouTrack: https://youtrack.jetbrains.com/issue/BDIDE-861
We going to solve this problem in the nearest hotfix as soon as possible!