Cannot create glue context debugging against AWS dev endpoint.

I'm having trouble getting remote debugging to work with an AWS glue endpoint.

I've been trying things in both the JetBrains tutorial and the Amazon one, but so far I've had no success.

My settings are: 

But when I attempt to debug the line:

sc = SparkContext.getOrCreate()

It throws the following error:

Exception in thread "main" java.lang.ExceptionInInitializerError
    at org.apache.spark.deploy.SparkSubmitArguments.$anonfun$loadEnvironmentArguments$3(SparkSubmitArguments.scala:157)
    at scala.Option.orElse(Option.scala:447)
    at org.apache.spark.deploy.SparkSubmitArguments.loadEnvironmentArguments(SparkSubmitArguments.scala:157)
    at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:115)
    at org.apache.spark.deploy.SparkSubmit$$anon$2$$anon$3.<init>(SparkSubmit.scala:1026)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.parseArguments(SparkSubmit.scala:1026)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:85)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.UnknownHostException: ip-[redacted]: [redacted]: Name or service not known
    at java.net.InetAddress.getLocalHost(InetAddress.java:1506)
    at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:1005)
    at org.apache.spark.util.Utils$.localIpAddress$lzycompute(Utils.scala:998)
    at org.apache.spark.util.Utils$.localIpAddress(Utils.scala:998)
    at org.apache.spark.util.Utils$.$anonfun$localCanonicalHostName$1(Utils.scala:1055)
    at scala.Option.getOrElse(Option.scala:189)
    at org.apache.spark.util.Utils$.localCanonicalHostName(Utils.scala:1055)
    at org.apache.spark.internal.config.package$.<init>(package.scala:990)
    at org.apache.spark.internal.config.package$.<clinit>(package.scala)
    ... 10 more
Caused by: java.net.UnknownHostException: ip-[redacted]: Name or service not known
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
    at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
    at java.net.InetAddress.getLocalHost(InetAddress.java:1501)
    ... 18 more

The IP represented by my edit [redacted] here is neither my local IP nor the dev endpoint IP. Which is confusing because I cannot source where this value is coming from.

Please sign in to leave a comment.