debugging implicits

Hi,

Working with implicits can be a complete nightmayer. The scala plugin is improving, but not quite there yet. Sometimes it is able to tell me which implicit is being used but sometimes gets it wrong. However, in the cases where no implicit is found, what I really need is a debug environment to let me drill down into the available implicits that where rejected, find out why they where, and work through adding hints to get it working.

Typically, if an implicit isn't found, I end up passing in the implicit using implicitly[Foo], and then if that isn't working, perhaps replacing this with an explicit reference to the implicit I think should be used. This is very time-consuming and error-prone. You end up stubbing in a lot of things that where being inferred correctly, and when the time comes to unstub step-by-step, you can get into a muddle. Also, you have to make a lot of choices about type parameters that are painful to get right and to get consistent, particularly when the implicit resolution process is being used to do type inference.

What I'd really like is an environment for exploring and debugging implicits. At the site where the implicit needs to be inferred, I'd like to be able to pop up a box that shows each implicit parameter as an expression, highlighting any that are problems. For those, I'd like to be able to select the one I was intending to be used, and get a justification back for why it is not being. If it itself relies upon an implicit, I want to be able to do this recursively. If it is using one implicit and I think it should be using another, I'd like to know why it picked the 'wrong' one - is it more specific, or does mine not apply for some reason? Perhaps there's a type parameter that I want to explicitly pin. Red could be used for the broken implicit, grey for one that matches and is the one that scalac will use, black for the implcits and type params I've manually selected that scalac would not have for whatever reason. Something like that.

Oh, and for moon-on-a-stick bonus points, I'd like this environment to stay live as I move implicits into and out of scope, and as I edit the defs, objects and vals that are implicit.

How much of this is possible, given how the scala plugin handles implicits now? It would be a real selling point for the plugin IMHO for people developing libraries like scalala and scalaz that relie heavily upon implicits.

Thanks,

Matthew

14 comments

These ideas are being explored by EPFL in the Type Debugger.

https://wiki.scala-lang.org/display/SW/ScalaDays+2011+Resources#ScalaDays2011Resources-TypeDebugger

The manual process you described is pretty close to the one I use. I find it pretty mechanical, and not overly onerous when tracking down implicit not found problems.

-jason

0

I'm planning to add some functionality for implicits and completion at the end of February (such as intentions to make calls explicit/implicit, removing ambiguous implicit from scope, possibly implicits autoimporting and something like you described for implicits debugging).

Best regards,
Alexander Podkhalyuzin.

0

on my personal wish list:
* show all implicit conversions in scope
* fold type annotations (like java generics) including the ":" so i can hover above them when i want to know what the types really are. it should be a small and unique symbol, ideally only one single char, but the current <~> would also be fine. then you could write:

def main(args:Array[String]) {....}
which would look like this:
def main(args <~>) {}

my reasons:

for primitives and common cases, the types are usually obvious. the compiler needs them, but to the human code reader, they are noise.
in case the human reader forgots what the type was or is reading new code, he/she can switch the folding off via a hotkey.
this would solve a dilemma i am currently in. i want to let the compiler infer as much as possibe because then, the code is shorter. this is no big deal with simple types, but when it comes to tuples and nested generics, it really helps. on the other hand, sometimes i forget what the exact type is, especially when it comes to tuples and nested generics ;). then i need a quick way to look them up. i know there is a show-type-popup-hotkey, but it only works for *one* symbol at a time.

now that i think about it - can you make it pop up everywhere and let me switch it on and off?

0

That is interesting idea to make popup everywhere.
However for your case I have an idea to open new read-only editor with all types/implicits (which can be configurable on the fly). Maybe it's better solution than popups?
As for all implicits in scope you possibly right (I mean there are many possibilities to improve Ctrl+Shift+Q action, this is one ofr them). It will be improved at the end of February.

Best regards,
Alexander Podkhalyuzin.

0

if you plan on make everything explicit, then opening a new readonly editor tab or window (i prefer editor tab) would be better than popups.

0

i checked all possible combinations i can think of. the plugin and scalac disagree on some of them.
the order of scalac is:

1. look in local scope
2. look in companion object that might be converted

but there are quite a few confusing rules.

add this at the top and ignore the fact that the method names make no sense:

object ImplicitResolutionOne {
  implicit def parse(s: String) = Integer.parseInt(s)

  implicit def parse(s: ImplicitResolutionOne) = 1

  implicit def parse(s: ImplicitResolutionTwo) = 2
}

class ImplicitResolutionOne(val str: String)

object ImplicitResolutionTwo {
  implicit def parse(s: ImplicitResolutionOne) = 3

  implicit def parse(s: ImplicitResolutionTwo) = 4
}

1. local implicit def > local import if on same level, no matter the order:

object Caller {
  def main(args: Array[String]) {
    {
      implicit def parse(i: ImplicitResolutionOne) = 5;
      import ImplicitResolutionTwo._
      val x: Int = new ImplicitResolutionOne("5")
      println(x)
    }
  }
}

2. local import > implicit def on outer level:

object Caller {
  def main(args: Array[String]) {
    implicit def parse(i: ImplicitResolutionOne) = 5;
    {
      import ImplicitResolutionTwo._
      val x: Int = new ImplicitResolutionOne("3")
      println(x)
    }
  }
}

3. if several imports are used, the nearest one wins
4. only implicits that are already defined are considered:

object Caller {
  def main(args: Array[String]) {
    {
      import ImplicitResolutionTwo._
      val x: Int = new ImplicitResolutionOne("3")
      println(x)
    }
  }

  implicit def parse(i: ImplicitResolutionOne) = 5;

}


most likely i also found a scalac bug:

object Caller {
  def main(args: Array[String]) {
    {
      import ImplicitResolutionTwo._

      val x: Int = new ImplicitResolutionOne("3")
  implicit def parse(i: ImplicitResolutionOne) = 5;//removing and adding the line makes the compiler pick another implicit
      println(x)

    }
  }



}

0

That is interesting idea. However possibly it will be much simpler to have right ordering of implicit conversions and possibly coloring.

Best regards,
Alexander Podkhalyuzin.

0

It seems that in last example compiler ignores forward reference.
For example following code will not compile:

object Caller {   def main(args: Array[String]) {     {       import ImplicitResolutionTwo._       val x: Int = new ImplicitResolutionOne("3")       implicit val parse: ImplicitResolutionOne => Int = _ => 5       println(x)     }   } }

Moreover following code also will not compile:

object Caller {   def main(args: Array[String]) {     {       import ImplicitResolutionTwo._       val x: Int = parse(new ImplicitResolutionOne("3"))       implicit def parse(i: ImplicitResolutionOne) = 5       println(x)     }   } }

So question is why explicit call to parse is not working, but implicit call is working. It's obviously compiler bug. Another thing is that plugin do nothing with forward reference errors, but it's just not implemented feature of error highlighting.

Best regards,
Alexander Podkhalyuzin.

0

some guy on the scala mailing list said:

The implicit which had been imported is shadowed by the definition of parse which follows it -- even though that definition is ineligible as an implicit, it shadows the name throughout the block.

so an implicit can be name shadowed which makes it not being searched. the one which shadowed the other implicit away is not a valid implicit because of the forward reference, so the code does not compile instead oif falling back to the first implicit which could have been taken.

0

That is exactly my thought.
So you have found three compiler bugs for 2.9.1 (have you written tickets about them?):
1. This code shouldn't compile, because of name shadowing, and problem with calling forward reference:

object ImplicitResolutionOne {
  implicit def parse(s:String) = Integer.parseInt(s)
  implicit def parse(s: ImplicitResolutionOne) = 1
  implicit def parse(s: ImplicitResolutionTwo) = 2
}

class ImplicitResolutionOne(val str:String)

object ImplicitResolutionTwo {
  implicit def parse(s: ImplicitResolutionOne) = 3
  implicit def parse(s: ImplicitResolutionTwo) = 4
}

class ImplicitResolutionTwo(val str: String)

object Caller {
  def main(args: Array[String]) {
    {
      import ImplicitResolutionTwo._
      val x: Int = new ImplicitResolutionOne("3")
      implicit def parse(i: ImplicitResolutionOne) = 5
      println(x)
    }
  }
}


2. This code shouldn't compile, because there is no names shadowng, two `parse` methods have same priority:

object ImplicitResolutionOne {   implicit def parse(s:String) = Integer.parseInt(s)   implicit def parse(s: ImplicitResolutionOne) = 1   implicit def parse(s: ImplicitResolutionTwo) = 2 } class ImplicitResolutionOne(val str:String) object ImplicitResolutionTwo {   implicit def parse(s: ImplicitResolutionOne) = 3   implicit def parse(s: ImplicitResolutionTwo) = 4 } class ImplicitResolutionTwo(val str: String) object Caller {   import ImplicitResolutionTwo._   import ImplicitResolutionOne._   def main(args: Array[String]) {     val x:Int = new ImplicitResolutionOne("3")     val y:Int = new ImplicitResolutionTwo("4")     println(x)     println(y)   } }


3. This code should print "5" instead of "1", here forward reference is ok:

object ImplicitResolutionOne {   implicit def parse(s:String) = Integer.parseInt(s)   implicit def parse(s: ImplicitResolutionOne) = 1   implicit def parse(s: ImplicitResolutionTwo) = 2 } class ImplicitResolutionOne(val str:String) object ImplicitResolutionTwo {   implicit def parse(s: ImplicitResolutionOne) = 3   implicit def parse(s: ImplicitResolutionTwo) = 4 } class ImplicitResolutionTwo(val str: String) object Caller {   def main(args: Array[String]) {     val x:Int = new ImplicitResolutionOne("1")     println(x)   }   implicit def parse(i: ImplicitResolutionOne) = 5 }



Best regards,
Alexander Podkhalyuzin.
0

Ok, thank you. It was great study of implicit resolution.

Best regards,
Alexander Podkhalyuzin.

0

the bug report for the double import bug:
https://issues.scala-lang.org/browse/SI-5376

the third case is not a bug, it's a compiler limitation:

object Correct2 {   object CompanionObject {     implicit def convert(co: CompanionObject) = "o: co/p: co"     implicit def convert(co: CompanionObject2) = "o: co/p: co2"   }   class CompanionObject   object CompanionObject2 {     implicit def convert(s: CompanionObject) = "o: co2/p: co"     implicit def convert(s: CompanionObject2) = "o: co2/p: co2"   }   class CompanionObject2   def pushToConsole(s: String) {     println(s)   }   def main(args: Array[String]) {     //pick conversion from companion object     pushToConsole(new CompanionObject)     pushToConsole(new CompanionObject2)     //import overrides conversion from companion object     import CompanionObject2._     pushToConsole(new CompanionObject)     pushToConsole(new CompanionObject2)     val i:Int = new CompanionObject     println(i)   }      implicit def convert(c:CompanionObject):Int = 99 }



the compiler can only use implicits that are defined before their usage or have an explicit result type. remove the result type of the last implicit def and you will get a proper error message because in this example, there is no fallback conversion.

the first case

object Caller {
  def main(args: Array[String]) {
    {
      import ImplicitResolutionTwo._
      val x: Int = new ImplicitResolutionOne("3")
      implicit def parse(i: ImplicitResolutionOne) = 5
      println(x)
    }
  }
}
 

is strange, but it is not a bug - it follows the rule "what is accessible at the call site without a prefix is a possible candidate for an implicit conversion". the implicit def (=5) makes all the other implicits inaccessible via name shadowing, but itself is not a valid candidate because it would be a forward reference. the compiler falls back to the companion object's implicit.
0

Please sign in to leave a comment.