How do I use SyntaxHighlighter.getTokenHighlights(type) to return context-specific text attributes? Follow

The general approach is to use an annotator for this. For newer IntelliJ versions you will want to use the newSilentAnnotation API with HighlightSeverity.INFORMATION. For example:

  annotationHolder.newSilentAnnotation(HighlightSeverity.INFORMATION).range(element)
      .textAttributes(MyPluginHighlighterColors.PROPERTY)
      .create()

You can use TextAttributes.ERASE_MARKER via enforced text attributes to remove the previous styling if necessary (e.g. to remove bold styling from a keyword) in an annotation before the one that sets the text attributes for the semantic type.

  annotationHolder.newSilentAnnotation(HighlightSeverity.INFORMATION).range(element)
      .enforcedTextAttributes(TextAttributes.ERASE_MARKER)
      .create()

For the colour settings page, you can use getAdditionalHighlightingTagToDescriptorMap to define tags to the semantic types and their associated text attributes. That allows you to have something like "event.<property>editor</property>" in the demo text.

I'm not sure how this information helps with my problem. I'm trying to find out how a particular PsiElement would otherwise be styled before I apply my own style, because my choice of style is intended to be a variation upon whatever styling would otherwise have been applied.

In fact, TextAttributes.ERASE_MARKER is the exact opposite of what I want to do.

Here's what I think the issue might be. When I check what type of element a PsiElement is like this:

val type = elem.elementType ?: elem.node.elementType

I get a very generic type like IDENTIFIER. In the file DefaultLanguageHighlighterColors.java, for instance, however, there are much more specific types defined, such as FUNCTION_DECLARATION, which is a specific variety of identifier, with a possibly specific color in the current theme.

So what I think I really need to somehow discover is that a particular PsiElement has been more specifically classified as FUNCTION_DECLARATION, for example, or the matching TextMate scope entity.name.function. If, for testing purposes, I look up:

colorsScheme.getAttributes(DefaultLanguageHighlighterColors.FUNCTION_DECLARATION)

...I get the kind of color and style info I'm hoping to get, but what I can't do yet is find out by querying a PsiElement that FUNCTION_DECLARATION is the key being used for the current highlighting of an element.

The semantic-based colour information is not set by the syntax highlighter, it is set by annotators, HighlightVisitors (as is the case for the above -- see the details below), or other mechanisms. You would most likely need to get the editor where the file is open, locate the place in the editor where the PsiElement is located, and query the text attributes of that text range. Although I'm not sure what you are trying to do. -- If it is to correctly highlight the code in a custom editor window (e.g. from a run action), you would need to ensure that the annotators and highlighters get run.

The Java plugin looks like it is using a HighlightVisitor to set the semantic styling:

For DefaultLanguageHighlighterColors.FUNCTION_DECLARATION, Java extends that to a JavaHighlightingColors.METHOD_DECLARATION_ATTRIBUTES property that is used to create a HighlightInfoType in JavaHighlightInfoTypes.METHOD_DECLARATION

The HighlightNamesUtil.getMethodNameHighlightType function returns that highlight info object if the isDeclaration parameter is true (or JavaHighlightInfoTypes.CONSTRUCTOR_DECLARATION if the method is also a constructor). That function is called by HighlightNamesUtil.highlightMethodName, which is used by HighlightVisitorImpl. That class is registered in the plugin XML as a highlightVisitor extension, which ends up calling the HighlightVisitor.analyze method. NOTE: Other classes are using that class for other purposes (e.g. to check for errors).

For annotators, the erase marker code I showed is used by annotators such as GroovyKeywordAnnotator to remove styling from keywords that are not actually keywords, and GroovyAnnotator has the logic to build the annotations for the different semantic types. If you search for newSilentAnnotator, you will see several languages (regex, json, xml, etc.) that are using that to apply the semantic based approach to add the semantic styling. I'm using that approach in a plugin of mine to add styling to the language the plugin supports.

I'm trying to make an IntelliJ version of an extension I created for Visual Studio Code: https://marketplace.visualstudio.com/items?itemName=kshetline.ligatures-limited

The IntelliJ API doesn't provide direct, precise control over where font ligatures are rendered, and where they aren't. There's no ligature flag that's part of an individual TextAttribute. You can only turn ligatures on and off globally at the level of a color scheme.

So I need to use a trick for suppressing ligatures, which is to vary the style of every other character in a ligature, with a highlight applied to each character one at a time. Notice how the characters '<', '=', and '>' are blended into a single glyph in the first image, and rendered separately in the next, as well as the change in the rendering for "***":

If these characters are plain, italic, or bold, I want them to stay plain, italic, or bold. The only other way to trick IntelliJ into rendering each character separately is to vary the foreground color in an alternating pattern. But I don't want that change of color itself to be clearly noticeable. I only want the side effect the subtle color change creates of breaking the ligature up into separate characters. So what I do is create a pair of colors which are only a very slight modification of the original color:

private fun getMatchingColors(color: Color): Array<Color> {
  if (!cachedColors.containsKey(color)) {
    val alpha = color.rgb and 0xFF000000.toInt()
    val rgb = color.rgb and 0x00FFFFFF

    if (color.blue > 253)
      cachedColors[color] = arrayOf(Color(alpha or (rgb - 1)), Color(alpha or (rgb - 2)))
    else
      cachedColors[color] = arrayOf(Color(alpha or (rgb + 1)), Color(alpha or (rgb + 2)))
  }

  return cachedColors[color]!!
}

In order for this trick to work properly, however, I need to know what color each character was going to be rendered with in the first place, before I apply my changes. I can only get the color right part of the time, so far, however, because I haven't found a consistent way to query the API about colors that are currently being used.

(It's not this tricky to do in VSCode — simply applying a no-op style one character at a time breaks ligatures apart, without having to worry about picking a particular color or font style to use, and without having to alternate styles.)

I'll see if any of what you suggest above leads to the color info that I'm looking for.

Just to add some detail, the plugin I'm trying to create is functioning primarily as a HighlightVisitor, but it's a HighlightVisitor that needs to know how other HighlightVisitors have highlighted the code — if that's possible. And if I can't get the TextAttribute info itself associated with a particular PsiElement, I want to know how that PsiElement has been more specifically semantically categorized than the very high-level, low-detail info returned by PsiElement.elementType, so I can get the most specific TextAttributesKey possible, and look that key up in the current color scheme.

This is not for a custom editor. This plugin is meant to provide control over ligatures in standard code editor panes.

You said "You would most likely need to get the editor where the file is open, locate the place in the editor where the PsiElement is located, and query the text attributes of that text range."

I do have the PsiFile, I do have the Editor, and I do know where the PsiElements that I wish to re-style are located. So, how exactly, given all of that available info, do I "query the text attributes of that text range"?

I tried to look at Editor.markupModel, the only thing in the API of the Editor that looks close to what I might want. First, I got an exception for trying to access markupModel while not on the proper dispatch thread.

I then used ApplicationManager.getApplication().invokeLater() to take a look, and found only about 50 RangeHighlighter instances (far fewer instances than differently-colored tokens on the screen), all of them with a null foreground color.

After much experimentation, I finally found a solution.

It wasn't Editor.markupModel that I wanted to use, it was EditorImpl.filteredDocumentMarkupModel that I needed — that's where all of the formatting info that I need to examine is being kept. Using filteredDocumentMarkupModel requires making sure, of course, that the editor instance you're provided with is an instance of EditorImpl. That always seems to be the case so far for my needs.

Just like markupModel, filteredDocumentMarkupModel can only be used on IntelliJ's application dispatch thread (otherwise an exception is thrown). This adds a bit of a complication to implementing a HighlightVisitor, because HighlightVisitors aren't called on such a dispatch thread.

The result is that I've had to redesign my HighlightVisitor to do only some of the highlighting work I'm trying to do during the time period when my analyze() and visit() methods are called. I also end up ignoring the HighlightInfoHolder that's passed to the HighlightVisitor. Instead I accumulate a list of info for the highlights that I want to make, then use ApplicationManager.getApplication().invokeLater() to create the highlights in a somewhat different manner than a HighlightVisitor would normally do, by adding those highlights via MarkupModel.addRangeHighlighter() on the proper dispatch thread instead.

So I have code that looks like this that runs after figuring out everything except the color of the highlights I want to create:

if (editor != null) {
  ApplicationManager.getApplication().invokeLater {
    val oldHighlighters = syntaxHighlighters[editor]

    if (oldHighlighters != null) {
      oldHighlighters.forEach { highlighter -> editor.markupModel.removeHighlighter(highlighter) }
      syntaxHighlighters.remove(editor)
    }

    if (newHighlights.size > 0)
      applyHighlighters(editor, syntaxHighlighter, defaultForeground, newHighlights)

    highlightForCaret(editor, editor.caretModel.logicalPosition)
  }
}

Notice that I have to keep track of old highlights that I've created, and clean them up first before adding new highlights — something that does not need to be done when using a HighlightInfoHolder.

The rest of the related code looks like this (please keep in mind some of this logic is very specific to my particular needs, but I hope it's a good illustration of getting at the formatting info that I'm accessing):

private fun applyHighlighters(editor: Editor, syntaxHighlighter: SyntaxHighlighter,
                              defaultForeground: Color, highlighters: ArrayList<LigatureHighlight>) {
  if (editor !is EditorImpl || highlighters.isEmpty())
    return

  val markupModel = editor.markupModel
  val newHighlights = ArrayList<RangeHighlighter>()
  val existingHighlighters = getHighlighters(editor)

  for (highlighter in highlighters) {
    val foreground = highlighter.color ?: getHighlightColors(highlighter.elem, syntaxHighlighter, editor, editor.colorsScheme,
      defaultForeground, existingHighlighters)[highlighter.index % 2]
    val background = if (highlighter.color != null) defaultForeground else null

    newHighlights.add(markupModel.addRangeHighlighter(
      highlighter.index, highlighter.index + highlighter.width, MY_LIGATURE_LAYER,
      TextAttributes(foreground, background, null, EffectType.BOXED, 0),
      HighlighterTargetArea.EXACT_RANGE
    ))
  }

  syntaxHighlighters[editor] = newHighlights
}

private fun getHighlightColors(elem: PsiElement, syntaxHighlighter: SyntaxHighlighter, editor: Editor?,
    colorsScheme: TextAttributesScheme, defaultForeground: Color,
    defaultHighlighters: List<RangeHighlighter>? = null): Array<Color> {
  val type = elem.elementType ?: elem.node.elementType
  val textAttrKeys = syntaxHighlighter.getTokenHighlights(type)
  val textAttrs = getTextAttributes(colorsScheme, textAttrKeys)
  var color = textAttrs?.foregroundColor ?: defaultForeground

  val range = elem.textRange
  val highlighters = defaultHighlighters ?: getHighlighters(editor)
  var maxLayer = -1
  var minWidth = Int.MAX_VALUE
  val startIndex = findFirstIndex(highlighters, range.startOffset)

  if (startIndex >= 0) {
    for (i in startIndex..highlighters.size) {
      val highlighter = highlighters[i]

      if (highlighter.startOffset <= range.startOffset && range.endOffset <= highlighter.endOffset) {
        val specificColor = highlighter.textAttributes!!.foregroundColor
        val width = highlighter.endOffset - highlighter.startOffset
        val layer = highlighter.layer

        if (width < minWidth || (width == minWidth && layer > maxLayer)) {
          color = specificColor
          minWidth = width
          maxLayer = layer
        }
      }
      else if (highlighter.startOffset > range.endOffset)
        break
    }
  }

  return getMatchingColors(color)
}

private fun getHighlighters(editor: Editor?): List<RangeHighlighter>
{
  if (editor !is EditorImpl)
    return listOf()

  val highlighters = editor.filteredDocumentMarkupModel.allHighlighters

  highlighters.sortWith(Comparator { a, b ->
    when {
      a.startOffset > b.startOffset -> 1
      a.startOffset < b.startOffset -> -1
      else -> {
        val aWidth = a.endOffset - a.startOffset
        val bWidth = b.endOffset - b.startOffset

        when {
          aWidth > bWidth -> -1
          aWidth < bWidth -> 1
          else -> 0
        }
      }
    }
  })

  return highlighters.filter {
    h -> h.layer != MY_LIGATURE_LAYER && h.layer != MY_SELECTION_LAYER && h.textAttributes?.foregroundColor != null
  }
}

// Not quite close enough to any pre-defined binary search to avoid handling this as a special case
private fun findFirstIndex(highlighters: List<RangeHighlighter>, offset: Int): Int{
  var low = 0
  var high = highlighters.size - 1
  var matched = false
  var mid = -1

  // This will narrow down *a* highlighter that contains `offset`, but not necessarily the first one
  while (low <= high) {
    mid = (low + high) / 2
    val highlighter = highlighters[mid]

    if (offset in highlighter.startOffset..highlighter.endOffset) {
      matched = true
      break
    }
    else if (highlighter.endOffset < offset)
      low = mid + 1
    else
      high = mid - 1
  }

  if (!matched)
    return -1

  // Make sure we find the first matching highlighter
  while (mid > 0) {
    val highlighter = highlighters[mid - 1]

    if (offset in highlighter.startOffset..highlighter.endOffset)
      --mid
    else
      break
  }

  return mid
}

private fun getTextAttributes(colorsScheme: TextAttributesScheme, textAttrKeys: Array<TextAttributesKey>) :
    TextAttributes? {
  var textAttrs: TextAttributes? = null

  for (key in textAttrKeys)
    textAttrs = TextAttributes.merge(textAttrs, colorsScheme.getAttributes(key))

  return textAttrs
}

It's a lot more work than I thought would be required, but it gets the job done. As far as I can tell there's no built-in, straight-forward method to get to the formatting info I want to see — I have to do the work myself of sifting through the full list of syntax highlighting for the entire document to find the formatting that matches a particular row and column of the document.