Provide correctly resolved "synthetic" psi element (Python)

Answered

The problem I am trying to solve

Hello community, I use the PyDebugValue.getFrameAccessor().evaluate(...) method to evaluate new PyDebugValues.
I have a single-line code editor like the Evaluate Expression-editor in IntelliJ, where the user can enter an expression. In my editor I want to include a synthetic identifier for example _df which resolves to such a previous evaluated PyDebugValue. Or at least to an object of the same type, to have correct code completion for the properties/fields of the type. 

The identifier _df should be available in my editor in the auto-completion popup AND correctly resolved.

Demo of the Problem

The problem I'm trying to solve can also be seen in the Evaluate Expression-editor in IntelliJ.

Here is a short example:

A string is evaluated, and the result is used to do further evaluations afterwards (in the second dialog).
The temporary identifier used to refer to the value is not resolved by the editor (marked by the red border) and also the auto-completion isn't possible. Nevertheless, it is a valid identifier, as the evaluation results of the statement in the second dialog shows.

Current way how I create the editor document

I create new Python documents when the user selects a Python variable in the debug view. Afterwards the document is assigned to a single line editor to let the user define a custom expression which is processed at a later point by my plugin. 

The document is created with:

val document = PyDebuggerEditorsProvider().createDocument(
    project, 
    XDebuggerUtil.getInstance().createExpression("", PythonLanguage.INSTANCE, null, EvaluationMode.EXPRESSION), // empty expression
    XDebuggerManager.getInstance(project).currentSession?.currentPosition,
    EvaluationMode.EXPRESSION,
)

The psi-file mapped to the returned document is a com.jetbrains.python.psi.impl.PyExpressionCodeFragmentImpl with the psi-element, specified by the 3rd parameter provided to the createDocument call, as context. All identifier available in the context are correctly resolved when doing auto-complete or entering identifier manually into the editor.

Entering other identifier also results in a red border.

What I have tried so far

I implemented a com.intellij.codeInsight.completion.CompletionContributor and a com.intellij.psi.PsiReferenceContributor. With the CompletionContributor I was able to provide the identifier _df. But I couldn't find a way to create a PsiReferenceContributor which satisfies my needs. It seems that I can only contribute psi-references for string literals, at least I had no success using other element pattern.

I also tried to chain two PyExpressionCodeFragmentImpl instances, where the first one defines the synthetic identifier _df:

// snippet to create the identifier "_df"
val code = """
    |from pandas import DataFrame
    |_df = DataFrame()
    |breakpoint()
""".trimMargin()

// create document which defines "_df" and has as context the sourcePosition taken from the debugger
val firstDocument = PyDebuggerEditorsProvider().createDocument(
    project,
    XDebuggerUtil.getInstance().createExpression(code, PythonLanguage.INSTANCE, null, EvaluationMode.CODE_FRAGMENT),
    XDebuggerManager.getInstance(project).currentSession?.currentPosition,
    EvaluationMode.CODE_FRAGMENT,
)
// calculate source-position for second document
val sourcePositionFromFirstFragment = XDebuggerUtil.getInstance().createPositionByElement(
    PsiDocumentManager.getInstance(project).getPsiFile(firstDocument)!!.lastChild
)

// create document for the editor
val secondDocument = PyDebuggerEditorsProvider().createDocument(
    project,
    XDebuggerUtil.getInstance().createExpression("", PythonLanguage.INSTANCE, null, EvaluationMode.EXPRESSION),
    sourcePositionFromFirstFragment,
    EvaluationMode.EXPRESSION,
)

With this approach, the identifier _df is included in the auto-completion popup. But can't be resolved after completion. The same applies to all elements in the file to which XDebuggerManager.getInstance(project).currentSession?.currentPosition currently points.

3 comments
Comment actions Permalink

Hi! As far as I understood from your explanation, you want to do the following in your editor:
1. Do not mark new variable as unresolved (remove red highlighting)
2. Provide completion corresponding to its type

Actually, there're two ways to implement your request:
1. The first option is to implement honest code insight with your own completion contributor. Actually, we do something similar in Type Renderers configuration windows. You can get fully qualified type name from `PyDebugValue`, resolve it, then implement your own completion contributor like we do it in `com.jetbrains.python.debugger.variablesview.usertyperenderers.codeinsight.TypeNameCompletionProvider`.  After that you can update variable's context with fully qualified name of the type. The use cases for builtin types and your custom types will be different and you should carefully handle all of them.

2. The second option is to use runtime completion, like we do it in Python Console or in Debugger. You didn't mention if you're in debugger or in Python Console, but in both cases we send request to the runtime and then show suggested variants (It's `com.jetbrains.python.debugger.PyDebugProcess#getCompletions` for Python debug process and `com.jetbrains.python.console.PydevConsoleCommunication#getCompletions` for Python Console).
If you implement a runtime completion, it won't need anything additional (like resolve or completion contributor), you just need to disable unresolved reference and that's it.

If I were implementing this functionality, I would selected the second option, because it's easier to implement and more reliable. But it's up to you, of course.

1
Comment actions Permalink

Thank you Elizabeth for the fast answer.

I initially started with a TextFieldCompletionProvider inspired by this code. However, I did not pursue this variant further due to its complexity.

Option 2 sounds interesting. In my plugin I use the Debugger.

However, I found today this little gem: PyReferenceResolveProvider

This seems to solve my problem quite easily. At least I could not see any problems in my manual tests yet.

This is my poc-implementation:

import com.intellij.openapi.util.Key
import com.jetbrains.python.codeInsight.controlflow.ScopeOwner
import com.jetbrains.python.psi.PyExpressionCodeFragment
import com.jetbrains.python.psi.PyQualifiedExpression
import com.jetbrains.python.psi.resolve.PyReferenceResolveProvider
import com.jetbrains.python.psi.resolve.PyResolveUtil
import com.jetbrains.python.psi.resolve.RatedResolveResult
import com.jetbrains.python.psi.types.TypeEvalContext

class PyCodeFragmentReferenceResolveProvider : PyReferenceResolveProvider {

companion object {
val RESOLVE_REFERENCES = Key.create<Boolean>("${Companion::class.java.name}.RESOLVE_REFERENCES")
}

override fun resolveName(element: PyQualifiedExpression, context: TypeEvalContext): List<RatedResolveResult> {
val origin = context.origin
if (
origin is ScopeOwner &&
element.containingFile is PyExpressionCodeFragment &&
RESOLVE_REFERENCES.get(element.containingFile.virtualFile, false)
) {
element.referencedName?.let { name ->
return PyResolveUtil.resolveLocally(origin, name)
.map { RatedResolveResult(RatedResolveResult.RATE_NORMAL, it) }
}
}
return emptyList()
}
}

In combination with my chained PyExpressionCodeFragmentImpl instances (mentioned in my first post), I get now full code completion also for my synthetic identifier _df. The identifier is later replaced with the correct reference to the PyDebugValue before I process the user input from the editor.

This solution is totally fine because in my case the evaluated PyDebugValues are always pandas DataFrames.

Is the use of PyReferenceResolveProvider OK? Or do you see any potential problems with this solution?

 

0
Comment actions Permalink

Yes, I think usage of PyReferenceResolveProvider is totally fine. Thank you for posting an update for other developers!

1

Please sign in to leave a comment.