Python type inference not working.

Answered

After loading a project, I'm visiting every PsiFile using `PsiRecursiveVisitor`.

I would like to infer the type of expression / function return type.

For even a simple python file containing a single line:

expr = 1 + 2

I want to calculate the type of expr

Using TypeEvalContext to calculate the type but TypeEvalContext::getType is always returning null.

@Override
public void visitPyAssignmentStatement(PyAssignmentStatement node) {
TypeEvalContext context = TypeEvalContext.codeCompletion(node.getProject(), node.getContainingFile());
PyExpression lhsElement = node.getLeftHandSideExpression();
if (lhsElement != null) {
PyType type = context.getType(lhsElement);
}
}

Am I missing something here?
What is the correct way to calculate type in python files?

0
18 comments

Is Python interpreter configured for the inspected project?

Type inference for string literals can be debugged in `com.jetbrains.python.psi.impl.PyStringLiteralExpressionImpl#getType`, how do you develop the plugin? With https://github.com/JetBrains/gradle-intellij-plugin?

Please note that visiting every project file is performance-heavy operation, if you described the goal, I could suggest better approach if any.

1

Please try using com.jetbrains.python.psi.types.TypeEvalContext#codeAnalysis instead

0

Yann Cebron

Tried using com.jetbrains.python.psi.types.TypeEvalContext#codeAnalysis too, but same results.

0

Yann Cebron

class Foo:
pass

a = Foo()
b = "str"

The TypeEvalContext::myEvaluated map also have null type for "str" and b.

0

Semyon Proshev

Yes, the python interpreter is configured for the inspected project (using virtual env).

Yes, developing using https://github.com/JetBrains/gradle-intellij-plugin (v1.0)

Tried debugging PyStringLiteralExpressionImpl#getType,

this.getObjectType("str")

the above snippet is returning null in PyBuiltinCache::getUnicodeType.

I'm attaching the debugging stack trace here.

0

Is Python interpreter configured for the inspected project?

I'm running the plug-in in headless mode. The inspected project is in the local file system.

What did you mean by inspected project?

Do I have to add python interpreter in my plug-in external libraries?

Kindly please explain.

Semyon Proshev

0

I mean the project which PsiFiles you visit with your visitor.

What value is shown if you execute `com.jetbrains.python.sdk.PythonSdkUtil#findPythonSdk(com.intellij.psi.PsiElement)` during debug session?

0

Thanks for your response.

I am getting null after executing PythonSdkUtil::findPythonSdk(PsiElement), even com.jetbrains.python.sdk.PythonSdkUtil#getAllSdks is returning an empty list.

This is my inspected project's structure:

project_name
|
└------------------ venv
|
|
└------------------ test
|
  └----------------- __init__.py
|
└----------------- main.py

I am using this API to load my project from the disk (Project path is the path to project_name).

com.intellij.ide.impl.ProjectUtil#openOrImport(java.lang.String, com.intellij.openapi.project.Project, boolean)

 

N.B.: I'm running the plugin in headless mode.

 

Semyon Proshev

0

Are you running IDEA + Python plugin or PyCharm in this mode?

0
We have implemented the plugin on appStarter extension point. To run this on headless mode, we are using Gradle runIDE task. This task is executing the following command internally. not certain if this mode is the IDEA+Python plugin or PyCharm (Also no PyCharm UI, running this from IDEA using Gradle).
/workspace/jbr-11_0_9-linux-x64-b944.49/jbr/bin/java -Didea.auto.reload.plugins=true -Didea.classpath.index.enabled=false -Didea.config.path=/workspace/idea/config -Didea.is.internal=true -Didea.platform.prefix=PyCharmCore -Didea.plugins.path=/workspace/plugins -Didea.required.plugins.id=com.our.product -Didea.system.path=/workspace/idea/system -Djava.awt.headless=true -Dsun.awt.disablegrab=true -Xms256m -Xmx8g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /workspace/pycharmPC-2020.2.4/lib/bootstrap.jar:/workspace/pycharmPC-2020.2.4/lib/extensions.jar:/workspace/pycharmPC-2020.2.4/lib/util.jar:/workspace/pycharmPC-2020.2.4/lib/jdom.jar:/workspace/pycharmPC-2020.2.4/lib/log4j.jar:/workspace/pycharmPC-2020.2.4/lib/jna.jar:/workspace/pycharmPC-2020.2.4/lib/trove4j.jar com.intellij.idea.Main runIDE

 

Semyon Proshev

0

It is PyCharm, does the inspected project contain .idea directory and .idea/misc.xml inside?

0

Yes, it does.

This is the content inside .idea/misc.xml

<project version="4">
<component name="ProjectRootManager" version="2" project-jdk-name="Python 3.8 (python_test)" project-jdk-type="Python SDK"/>
</project>

Semyon Proshev

0

If an interpreter is specified in .idea/misc.xml PyCharm does not run its auto-configuration, possible options:

  • remove .idea/misc.xml
  • or add
    Python 3.8 (python_test)
    to options/jdk.table.xml in config directory, better to do it through PyCharm
0

After removing idea/misc.xml PythonSdkUtil::findPythonSdk(PsiElement) returned null again.

Tried updating my options/jdk.table.xml (updated contents: https://pastebin.com/heRftn40) inside ~/.config/JetBrains/IntelliJIdea2021.2/options/, still no luck.

 

I don't want to update the XML file for every new project I want to inspect.

I would like to use the interpreter used in the inspected project or fall back to some default interpreter.

Currently having no luck finding python SDK while running plugin.

Semyon Proshev

0

Based on

-Didea.config.path=/workspace/idea/config

key in command line, `/workspace/idea/config/options/jdk.table.xml` should be updated.

Regarding first option, sorry, it should be changed to `remove .idea`.

If `.idea` is presented and interpreter is not listed in `config/options/jdk.table.xml`, feel free to set up interpreter in your plugin, don't hesitate to ask for helper methods.

0

Can you give me any example of how I can manually set up an interpreter in my plugin using helper methods?

 

I really appreciate your help.

 

N.B.: After removing .idea folder still found no SDK using PythonSdkUtil::findPythonSdk(PsiElement).

Semyon Proshev

0

Hello,

I'm using this approach to set up a python interpreter on the project.

public static void setupSDKFromConfig(Project project) {
String SDKPath = ConfigurationUtility.get(ConfigurationUtility.ConfigOptions.SDK_PATH);
if (ProjectRootManager.getInstance(project).getProjectSdk() == null) {
if (PythonSdkType.getInstance().isValidSdkHome(SDKPath)) {
Sdk sdk = SdkConfigurationUtil.createAndAddSDK(SDKPath, PythonSdkType.getInstance());
SdkConfigurationUtil.setDirectoryProjectSdk(project, sdk);
}
}
}

And for testing, I'm using https://github.com/JetBrains/gradle-python-envs.

Kindly tell me is the right way? Is there any better approach than this? Semyon Proshev

0

Please sign in to leave a comment.