Sciview Data fails with pandas new nullable integer types
Answered
According to https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html#nullable-integer-data-type, one can have a nullable integer data type in Series and DataFrames.
However, even though in iPython console this works fine:
import pandas as pd
s = pd.Series([1.0, 2.0, np.nan, 4.0])
s2 = s.astype('Int32')
When viewing s2 in the Sciview Data panel, I get the error:
ValueError: cannot convert float NaN to integer
Please sign in to leave a comment.
Hi, thank you for the report. This is a known issue: https://youtrack.jetbrains.com/issue/PY-34650
Any update on this? Now it appears to not even render nor show an error. The Dataframe viewer just says "empty" with "Nothing to show".
Any update on this ?
This not only fails, it fails so ungracefully with complete lockup of interface.
This is basic display of data that is in a valid format within python.
You can avoid "cannot convert float NaN to integer" with a mask method. Note first that in python NaN is defined as the number which is not equal to itself:
It might be worth avoiding use of np.NaN altogether. NaN literally means "not a number", and it cannot be converted to an integer. In general, Python prefers raising an exception to returning NaN, so things like
sqrt(-1) and log(0.0) will generally raise instead of returning NaN. However, you may get this value back from some other library. From v0.24, you actually can. Pandas introduces Nullable Integer Data Types
which allows integers to coexist with NaNs. Also, even at the lastest versions of pandas if the column is object type you would have to convert into float first, something like:
NB: You have to go through numpy float first and then to nullable Int32, for some reason.