>>> from env_helper import info; info()
页面更新时间： 2020-03-08 18:01:19
操作系统/OS: Linux-4.19.0-8-amd64-x86_64-with-debian-10.3 ；Python: 3.7.3

7.6. Pandas稀疏数据

当任何匹配特定值的数据(NaN/缺失值，尽管可以选择任何值)被省略时，稀疏对象被“压缩”。 一个特殊的SparseIndex对象跟踪数据被“稀疏”的地方。 这将在一个例子中更有意义。 所有的标准Pandas数据结构都应用了to_sparse方法 -

>>> import pandas as pd
>>> import numpy as np
>>>
>>> ts = pd.Series(np.random.randn(10))
>>> ts[2:-2] = np.nan
>>> sts = ts.to_sparse()
>>> print (sts)
0    0.521074
1    0.601970
2         NaN
3         NaN
4         NaN
5         NaN
6         NaN
7         NaN
8    0.078770
9   -0.320702
dtype: float64
BlockIndex
Block locations: array([0, 8], dtype=int32)
Block lengths: array([2, 2], dtype=int32)

为了内存效率的原因，所以需要稀疏对象的存在。

现在假设有一个大的NA DataFrame并执行下面的代码 -

>>> import pandas as pd
>>> import numpy as np
>>>
>>> df = pd.DataFrame(np.random.randn(10000, 4))
>>> df.ix[:9998] = np.nan
>>> sdf = df.to_sparse()
>>>
>>> print (sdf.density)
0.0001
/usr/lib/python3/dist-packages/ipykernel_launcher.py:5: DeprecationWarning:
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing

See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
  """

通过调用to_dense可以将任何稀疏对象转换回标准密集形式 -

>>> import pandas as pd
>>> import numpy as np
>>> ts = pd.Series(np.random.randn(10))
>>> ts[2:-2] = np.nan
>>> sts = ts.to_sparse()
>>> print (sts.to_dense())
0    0.610515
1   -0.845538
2         NaN
3         NaN
4         NaN
5         NaN
6         NaN
7         NaN
8   -0.489403
9   -0.750825
dtype: float64

稀疏Dtypes

稀疏数据应该具有与其密集表示相同的dtype。 目前，支持float64，int64和booldtypes。 取决于原始的dtype，fill_value默认值的更改 -

float64 − np.nan
int64 − 0
bool − False

执行下面的代码来理解相同的内容 -

>>> import pandas as pd
>>> import numpy as np
>>>
>>> s = pd.Series([1, np.nan, np.nan])
>>> print (s)
>>> print ("=============================")
>>> s.to_sparse()
>>> print (s)
0    1.0
1    NaN
2    NaN
dtype: float64
=============================
0    1.0
1    NaN
2    NaN
dtype: float64