在Pandas数据框中转换分类数据

问题:在Pandas数据框中转换分类数据

我有一个带有这种类型的数据的数据框(列太多):

col1        int64
col2        int64
col3        category
col4        category
col5        category

列看起来像这样:

Name: col3, dtype: category
Categories (8, object): [B, C, E, G, H, N, S, W]

我想像这样将列中的所有值转换为整数:

[1, 2, 3, 4, 5, 6, 7, 8]

我通过以下方法解决了这一问题:

dataframe['c'] = pandas.Categorical.from_array(dataframe.col3).codes

现在,我的数据框中有两列-旧列col3和新c列,需要删除旧列。

那是不好的做法。它是可行的,但是在我的数据框中有很多列,我不想手动进行。

pythonic如何巧妙地实现呢?

I have a dataframe with this type of data (too many columns):

col1        int64
col2        int64
col3        category
col4        category
col5        category

Columns seems like this:

Name: col3, dtype: category
Categories (8, object): [B, C, E, G, H, N, S, W]

I want to convert all value in columns to integer like this:

[1, 2, 3, 4, 5, 6, 7, 8]

I solved this for one column by this:

dataframe['c'] = pandas.Categorical.from_array(dataframe.col3).codes

Now I have two columns in my dataframe – old col3 and new c and need to drop old columns.

That’s bad practice. It’s work but in my dataframe many columns and I don’t want do it manually.

How do this pythonic and just cleverly?


回答 0

首先,要将“分类”列转换为其数字代码,可以使用以下命令更轻松地做到这一点dataframe['c'].cat.codes
此外,可以使用来自动选择数据框中具有特定dtype的所有列select_dtypes。这样,您可以将上述操作应用于多个自动选择的列。

首先制作一个示例数据框:

In [75]: df = pd.DataFrame({'col1':[1,2,3,4,5], 'col2':list('abcab'),  'col3':list('ababb')})

In [76]: df['col2'] = df['col2'].astype('category')

In [77]: df['col3'] = df['col3'].astype('category')

In [78]: df.dtypes
Out[78]:
col1       int64
col2    category
col3    category
dtype: object

然后通过使用select_dtypes选择列,然后将其应用于.cat.codes这些列中的每一个,您可以获得以下结果:

In [80]: cat_columns = df.select_dtypes(['category']).columns

In [81]: cat_columns
Out[81]: Index([u'col2', u'col3'], dtype='object')

In [83]: df[cat_columns] = df[cat_columns].apply(lambda x: x.cat.codes)

In [84]: df
Out[84]:
   col1  col2  col3
0     1     0     0
1     2     1     1
2     3     2     0
3     4     0     1
4     5     1     1

First, to convert a Categorical column to its numerical codes, you can do this easier with: dataframe['c'].cat.codes.
Further, it is possible to select automatically all columns with a certain dtype in a dataframe using select_dtypes. This way, you can apply above operation on multiple and automatically selected columns.

First making an example dataframe:

In [75]: df = pd.DataFrame({'col1':[1,2,3,4,5], 'col2':list('abcab'),  'col3':list('ababb')})

In [76]: df['col2'] = df['col2'].astype('category')

In [77]: df['col3'] = df['col3'].astype('category')

In [78]: df.dtypes
Out[78]:
col1       int64
col2    category
col3    category
dtype: object

Then by using select_dtypes to select the columns, and then applying .cat.codes on each of these columns, you can get the following result:

In [80]: cat_columns = df.select_dtypes(['category']).columns

In [81]: cat_columns
Out[81]: Index([u'col2', u'col3'], dtype='object')

In [83]: df[cat_columns] = df[cat_columns].apply(lambda x: x.cat.codes)

In [84]: df
Out[84]:
   col1  col2  col3
0     1     0     0
1     2     1     1
2     3     2     0
3     4     0     1
4     5     1     1

回答 1

这对我有用:

pandas.factorize( ['B', 'C', 'D', 'B'] )[0]

输出:

[0, 1, 2, 0]

This works for me:

pandas.factorize( ['B', 'C', 'D', 'B'] )[0]

Output:

[0, 1, 2, 0]

回答 2

如果您只关心增加一列并在以后将其删除,则只需在第一处使用新列即可。

dataframe = pd.DataFrame({'col1':[1,2,3,4,5], 'col2':list('abcab'),  'col3':list('ababb')})
dataframe.col3 = pd.Categorical.from_array(dataframe.col3).codes

大功告成 现在Categorical.from_array已弃用,请Categorical直接使用

dataframe.col3 = pd.Categorical(dataframe.col3).codes

如果您还需要从索引到标签的映射,那么还有更好的方法

dataframe.col3, mapping_index = pd.Series(dataframe.col3).factorize()

检查下面

print(dataframe)
print(mapping_index.get_loc("c"))

If your concern was only that you making a extra column and deleting it later, just dun use a new column at the first place.

dataframe = pd.DataFrame({'col1':[1,2,3,4,5], 'col2':list('abcab'),  'col3':list('ababb')})
dataframe.col3 = pd.Categorical.from_array(dataframe.col3).codes

You are done. Now as Categorical.from_array is deprecated, use Categorical directly

dataframe.col3 = pd.Categorical(dataframe.col3).codes

If you also need the mapping back from index to label, there is even better way for the same

dataframe.col3, mapping_index = pd.Series(dataframe.col3).factorize()

check below

print(dataframe)
print(mapping_index.get_loc("c"))

回答 3

这里需要转换多列。因此,我使用的一种方法是..

for col_name in df.columns:
    if(df[col_name].dtype == 'object'):
        df[col_name]= df[col_name].astype('category')
        df[col_name] = df[col_name].cat.codes

这会将所有字符串/对象类型列转换为类别。然后将代码应用于每种类别。

Here multiple columns need to be converted. So, one approach i used is ..

for col_name in df.columns:
    if(df[col_name].dtype == 'object'):
        df[col_name]= df[col_name].astype('category')
        df[col_name] = df[col_name].cat.codes

This converts all string / object type columns to categorical. Then applies codes to each type of category.


回答 4

为了转换数据集数据的C列中的分类数据,我们需要执行以下操作:

from sklearn.preprocessing import LabelEncoder 
labelencoder= LabelEncoder() #initializing an object of class LabelEncoder
data['C'] = labelencoder.fit_transform(data['C']) #fitting and transforming the desired categorical column.

For converting categorical data in column C of dataset data, we need to do the following:

from sklearn.preprocessing import LabelEncoder 
labelencoder= LabelEncoder() #initializing an object of class LabelEncoder
data['C'] = labelencoder.fit_transform(data['C']) #fitting and transforming the desired categorical column.

回答 5

@ Quickbeam2k1,请参见下文-

dataset=pd.read_csv('Data2.csv')
np.set_printoptions(threshold=np.nan)
X = dataset.iloc[:,:].values

使用sklearn

from sklearn.preprocessing import LabelEncoder
labelencoder_X=LabelEncoder()
X[:,0] = labelencoder_X.fit_transform(X[:,0])

@Quickbeam2k1 ,see below –

dataset=pd.read_csv('Data2.csv')
np.set_printoptions(threshold=np.nan)
X = dataset.iloc[:,:].values

Using sklearn

from sklearn.preprocessing import LabelEncoder
labelencoder_X=LabelEncoder()
X[:,0] = labelencoder_X.fit_transform(X[:,0])

回答 6

我要做的是,我 replace重视。

像这样-

df['col'].replace(to_replace=['category_1', 'category_2', 'category_3'], value=[1, 2, 3], inplace=True)

这样,如果该col列具有分类值,则将它们替换为数值。

What I do is, I replace values.

Like this-

df['col'].replace(to_replace=['category_1', 'category_2', 'category_3'], value=[1, 2, 3], inplace=True)

In this way, if the col column has categorical values, they get replaced by the numerical values.


回答 7

对于特定的列,如果您不关心顺序,请使用此

df['col1_num'] = df['col1'].apply(lambda x: np.where(df['col1'].unique()==x)[0][0])

如果您关心订购,请将其指定为列表并使用它

df['col1_num'] = df['col1'].apply(lambda x: ['first', 'second', 'third'].index(x))

For a certain column, if you don’t care about the ordering, use this

df['col1_num'] = df['col1'].apply(lambda x: np.where(df['col1'].unique()==x)[0][0])

If you care about the ordering, specify them as a list and use this

df['col1_num'] = df['col1'].apply(lambda x: ['first', 'second', 'third'].index(x))