sklearn中的“ transform”和“ fit_transform”有什么区别

问题:sklearn中的“ transform”和“ fit_transform”有什么区别

在sklearn-python工具箱中,有两个函数transformfit_transformabout sklearn.decomposition.RandomizedPCA。两种功能的说明如下

但是它们之间有什么区别?

In the sklearn-python toolbox, there are two functions transform and fit_transform about sklearn.decomposition.RandomizedPCA. The description of two functions are as follows

But what is the difference between them ?


回答 0

在这里,仅当您已经在矩阵上计算了PCA时,才可以使用pca.transform的区别

   In [12]: pc2 = RandomizedPCA(n_components=3)

    In [13]: pc2.transform(X) # can't transform because it does not know how to do it.
    ---------------------------------------------------------------------------
    AttributeError                            Traceback (most recent call last)
    <ipython-input-13-e3b6b8ea2aff> in <module>()
    ----> 1 pc2.transform(X)

    /usr/local/lib/python3.4/dist-packages/sklearn/decomposition/pca.py in transform(self, X, y)
        714         # XXX remove scipy.sparse support here in 0.16
        715         X = atleast2d_or_csr(X)
    --> 716         if self.mean_ is not None:
        717             X = X - self.mean_
        718 

    AttributeError: 'RandomizedPCA' object has no attribute 'mean_'

    In [14]: pc2.ftransform(X) 
    pc2.fit            pc2.fit_transform  

    In [14]: pc2.fit_transform(X)
    Out[14]: 
    array([[-1.38340578, -0.2935787 ],
           [-2.22189802,  0.25133484],
           [-3.6053038 , -0.04224385],
           [ 1.38340578,  0.2935787 ],
           [ 2.22189802, -0.25133484],
           [ 3.6053038 ,  0.04224385]])

如果要使用.transform,则需要将转换规则教给您的pca

In [20]: pca = RandomizedPCA(n_components=3)

In [21]: pca.fit(X)
Out[21]: 
RandomizedPCA(copy=True, iterated_power=3, n_components=3, random_state=None,
       whiten=False)

In [22]: pca.transform(z)
Out[22]: 
array([[ 2.76681156,  0.58715739],
       [ 1.92831932,  1.13207093],
       [ 0.54491354,  0.83849224],
       [ 5.53362311,  1.17431479],
       [ 6.37211535,  0.62940125],
       [ 7.75552113,  0.92297994]])

In [23]: 

特别地,PCA变换将通过矩阵X的PCA分解获得的基数更改应用于矩阵Z。

The .transform method is meant for when you have already computed PCA, i.e. if you have already called its .fit method.

In [12]: pc2 = RandomizedPCA(n_components=3)

In [13]: pc2.transform(X) # can't transform because it does not know how to do it.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-13-e3b6b8ea2aff> in <module>()
----> 1 pc2.transform(X)

/usr/local/lib/python3.4/dist-packages/sklearn/decomposition/pca.py in transform(self, X, y)
    714         # XXX remove scipy.sparse support here in 0.16
    715         X = atleast2d_or_csr(X)
--> 716         if self.mean_ is not None:
    717             X = X - self.mean_
    718 

AttributeError: 'RandomizedPCA' object has no attribute 'mean_'

In [14]: pc2.ftransform(X) 
pc2.fit            pc2.fit_transform  

In [14]: pc2.fit_transform(X)
Out[14]: 
array([[-1.38340578, -0.2935787 ],
       [-2.22189802,  0.25133484],
       [-3.6053038 , -0.04224385],
       [ 1.38340578,  0.2935787 ],
       [ 2.22189802, -0.25133484],
       [ 3.6053038 ,  0.04224385]])
    
  

So you want to fit RandomizedPCA and then transform as:

In [20]: pca = RandomizedPCA(n_components=3)

In [21]: pca.fit(X)
Out[21]: 
RandomizedPCA(copy=True, iterated_power=3, n_components=3, random_state=None,
       whiten=False)

In [22]: pca.transform(z)
Out[22]: 
array([[ 2.76681156,  0.58715739],
       [ 1.92831932,  1.13207093],
       [ 0.54491354,  0.83849224],
       [ 5.53362311,  1.17431479],
       [ 6.37211535,  0.62940125],
       [ 7.75552113,  0.92297994]])

In [23]: 

In particular PCA .transform applies the change of basis obtained through the PCA decomposition of the matrix X to the matrix Z.


回答 1

scikit-learn estimator api中

fit() :用于从训练数据生成学习模型参数

transform():从fit()方法生成的参数,应用于模型以生成转换后的数据集。

fit_transform()fit()transform()api在同一数据集上的组合

结帐章-4从这本书及答案从stackexchange为了更清楚

In scikit-learn estimator api,

fit() : used for generating learning model parameters from training data

transform() : parameters generated from fit() method,applied upon model to generate transformed data set.

fit_transform() : combination of fit() and transform() api on same data set

Checkout Chapter-4 from this book & answer from stackexchange for more clarity


回答 2

这些方法用于确定给定数据的中心/特征标度。它基本上有助于规范特定范围内的数据

为此,我们使用Z评分方法。

我们在训练数据集上执行此操作。

1. Fit():方法计算参数μ和σ并将其保存为内部对象。

2. Transform():使用这些计算出的参数的方法将转换应用于特定的数据集。

3. Fit_transform():将fit()和transform()方法结合在一起以进行数据集转换。

特征缩放/标准化的代码片段(在train_test_split之后)。

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
sc.fit_transform(X_train)
sc.transform(X_test)

我们在测试集上应用相同的(训练集相同的两个参数μ和σ(值))参数转换。

These methods are used to center/feature scale of a given data. It basically helps to normalize the data within a particular range

For this, we use Z-score method.

We do this on the training set of data.

1.Fit(): Method calculates the parameters μ and σ and saves them as internal objects.

2.Transform(): Method using these calculated parameters apply the transformation to a particular dataset.

3.Fit_transform(): joins the fit() and transform() method for transformation of dataset.

Code snippet for Feature Scaling/Standardisation(after train_test_split).

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
sc.fit_transform(X_train)
sc.transform(X_test)

We apply the same(training set same two parameters μ and σ (values)) parameter transformation on our testing set.


回答 3

两种方法之间的通用差异:

  • 适合(raw_documents [,y]):学习原始文档中所有标记的词汇词典。
  • fit_transform(raw_documents [,y]):学习词汇词典并返回术语文档矩阵。这等效于紧随其后的变换,但实现效率更高。
  • transform(raw_documents):将文档转换为文档术语矩阵。使用适合的词汇表或提供给构造函数的词汇表从原始文本文档中提取令牌计数。

fit_transform和transform都返回相同的Document-term矩阵。

资源

Generic difference between the methods:

  • fit(raw_documents[, y]): Learn a vocabulary dictionary of all tokens in the raw documents.
  • fit_transform(raw_documents[, y]): Learn the vocabulary dictionary and return term-document matrix. This is equivalent to fit followed by the transform, but more efficiently implemented.
  • transform(raw_documents): Transform documents to document-term matrix. Extract token counts out of raw text documents using the vocabulary fitted with fit or the one provided to the constructor.

Both fit_transform and transform returns the same, Document-term matrix.

Source


回答 4

这里.fit()&的基本区别.fit_transform()

。适合():

在有两个对象/参数(x,y)的监督学习中用于拟合模型并使模型运行,我们知道我们要预测的内容

.fit_transform():

在无监督学习中使用一个对象/参数(x),而我们不知道该预测什么。

Here the basic difference between .fit() & .fit_transform():

.fit():

is use in the Supervised learning having two object/parameter(x,y) to fit model and make model to run, where we know that what we are going to predict

.fit_transform():

is use in Unsupervised Learning having one object/parameter(x), where we don’t know, what we are going to predict.


回答 5

用外行的术语来说,fit_transform意味着先进行一些计算,然后再进行转换(例如,根据一些数据计算列的均值,然后替换缺少的值)。因此,对于训练集,您既需要计算又要进行转换。

但是对于测试集,机器学习根据训练集中学习到的内容应用预测,因此不需要计算,只需执行转换即可。

In layman’s terms, fit_transform means to do some calculation and then do transformation (say calculating the means of columns from some data and then replacing the missing values). So for training set, you need to both calculate and do transformation.

But for testing set, Machine learning applies prediction based on what was learned during the training set and so it doesn’t need to calculate, it just performs the transformation.


回答 6

为什么和何时使用每一个:

所有答复都很好,但是我将重点介绍为什么和何时使用每种方法。

fit(),transform(),fit_transform()

通常,我们有一个监督学习问题,其中(X,y)作为数据集,我们将其分为训练数据和测试数据:

import numpy as np
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y)

X_train_vectorized = model.fit_transform(X_train)
X_test_vectorized = model.transform(X_test)

想象一下,我们正在安装一个令牌生成器,如果我们安装了X,我们会将测试数据包含到令牌生成器中,但是我已经多次看到这个错误!

正确的方法是只适合X_train,因为您不知道“您的未来数据”,因此您不能使用X_test数据来拟合任何东西!

然后,您可以转换测试数据,但是要分别进行转换,这就是为什么存在不同方法的原因。

最后的提示:X_train_transformed = model.fit_transform(X_train)等同于: X_train_transformed = model.fit(X_train).transform(X_train),但是第一个提示更快。

请注意,我所谓的“模型”通常是缩放器,tfidf转换器,其他类型的矢量化器,令牌化器…

Why and When use each one:

All the responses are quite good, but I would make emphasis in WHY and WHEN use each method.

fit(), transform(), fit_transform()

Usually we have a supervised learning problem with (X, y) as out dataset, and we split it into training data and test data:

import numpy as np
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y)

X_train_vectorized = model.fit_transform(X_train)
X_test_vectorized = model.transform(X_test)

Imagine we are fitting a tokenizer, if we fit X we are including testing data into the tokenizer, but I have seen this error many times!

The correct is to fit ONLY with X_train, because you don’t know “your future data” so you cannot use X_test data for fitting anything!

Then you can transform your test data, but separately, that’s why there are different methods.

Final tip: X_train_transformed = model.fit_transform(X_train) is equivalent to: X_train_transformed = model.fit(X_train).transform(X_train), but the first one is faster.

Note that what I call “model” usually will be a scaler, a tfidf transformer, other kind of vectorizer, a tokenizer…