标签归档：confidence-interval

知识问答

根据样本数据计算置信区间

问题：根据样本数据计算置信区间

我有一些样本数据，假设正态分布，我希望为它们计算一个置信区间。

我已经找到并安装了numpy和scipy软件包，并获得了numpy以返回均值和标准差（numpy.mean（data），其中data为列表）。任何关于获得样本置信区间的建议将不胜感激。

点击查看英文原文

I have sample data which I would like to compute a confidence interval for, assuming a normal distribution.

I have found and installed the numpy and scipy packages and have gotten numpy to return a mean and standard deviation (numpy.mean(data) with data being a list). Any advice on getting a sample confidence interval would be much appreciated.

回答 0

import numpy as np
import scipy.stats


def mean_confidence_interval(data, confidence=0.95):
    a = 1.0 * np.array(data)
    n = len(a)
    m, se = np.mean(a), scipy.stats.sem(a)
    h = se * scipy.stats.t.ppf((1 + confidence) / 2., n-1)
    return m, m-h, m+h

你可以这样计算

点击查看英文原文 
import numpy as np
import scipy.stats


def mean_confidence_interval(data, confidence=0.95):
    a = 1.0 * np.array(data)
    n = len(a)
    m, se = np.mean(a), scipy.stats.sem(a)
    h = se * scipy.stats.t.ppf((1 + confidence) / 2., n-1)
    return m, m-h, m+h

you can calculate like this way.

回答 1

这是shasan代码的简化版本，用于计算数组均值的95％置信区间a：

import numpy as np, scipy.stats as st

st.t.interval(0.95, len(a)-1, loc=np.mean(a), scale=st.sem(a))

但是使用StatsModels tconfint_mean可以说是更好的选择：

import statsmodels.stats.api as sms

sms.DescrStatsW(a).tconfint_mean()

两者的基本假设是，样本（数组a）是独立于具有未知标准偏差的正态分布绘制的（请参阅MathWorld或Wikipedia）。

对于大样本量n，样本均值是正态分布的，并且可以使用st.norm.interval()（如Jaime的评论中所建议的）计算其置信区间。但是上述解决方案对于较小的n也是正确的，n st.norm.interval()给出的置信区间太窄（即“假置信度”）。有关更多详细信息，请参阅我对类似问题的回答（以及此处的Russ的评论之一）。

这是一个示例，其中正确的选项给出（基本上）相同的置信区间：

In [9]: a = range(10,14)

In [10]: mean_confidence_interval(a)
Out[10]: (11.5, 9.4457397432391215, 13.554260256760879)

In [11]: st.t.interval(0.95, len(a)-1, loc=np.mean(a), scale=st.sem(a))
Out[11]: (9.4457397432391215, 13.554260256760879)

In [12]: sms.DescrStatsW(a).tconfint_mean()
Out[12]: (9.4457397432391197, 13.55426025676088)

最后，使用st.norm.interval()以下错误结果：

In [13]: st.norm.interval(0.95, loc=np.mean(a), scale=st.sem(a))
Out[13]: (10.23484868811834, 12.76515131188166)
点击查看英文原文

Here a shortened version of shasan’s code, calculating the 95% confidence interval of the mean of array a:

import numpy as np, scipy.stats as st

st.t.interval(0.95, len(a)-1, loc=np.mean(a), scale=st.sem(a))

But using StatsModels’ tconfint_mean is arguably even nicer:

import statsmodels.stats.api as sms

sms.DescrStatsW(a).tconfint_mean()

The underlying assumptions for both are that the sample (array a) was drawn independently from a normal distribution with unknown standard deviation (see MathWorld or Wikipedia).

For large sample size n, the sample mean is normally distributed, and one can calculate its confidence interval using st.norm.interval() (as suggested in Jaime’s comment). But the above solutions are correct also for small n, where st.norm.interval() gives confidence intervals that are too narrow (i.e., “fake confidence”). See my answer to a similar question for more details (and one of Russ’s comments here).

Here an example where the correct options give (essentially) identical confidence intervals:

In [9]: a = range(10,14)

In [10]: mean_confidence_interval(a)
Out[10]: (11.5, 9.4457397432391215, 13.554260256760879)

In [11]: st.t.interval(0.95, len(a)-1, loc=np.mean(a), scale=st.sem(a))
Out[11]: (9.4457397432391215, 13.554260256760879)

In [12]: sms.DescrStatsW(a).tconfint_mean()
Out[12]: (9.4457397432391197, 13.55426025676088)

And finally, the incorrect result using st.norm.interval():

In [13]: st.norm.interval(0.95, loc=np.mean(a), scale=st.sem(a))
Out[13]: (10.23484868811834, 12.76515131188166)

回答 2

首先从查找表中查找所需的置信区间的z值。置信区间为，其中是您的样本均值的估计标准偏差，由给出，其中是从样本数据计算出的标准偏差，是样本量。mean +/- z*sigmasigmasigma = s / sqrt(n)sn

点击查看英文原文

Start with looking up the z-value for your desired confidence interval from a look-up table. The confidence interval is then mean +/- z*sigma, where sigma is the estimated standard deviation of your sample mean, given by sigma = s / sqrt(n), where s is the standard deviation computed from your sample data and n is your sample size.

回答 3

从开始Python 3.8，标准库将NormalDist对象作为statistics模块的一部分提供：

from statistics import NormalDist

def confidence_interval(data, confidence=0.95):
  dist = NormalDist.from_samples(data)
  z = NormalDist().inv_cdf((1 + confidence) / 2.)
  h = dist.stdev * z / ((len(data) - 1) ** .5)
  return dist.mean - h, dist.mean + h

这个：

  • NormalDist从数据样本创建一个对象（NormalDist.from_samples(data)，使我们可以通过NormalDist.mean和访问样本的均值和标准差NormalDist.stdev。

  • 使用累积分布函数（）的反函数，针对给定的置信度，Z-score基于标准正态分布（用表示）计算。NormalDist()inv_cdf

  • 根据样本的标准偏差和平均值产生置信区间。

假设样本量足够大（可以超过100个点），以便使用标准正态分布而不是学生的t分布来计算z值。

点击查看英文原文

Starting Python 3.8, the standard library provides the NormalDist object as part of the statistics module:

from statistics import NormalDist

def confidence_interval(data, confidence=0.95):
  dist = NormalDist.from_samples(data)
  z = NormalDist().inv_cdf((1 + confidence) / 2.)
  h = dist.stdev * z / ((len(data) - 1) ** .5)
  return dist.mean - h, dist.mean + h

This:

  • Creates a NormalDist object from the data sample (NormalDist.from_samples(data), which gives us access to the sample’s mean and standard deviation via NormalDist.mean and NormalDist.stdev.

  • Compute the Z-score based on the standard normal distribution (represented by NormalDist()) for the given confidence using the inverse of the cumulative distribution function (inv_cdf).

  • Produces the confidence interval based on the sample’s standard deviation and mean.

This assumes the sample size is big enough (let’s say more than ~100 points) in order to use the standard normal distribution rather than the student’s t distribution to compute the z value.