使用pandas.to_datetime时仅保留日期部分

问题:使用pandas.to_datetime时仅保留日期部分

pandas.to_datetime用来解析数据中的日期。默认情况下,熊猫代表日期,datetime64[ns]即使所有日期都是每天也是如此。我想知道是否存在一种优雅/巧妙的方法来将日期转换为datetime.date或,datetime64[D]以便当我将数据写入CSV时,日期不附加00:00:00。我知道我可以手动逐个元素地转换类型:

[dt.to_datetime().date() for dt in df.dates]

但这确实很慢,因为我有很多行,这有点违反了使用目的pandas.to_datetime。有没有一种方法可以一次转换dtype整个列?或者,是否pandas.to_datetime支持精度规范,以便在处理日常数据时可以省去时间部分?

I use pandas.to_datetime to parse the dates in my data. Pandas by default represents the dates with datetime64[ns] even though the dates are all daily only. I wonder whether there is an elegant/clever way to convert the dates to datetime.date or datetime64[D] so that, when I write the data to CSV, the dates are not appended with 00:00:00. I know I can convert the type manually element-by-element:

[dt.to_datetime().date() for dt in df.dates]

But this is really slow since I have many rows and it sort of defeats the purpose of using pandas.to_datetime. Is there a way to convert the dtype of the entire column at once? Or alternatively, does pandas.to_datetime support a precision specification so that I can get rid of the time part while working with daily data?


回答 0

从版本开始,0.15.0现在可以轻松地通过.dt仅访问日期组件来完成此操作:

df['just_date'] = df['dates'].dt.date

上面的方法返回一个datetime.datedtype,如果您想要一个a,datetime64则可以normalize将时间分量设置为午夜,以便将所有值设置为00:00:00

df['normalised_date'] = df['dates'].dt.normalize()

这会使dtype保持不变,datetime64但显示屏仅显示该date值。

Since version 0.15.0 this can now be easily done using .dt to access just the date component:

df['just_date'] = df['dates'].dt.date

The above returns a datetime.date dtype, if you want to have a datetime64 then you can just normalize the time component to midnight so it sets all the values to 00:00:00:

df['normalised_date'] = df['dates'].dt.normalize()

This keeps the dtype as datetime64, but the display shows just the date value.


回答 1

简单的解决方案:

df['date_only'] = df['date_time_column'].dt.date

Simple Solution:

df['date_only'] = df['date_time_column'].dt.date

回答 2

虽然我赞成EdChum的答案,这是对OP提出的问题的最直接答案,但它并不能真正解决性能问题(它仍然依赖于python datetime对象,因此对它们的任何操作都不会被矢量化-即,它会很慢)。

性能更好的替代方法是使用df['dates'].dt.floor('d')。严格来说,它不会“仅保留日期部分”,因为它只是将时间设置为00:00:00。但是它确实可以按OP的要求运行,例如:

  • 打印到屏幕
  • 保存到csv
  • 使用列来 groupby

…并且效率更高,因为该操作已矢量化。

编辑:其实,在OP的宁愿答案很可能是“最近的版本pandas没有时间写为csv如果是00:00:00对所有的意见”。

While I upvoted EdChum’s answer, which is the most direct answer to the question the OP posed, it does not really solve the performance problem (it still relies on python datetime objects, and hence any operation on them will be not vectorized – that is, it will be slow).

A better performing alternative is to use df['dates'].dt.floor('d'). Strictly speaking, it does not “keep only date part”, since it just sets the time to 00:00:00. But it does work as desired by the OP when, for instance:

  • printing to screen
  • saving to csv
  • using the column to groupby

… and it is much more efficient, since the operation is vectorized.

EDIT: in fact, the answer the OP’s would have preferred is probably “recent versions of pandas do not write the time to csv if it is 00:00:00 for all observations”.


回答 3

熊猫DatetimeIndexSeries有一种方法normalize可以完全满足您的需求。

您可以在此答案中了解更多信息。

可以用作 ser.dt.normalize()

Pandas DatetimeIndex and Series have a method called normalize that does exactly what you want.

You can read more about it in this answer.

It can be used as ser.dt.normalize()


回答 4

熊猫v0.13 +:to_csvdate_format参数一起使用

尽可能避免将您的datetime64[ns]系列转换为objectdtype系列的datetime.date对象。后者通常使用构造pd.Series.dt.date,存储为指针数组,相对于基于NumPy的纯序列而言效率低下。

由于在写入CSV时您关注的是格式,因此只需使用date_format参数to_csv。例如:

df.to_csv(filename, date_format='%Y-%m-%d')

有关格式设置约定,请参见Python的strftime指令

Pandas v0.13+: Use to_csv with date_format parameter

Avoid, where possible, converting your datetime64[ns] series to an object dtype series of datetime.date objects. The latter, often constructed using pd.Series.dt.date, is stored as an array of pointers and is inefficient relative to a pure NumPy-based series.

Since your concern is format when writing to CSV, just use the date_format parameter of to_csv. For example:

df.to_csv(filename, date_format='%Y-%m-%d')

See Python’s strftime directives for formatting conventions.


回答 5

这是提取日期的简单方法:

import pandas as pd

d='2015-01-08 22:44:09' 
date=pd.to_datetime(d).date()
print(date)

This is a simple way to extract the date:

import pandas as pd

d='2015-01-08 22:44:09' 
date=pd.to_datetime(d).date()
print(date)

回答 6

转换为datetime64[D]

df.dates.values.astype('M8[D]')

尽管将其重新分配给DataFrame col将其恢复为[ns]。

如果您想要实际的datetime.date

dt = pd.DatetimeIndex(df.dates)
dates = np.array([datetime.date(*date_tuple) for date_tuple in zip(dt.year, dt.month, dt.day)])

Converting to datetime64[D]:

df.dates.values.astype('M8[D]')

Though re-assigning that to a DataFrame col will revert it back to [ns].

If you wanted actual datetime.date:

dt = pd.DatetimeIndex(df.dates)
dates = np.array([datetime.date(*date_tuple) for date_tuple in zip(dt.year, dt.month, dt.day)])

回答 7

如果有人看到此旧帖子,请给出一个最新的答案。

转换为日期时间时添加“ utc = False”将删除时区部分,仅将日期保留为datetime64 [ns]数据类型。

pd.to_datetime(df['Date'], utc=False)

您将能够将其保存在excel中,而不会出现错误“ ValueError:Excel不支持带时区的日期时间。在写入Excel之前,请确保日期时间不知道时区。”

Just giving a more up to date answer in case someone sees this old post.

Adding “utc=False” when converting to datetime will remove the timezone component and keep only the date in a datetime64[ns] data type.

pd.to_datetime(df['Date'], utc=False)

You will be able to save it in excel without getting the error “ValueError: Excel does not support datetimes with timezones. Please ensure that datetimes are timezone unaware before writing to Excel.”


回答 8

我希望能够更改数据框中一组列的类型,然后删除保持一天的时间。round(),floor(),ceil()全部工作

df[date_columns] = df[date_columns].apply(pd.to_datetime)
df[date_columns] = df[date_columns].apply(lambda t: t.dt.floor('d'))

I wanted to be able to change the type for a set of columns in a data frame and then remove the time keeping the day. round(), floor(), ceil() all work

df[date_columns] = df[date_columns].apply(pd.to_datetime)
df[date_columns] = df[date_columns].apply(lambda t: t.dt.floor('d'))