How to Convert DataFrame Column to Datetime in Pandas

  1. Pandas to_datetime function to convert DataFrame column to datetime
  2. DataFrame apply Method to Convert DataFrame Column to Datetime
  3. Methods to Convert DataFrame Column to Datetime Performance Comparison

We will introduce methods to convert Pandas DataFrame column to Python datetime.

  • Pandas to_datetime() function
  • DataFrame apply method to operate on elements in column

We will use the same DataFrame below in this article.

>>> from datetime import datetime, timedelta
>>> from pandas import DataFrame
>>> df = DataFrame.from_items(
    [('Alfa', [1, 2, 3]), 
     ('Bravo', [4, 5, 6]), 
     ('Datetime', [datetime.strftime(datetime.now()-timedelta(days=_), "%m/%d/%Y, %H:%M:%S") for _ in range(3)])], 
    orient='index', 
    columns=['A', 'B', 'C']).T
>>> df
  Alfa Bravo              Datetime
A    1     4  12/07/2019, 14:08:55
B    2     5  12/06/2019, 14:08:55
C    3     6  12/05/2019, 14:08:55

Pandas to_datetime function to convert DataFrame column to datetime

Pandas to_datetime function converts the given argument to datetime.

pandas.to_datetime(param, format="")

The format specifies the pattern of the datetime string. It is the same with format in stftime or strptime in Python datetime module.

>>> df
  Alfa Bravo              Datetime
A    1     4  12/07/2019, 14:08:55
B    2     5  12/06/2019, 14:08:55
C    3     6  12/05/2019, 14:08:55
>>> df['Datetime'] = pd.to_datetime(df['Datetime'], format="%m/%d/%Y, %H:%M:%S")
>>> df
  Alfa Bravo            Datetime
A    1     4 2019-12-07 14:08:55
B    2     5 2019-12-06 14:08:55
C    3     6 2019-12-05 14:08:55

to_datetime() function doesn’t modify the DataFrame data in-place, therefore we need to assign the returned Pandas Series to the specific DataFrame column.

to_datetime() Function Is Smart to Convert to Datetime

to_datetime() function could do the conversion to datetime in a smart way without given the datetime format string. It will find the string pattern automatically and smartly.

>>> df['Datetime'] = pd.to_datetime(df['Datetime'])
>>> df
  Alfa Bravo            Datetime
A    1     4 2019-12-07 14:08:55
B    2     5 2019-12-06 14:08:55
C    3     6 2019-12-05 14:08:55
Warning

Although to_datetime could do its job without giving the format smartly, the conversion speed is much lower than that when the format is given.

We could set the option infer_datetime_format of to_datetime to be True to switch the conversion to a faster mode if the format of the datetime string could be inferred without giving the format string.

It could increase the parsing speed by 5~6 times.

DataFrame apply Method to Convert DataFrame Column to Datetime

apply(func, *args, **kwds)

apply method of DataFrame applies function func to each column or row.

We could use lambda function in the place of func for the simplicity.

>>> df['Datetime'] = df['Datetime'].apply(lambda _: datetime.strptime(_,"%m/%d/%Y, %H:%M:%S"))
		   
>>> df
		   
  Alfa Bravo            Datetime
A    1     4 2019-12-07 14:44:35
B    2     5 2019-12-06 14:44:35
C    3     6 2019-12-05 14:44:35

Methods to Convert DataFrame Column to Datetime Performance Comparison

Let’s use timeit to compare the timing performance of both methods introduced in this article.

In [1]: %timeit pd.to_datetime(df['Datetime'], format="%m/%d/%Y, %H:%M:%S")
452 µs ± 85.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [2]: %timeit df['Datetime'].apply(lambda _: datetime.strptime(_,"%m/%d/%Y, %H:%M:%S"))
377 µs ± 58.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

The apply method is even slightly better than Pandas native to_datetime method, with around 80% of the execution time of to_datetime function.

Related Article - Pandas DataFrame

  • How to Get Pandas DataFrame Column Headers as a List
  • How to Delete Pandas DataFrame Column
  • Related Article - Pandas DataFrame Column

  • How to Get Pandas DataFrame Column Headers as a List
  • How to Delete Pandas DataFrame Column
  • How to Get Pandas DataFrame Column Headers as a List
  • How to Delete Pandas DataFrame Column
  • Related Article - Python datetime

  • How to Get Pandas DataFrame Column Headers as a List
  • How to Delete Pandas DataFrame Column
  • How to Get Pandas DataFrame Column Headers as a List
  • How to Delete Pandas DataFrame Column
  • How to Get the Current Time in Python
  • How to convert string to datetime in Python
  • comments powered by Disqus