How to Convert DataFrame Column to Datetime in Pandas

1. Pandas to_datetime function to convert DataFrame column to datetime
2. DataFrame apply Method to Convert DataFrame Column to Datetime
3. Methods to Convert DataFrame Column to Datetime Performance Comparison

We will introduce methods to convert Pandas DataFrame column to Python datetime.

• Pandas to_datetime() function
• DataFrame apply method to operate on elements in column

We will use the same DataFrame below in this article.

>>> from datetime import datetime, timedelta
>>> from pandas import DataFrame
>>> df = DataFrame.from_items(
    [('Alfa', [1, 2, 3]), 
     ('Bravo', [4, 5, 6]), 
     ('Datetime', [datetime.strftime(datetime.now()-timedelta(days=_), "%m/%d/%Y, %H:%M:%S") for _ in range(3)])], 
    orient='index', 
    columns=['A', 'B', 'C']).T
>>> df
  Alfa Bravo              Datetime
A    1     4  12/07/2019, 14:08:55
B    2     5  12/06/2019, 14:08:55
C    3     6  12/05/2019, 14:08:55

Pandas to_datetime function to convert DataFrame column to datetime

Pandas to_datetime function converts the given argument to datetime.

pandas.to_datetime(param, format="")

The format specifies the pattern of the datetime string. It is the same with format in stftime or strptime in Python datetime module.

>>> df
  Alfa Bravo              Datetime
A    1     4  12/07/2019, 14:08:55
B    2     5  12/06/2019, 14:08:55
C    3     6  12/05/2019, 14:08:55
>>> df['Datetime'] = pd.to_datetime(df['Datetime'], format="%m/%d/%Y, %H:%M:%S")
>>> df
  Alfa Bravo            Datetime
A    1     4 2019-12-07 14:08:55
B    2     5 2019-12-06 14:08:55
C    3     6 2019-12-05 14:08:55

to_datetime() function doesn’t modify the DataFrame data in-place, therefore we need to assign the returned Pandas Series to the specific DataFrame column.

to_datetime() Function Is Smart to Convert to Datetime

to_datetime() function could do the conversion to datetime in a smart way without given the datetime format string. It will find the string pattern automatically and smartly.

>>> df['Datetime'] = pd.to_datetime(df['Datetime'])
>>> df
  Alfa Bravo            Datetime
A    1     4 2019-12-07 14:08:55
B    2     5 2019-12-06 14:08:55
C    3     6 2019-12-05 14:08:55

DataFrame apply Method to Convert DataFrame Column to Datetime

apply(func, *args, **kwds)

apply method of DataFrame applies function func to each column or row.

We could use lambda function in the place of func for the simplicity.

>>> df['Datetime'] = df['Datetime'].apply(lambda _: datetime.strptime(_,"%m/%d/%Y, %H:%M:%S"))
		   
>>> df
		   
  Alfa Bravo            Datetime
A    1     4 2019-12-07 14:44:35
B    2     5 2019-12-06 14:44:35
C    3     6 2019-12-05 14:44:35

Methods to Convert DataFrame Column to Datetime Performance Comparison

Let’s use timeit to compare the timing performance of both methods introduced in this article.

In [1]: %timeit pd.to_datetime(df['Datetime'], format="%m/%d/%Y, %H:%M:%S")
452 µs ± 85.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [2]: %timeit df['Datetime'].apply(lambda _: datetime.strptime(_,"%m/%d/%Y, %H:%M:%S"))
377 µs ± 58.5 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

The apply method is even slightly better than Pandas native to_datetime method, with around 80% of the execution time of to_datetime function.

Related Article - Pandas DataFrame

Related Article - Pandas DataFrame Column

Related Article - Python datetime