PY strptime very slow

Posted on
Fri Feb 13, 2015 11:39 pm
kw123 offline
User avatar
Posts: 8333
Joined: May 12, 2013
Location: Dallas, TX

PY strptime very slow

just FYI if you need to read a lot of date strings and need to do time calculations:


Code: Select all
tt= "20150101010101"
for i in range(100000):
   a=mktime(strptime(tt,"%Y%m%d%H%M%S"))
= 3.9 seconds

for i in range(100000):
    a=mktime(time.struct_time((int(tt[0:4]),int(tt[4:6]),int(tt[6:8]),int(tt[8:10]),int(tt[10:12]),int(tt[12:14]),0,0,0)))
= 1.7 seconds

for i in range(100000):
   a=datetime.strptime(tt[:12],'%Y%m%d%H%M')
2.7 seconds:

for i in range(100000):
   a=datetime(int(tt[0:4]),int(tt[4:6]),int(tt[6:8]),int(tt[8:10]),int(tt[10:12]))
0.6 seconds:


replacing "strptime" with simple mapping saves 50%-80% of execution time.
not a lot when you have to do this once or twice but dealing with 10,000,000 records makes a difference of several MINUTES of execution time

Karl

Page 1 of 1

Who is online

Users browsing this forum: No registered users and 6 guests