Python性能调试笔记

::-- Roka [DateTime(2007-04-26T14:46:55Z)] TableOfContents

1. String Concatenation

(1)

Normal Code: 

   1 s = ""
   2 for substring in list:
   3     s += substring

Optimized Code: 

   1 s = "".join(list)

(2)

Normal Code: 

   1 s = ""
   2 for x in list:
   3     s += someFunction(x)

Optimized Code: 

   1 slist = [someFunction(x) for x in somelist]
   2 s = "".join(slist)

(3)

Normal Code: 

   1 out = "<html>" + head + prologue + query + tail + "</html>"

Optimized Code: 

   1 out = "<html>%(head)s%(prologue)s%(query)s%(tail)s</html>" % locals()

1.1. Loops

(1)Converting to upper case:

Normal Code: 

   1 newlist = []
   2 for word in oldlist:
   3     newlist.append(word.upper())

Optimized Code: 

   1 #(map() is fast but will be removed from Py3000)
   2 newlist = map(str.upper, oldlist)
   3 #Or(List comprehensions, Py > 2.0)
   4 newlist = [s.upper() for s in oldlist]
   5 #Or(Generator expressions, Py > 2.4)
   6 newlist = (s.upper() for s in oldlist)

1.2. OOP

(1)Suppose you cannot use map() or list comprehension, just remember Avoiding dots: 

   1 upper = str.upper
   2 newlist = []
   3 append = newlist.append
   4 # loop without dots
   5 for word in list:
   6     append(upper(word))

1.3. Local Variables

(1)Final speedup method is to use local instead of global vars. 

   1 def func():
   2     upper = str.upper
   3     newlist = []
   4     append = newlist.append
   5     for word in words:
   6         append(upper(word))
   7     return newlist

1.4. Dictionary

(1)Avoid if in for loops:

Normal Code: 

   1 wdict= {}
   2 for word in words:
   3     if word not in wdict:
   4         wdict[word] = 0
   5     wdict[word] += 1

Optimized Code: 

   1 #(Py < 2.x)
   2 wdict = {}
   3 for word in words:
   4     try:
   5         wdict[word] += 1
   6     except KeyError:
   7         wdict[word] = 1
   8 
   9 #(Py > 2.x)
  10 wdict = {}
  11 get = wdict.get
  12 for word in words:
  13     wdict[word] = get(word, 0) + 1

Also , if the value stored in the dict is an object or a list, you could also use the dict.setdefault method, e.g. 

   1 wdict.setdefault(key, []).append(newElement)

This avoids having to lookup the twice.

1.5. Import

(1)import inside the function is more efficiently.

(2)Do import once, 

   1 #check
   2 pack = None
   3 
   4 def parse_pack():
   5     global pack
   6     if pack is None:
   7         import pack
   8     ...

1.6. Data Aggregation

(1)Avoiding function call in for loop

Normal Code: 

   1 import time
   2 x = 0
   3 def doit(i):
   4     global x
   5     x = x + 1
   6 
   7 list = range(100000)
   8 t = time.time()
   9 for i in list:
  10     doit(i)
  11 
  12 print "%.3f" %(time.time() -t )

Optimized Code: 

   1 import time
   2 x = 0
   3 def doit(i):
   4     global x
   5     for i in list:
   6         x = x + 1
   7     x = x + 1
   8 
   9 list = range(100000)
  10 t = time.time()
  11 doit(list)
  12 
  13 print "%.3f" %(time.time() -t )

(What?? about 4 times faster!! )

1.7. range() -> xrange()

It is implemented in Pure C.

2. 交流

PageComment2