Skip to content

Calling hotshot, a python profiler, from command line

Python Logo

some script I looked for some convenient way to call hotshot from command line just like it is possible with profile \ cProfile out-of-the-box.

Since I couldn’t find any, I wrote this simple wrapper:

import hotshot, hotshot.stats
import os
import sys


if __name__ == "__main__":
    if len(sys.argv) == 0:
        sys.exit()
    
    
    if len(sys.argv) == 3:
        script = sys.argv[2]
        repetitions = int(sys.argv[1])
    else:
        script = sys.argv[1]
        repetitions = 1
    
    # stolen from profile.py
    # add dirname to python-search-path
    dirname = os.path.dirname(script)
    sys.path.insert(0, dirname)
    
    hotshot_data = "hotshot_hotcoc_stats"
    
    profile = hotshot.Profile(hotshot_data)
    # execute the script - instead of "%r" - "'%s'" would also work
    for i in range(repetitions):
        profile.run("execfile(%r)" % (script,))
    profile.close()
    
    stats = hotshot.stats.load(hotshot_data)
    # removing directory-information
    # time: time spent in function without sub-calls
    stats = stats.sort_stats('time', 'calls')
    stats.print_stats()

Dropping this as „hotcoc.py“ (or whatever you like) in your path allows something like

python -m hotcoc 5 myscript.py

which times 5 repetitions of myscript.py using hotshot.

I just noticed that hotshot seems to be deprecated. The script can be used to wrap profile or cProfile, too, so that they are provided with your favored options.

As a side note

In the script I profiled I was using something like

if key not in my_dict:
    my_dict[key] = []
my_dict[key].append(some_data)

Reading Santiago Gala’s Python, Erlang, Map/Reduce I stumpled across collections.defaultdict and decided to use it:

my_dict = defaultdict(list)
my_dict[key].append(some_data)

Using defaultdict if a key is not found in the dictionary the factory-function provided with the constructor is called to get a default value for a key in the dict. In the example an empty list is used.

Timing this I found the implementation using defaultdict is slower by about a quarter (which supported my results under PythonIdoms#exceptions.

This may be due to most of the keys not being present in the dict for my problem.

So, defaultdict makes the code more readible, more elegant but not always faster (or equally ‚fast‘).

For the script at hand I switched back to the ‚key in dict‘-implementation.

{ 1 } Trackback

  1. […] Post Calling hotshot, a python profiler, from command line habe ich zur Demonstration im Nachhinein Highlighting aktiviert. Related […]

Post a Comment

Your email is never published nor shared. Required fields are marked *