The Python Profilers
********************

**Source code:** Lib/profile.py and Lib/pstats.py

======================================================================


Introduction to the profilers
=============================

"cProfile" and "profile" provide *deterministic profiling* of Python
programs. A *profile* is a set of statistics that describes how often
and for how long various parts of the program executed. These
statistics can be formatted into reports via the "pstats" module.

The Python standard library provides two different implementations of
the same profiling interface:

1. "cProfile" is recommended for most users; it's a C extension with
   reasonable overhead that makes it suitable for profiling long-
   running programs.  Based on "lsprof", contributed by Brett Rosen
   and Ted Czotter.

2. "profile", a pure Python module whose interface is imitated by
   "cProfile", but which adds significant overhead to profiled
   programs. If you're trying to extend the profiler in some way, the
   task might be easier with this module.  Originally designed and
   written by Jim Roskind.

备注:

  The profiler modules are designed to provide an execution profile
  for a given program, not for benchmarking purposes (for that, there
  is "timeit" for reasonably accurate results).  This particularly
  applies to benchmarking Python code against C code: the profilers
  introduce overhead for Python code, but not for C-level functions,
  and so the C code would seem faster than any Python one.


Instant User's Manual
=====================

This section is provided for users that "don't want to read the
manual." It provides a very brief overview, and allows a user to
rapidly perform profiling on an existing application.

To profile a function that takes a single argument, you can do:

   import cProfile
   import re
   cProfile.run('re.compile("foo|bar")')

(Use "profile" instead of "cProfile" if the latter is not available on
your system.)

The above action would run "re.compile()" and print profile results
like the following:

         214 function calls (207 primitive calls) in 0.002 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.002    0.002 {built-in method builtins.exec}
        1    0.000    0.000    0.001    0.001 <string>:1(<module>)
        1    0.000    0.000    0.001    0.001 __init__.py:250(compile)
        1    0.000    0.000    0.001    0.001 __init__.py:289(_compile)
        1    0.000    0.000    0.000    0.000 _compiler.py:759(compile)
        1    0.000    0.000    0.000    0.000 _parser.py:937(parse)
        1    0.000    0.000    0.000    0.000 _compiler.py:598(_code)
        1    0.000    0.000    0.000    0.000 _parser.py:435(_parse_sub)

The first line indicates that 214 calls were monitored.  Of those
calls, 207 were *primitive*, meaning that the call was not induced via
recursion. The next line: "Ordered by: cumulative time" indicates the
output is sorted by the "cumtime" values. The column headings include:

ncalls
   for the number of calls.

tottime
   for the total time spent in the given function (and excluding time
   made in calls to sub-functions)

percall
   is the quotient of "tottime" divided by "ncalls"

cumtime
   is the cumulative time spent in this and all subfunctions (from
   invocation till exit). This figure is accurate *even* for recursive
   functions.

percall
   is the quotient of "cumtime" divided by primitive calls

filename:lineno(function)
   provides the respective data of each function

When there are two numbers in the first column (for example "3/1"), it
means that the function recursed.  The second value is the number of
primitive calls and the former is the total number of calls.  Note
that when the function does not recurse, these two values are the
same, and only the single figure is printed.

Instead of printing the output at the end of the profile run, you can
save the results to a file by specifying a filename to the "run()"
function:

   import cProfile
   import re
   cProfile.run('re.compile("foo|bar")', 'restats')

The "pstats.Stats" class reads profile results from a file and formats
them in various ways.

The files "cProfile" and "profile" can also be invoked as a script to
profile another script.  For example:

   python -m cProfile [-o output_file] [-s sort_order] (-m module | myscript.py)

-o <output_file>

   Writes the profile results to a file instead of to stdout.

-s <sort_order>

   Specifies one of the "sort_stats()" sort values to sort the output
   by. This only applies when "-o" is not supplied.

-m <module>

   Specifies that a module is being profiled instead of a script.

   Added in version 3.7: Added the "-m" option to "cProfile".

   Added in version 3.8: Added the "-m" option to "profile".

The "pstats" module's "Stats" class has a variety of methods for
manipulating and printing the data saved into a profile results file:

   import pstats
   from pstats import SortKey
   p = pstats.Stats('restats')
   p.strip_dirs().sort_stats(-1).print_stats()

The "strip_dirs()" method removed the extraneous path from all the
module names. The "sort_stats()" method sorted all the entries
according to the standard module/line/name string that is printed. The
"print_stats()" method printed out all the statistics.  You might try
the following sort calls:

   p.sort_stats(SortKey.NAME)
   p.print_stats()

The first call will actually sort the list by function name, and the
second call will print out the statistics.  The following are some
interesting calls to experiment with:

   p.sort_stats(SortKey.CUMULATIVE).print_stats(10)

This sorts the profile by cumulative time in a function, and then only
prints the ten most significant lines.  If you want to understand what
algorithms are taking time, the above line is what you would use.

If you were looking to see what functions were looping a lot, and
taking a lot of time, you would do:

   p.sort_stats(SortKey.TIME).print_stats(10)

to sort according to time spent within each function, and then print
the statistics for the top ten functions.

You might also try:

   p.sort_stats(SortKey.FILENAME).print_stats('__init__')

This will sort all the statistics by file name, and then print out
statistics for only the class init methods (since they are spelled
with "__init__" in them).  As one final example, you could try:

   p.sort_stats(SortKey.TIME, SortKey.CUMULATIVE).print_stats(.5, 'init')

This line sorts statistics with a primary key of time, and a secondary
key of cumulative time, and then prints out some of the statistics. To
be specific, the list is first culled down to 50% (re: ".5") of its
original size, then only lines containing "init" are maintained, and
that sub-sub-list is printed.

If you wondered what functions called the above functions, you could
now ("p" is still sorted according to the last criteria) do:

   p.print_callers(.5, 'init')

and you would get a list of callers for each of the listed functions.

If you want more functionality, you're going to have to read the
manual, or guess what the following functions do:

   p.print_callees()
   p.add('restats')

Invoked as a script, the "pstats" module is a statistics browser for
reading and examining profile dumps.  It has a simple line-oriented
interface (implemented using "cmd") and interactive help.


"profile" and "cProfile" Module Reference
=========================================

Both the "profile" and "cProfile" modules provide the following
functions:

profile.run(command, filename=None, sort=-1)

   此函数接受一个可被传递给 "exec()" 函数的单独参数，以及一个可选的文
   件名。在所有情况下这个例程都会执行:

      exec(command, __main__.__dict__, __main__.__dict__)

   并收集执行过程中的性能分析统计数据。如果未提供文件名，则此函数会自
   动创建一个 "Stats" 实例并打印一个简单的性能分析报告。如果指定了
   sort 值，则它会被传递给这个 "Stats" 实例以控制结果的排序方式。

profile.runctx(command, globals, locals, filename=None, sort=-1)

   此函数类似于 "run()"，带有为 *command* 字符串提供 globals 和 locals
   映射对象的附加参数。这个例程会执行:

      exec(command, globals, locals)

   并像在上述的 "run()" 函数中一样收集性能分析数据。

class profile.Profile(timer=None, timeunit=0.0, subcalls=True, builtins=True)

   This class is normally only used if more precise control over
   profiling is needed than what the "cProfile.run()" function
   provides.

   可以通过 *timer* 参数提供一个自定义计时器来测量代码运行花费了多长时
   间。它必须是一个返回代表当前时间的单个数字的函数。如果该数字为整数
   ，则 *timeunit* 指定一个表示每个时间单位持续时间的乘数。例如，如果
   定时器返回以千秒为计量单位的时间值，则时间单位将为 ".001"。

   直接使用 "Profile" 类将允许格式化性能分析结果而无需将性能分析数据写
   入到文件:

      import cProfile, pstats, io
      from pstats import SortKey
      pr = cProfile.Profile()
      pr.enable()
      # ... do something ...
      pr.disable()
      s = io.StringIO()
      sortby = SortKey.CUMULATIVE
      ps = pstats.Stats(pr, stream=s).sort_stats(sortby)
      ps.print_stats()
      print(s.getvalue())

   The "Profile" class can also be used as a context manager
   (supported only in "cProfile" module. see 上下文管理器类型):

      import cProfile

      with cProfile.Profile() as pr:
          # ... do something ...

          pr.print_stats()

   在 3.8 版本发生变更: 添加了上下文管理器支持。

   enable()

      Start collecting profiling data. Only in "cProfile".

   disable()

      Stop collecting profiling data. Only in "cProfile".

   create_stats()

      停止收集分析数据，并在内部将结果记录为当前 profile。

   print_stats(sort=-1)

      根据当前性能分析数据创建一个 "Stats" 对象并将结果打印到 stdout。

      The *sort* parameter specifies the sorting order of the
      displayed statistics. It accepts a single key or a tuple of keys
      to enable multi-level sorting, as in "Stats.sort_stats".

      Added in version 3.13: 现在 "print_stats()" 可接受一个由键组成的
      元组。

   dump_stats(filename)

      将当前 profile 的结果写入 *filename* 。

   run(cmd)

      通过 "exec()" 对该命令进行性能分析。

   runctx(cmd, globals, locals)

      通过 "exec()" 并附带指定的全局和局部环境对该命令进行性能分析。

   runcall(func, /, *args, **kwargs)

      对 "func(*args, **kwargs)" 进行性能分析

请注意性能分析只有在被调用的命令/函数确实能返回时才可用。如果解释器被
终结（例如在被调用的命令/函数执行期间通过 "sys.exit()" 调用）则将不会
打印性能分析结果。


The "Stats" Class
=================

Analysis of the profiler data is done using the "Stats" class.

class pstats.Stats(*filenames or profile, stream=sys.stdout)

   This class constructor creates an instance of a "statistics object"
   from a *filename* (or list of filenames) or from a "Profile"
   instance. Output will be printed to the stream specified by
   *stream*.

   The file selected by the above constructor must have been created
   by the corresponding version of "profile" or "cProfile".  To be
   specific, there is *no* file compatibility guaranteed with future
   versions of this profiler, and there is no compatibility with files
   produced by other profilers, or the same profiler run on a
   different operating system.  If several files are provided, all the
   statistics for identical functions will be coalesced, so that an
   overall view of several processes can be considered in a single
   report.  If additional files need to be combined with data in an
   existing "Stats" object, the "add()" method can be used.

   Instead of reading the profile data from a file, a
   "cProfile.Profile" or "profile.Profile" object can be used as the
   profile data source.

   "Stats" objects have the following methods:

   strip_dirs()

      This method for the "Stats" class removes all leading path
      information from file names.  It is very useful in reducing the
      size of the printout to fit within (close to) 80 columns.  This
      method modifies the object, and the stripped information is
      lost.  After performing a strip operation, the object is
      considered to have its entries in a "random" order, as it was
      just after object initialization and loading. If "strip_dirs()"
      causes two function names to be indistinguishable (they are on
      the same line of the same filename, and have the same function
      name), then the statistics for these two entries are accumulated
      into a single entry.

   add(*filenames)

      This method of the "Stats" class accumulates additional
      profiling information into the current profiling object.  Its
      arguments should refer to filenames created by the corresponding
      version of "profile.run()" or "cProfile.run()". Statistics for
      identically named (re: file, line, name) functions are
      automatically accumulated into single function statistics.

   dump_stats(filename)

      Save the data loaded into the "Stats" object to a file named
      *filename*.  The file is created if it does not exist, and is
      overwritten if it already exists.  This is equivalent to the
      method of the same name on the "profile.Profile" and
      "cProfile.Profile" classes.

   sort_stats(*keys)

      This method modifies the "Stats" object by sorting it according
      to the supplied criteria.  The argument can be either a string
      or a SortKey enum identifying the basis of a sort (example:
      "'time'", "'name'", "SortKey.TIME" or "SortKey.NAME"). The
      SortKey enums argument have advantage over the string argument
      in that it is more robust and less error prone.

      When more than one key is provided, then additional keys are
      used as secondary criteria when there is equality in all keys
      selected before them.  For example, "sort_stats(SortKey.NAME,
      SortKey.FILE)" will sort all the entries according to their
      function name, and resolve all ties (identical function names)
      by sorting by file name.

      For the string argument, abbreviations can be used for any key
      names, as long as the abbreviation is unambiguous.

      The following are the valid string and SortKey:

      +--------------------+-----------------------+------------------------+
      | Valid String Arg   | Valid enum Arg        | Meaning                |
      |====================|=======================|========================|
      | "'calls'"          | SortKey.CALLS         | call count             |
      +--------------------+-----------------------+------------------------+
      | "'cumulative'"     | SortKey.CUMULATIVE    | cumulative time        |
      +--------------------+-----------------------+------------------------+
      | "'cumtime'"        | N/A                   | cumulative time        |
      +--------------------+-----------------------+------------------------+
      | "'file'"           | N/A                   | file name              |
      +--------------------+-----------------------+------------------------+
      | "'filename'"       | SortKey.FILENAME      | file name              |
      +--------------------+-----------------------+------------------------+
      | "'module'"         | N/A                   | file name              |
      +--------------------+-----------------------+------------------------+
      | "'ncalls'"         | N/A                   | call count             |
      +--------------------+-----------------------+------------------------+
      | "'pcalls'"         | SortKey.PCALLS        | primitive call count   |
      +--------------------+-----------------------+------------------------+
      | "'line'"           | SortKey.LINE          | line number            |
      +--------------------+-----------------------+------------------------+
      | "'name'"           | SortKey.NAME          | function name          |
      +--------------------+-----------------------+------------------------+
      | "'nfl'"            | SortKey.NFL           | name/file/line         |
      +--------------------+-----------------------+------------------------+
      | "'stdname'"        | SortKey.STDNAME       | standard name          |
      +--------------------+-----------------------+------------------------+
      | "'time'"           | SortKey.TIME          | internal time          |
      +--------------------+-----------------------+------------------------+
      | "'tottime'"        | N/A                   | internal time          |
      +--------------------+-----------------------+------------------------+

      Note that all sorts on statistics are in descending order
      (placing most time consuming items first), where as name, file,
      and line number searches are in ascending order (alphabetical).
      The subtle distinction between "SortKey.NFL" and
      "SortKey.STDNAME" is that the standard name is a sort of the
      name as printed, which means that the embedded line numbers get
      compared in an odd way.  For example, lines 3, 20, and 40 would
      (if the file names were the same) appear in the string order 20,
      3 and 40. In contrast, "SortKey.NFL" does a numeric compare of
      the line numbers. In fact, "sort_stats(SortKey.NFL)" is the same
      as "sort_stats(SortKey.NAME, SortKey.FILENAME, SortKey.LINE)".

      For backward-compatibility reasons, the numeric arguments "-1",
      "0", "1", and "2" are permitted.  They are interpreted as
      "'stdname'", "'calls'", "'time'", and "'cumulative'"
      respectively.  If this old style format (numeric) is used, only
      one sort key (the numeric key) will be used, and additional
      arguments will be silently ignored.

      Added in version 3.7: Added the SortKey enum.

   reverse_order()

      This method for the "Stats" class reverses the ordering of the
      basic list within the object.  Note that by default ascending vs
      descending order is properly selected based on the sort key of
      choice.

   print_stats(*restrictions)

      This method for the "Stats" class prints out a report as
      described in the "profile.run()" definition.

      The order of the printing is based on the last "sort_stats()"
      operation done on the object (subject to caveats in "add()" and
      "strip_dirs()").

      The arguments provided (if any) can be used to limit the list
      down to the significant entries.  Initially, the list is taken
      to be the complete set of profiled functions.  Each restriction
      is either an integer (to select a count of lines), or a decimal
      fraction between 0.0 and 1.0 inclusive (to select a percentage
      of lines), or a string that will be interpreted as a regular
      expression (to pattern match the standard name that is printed).
      If several restrictions are provided, then they are applied
      sequentially. For example:

         print_stats(.1, 'foo:')

      would first limit the printing to first 10% of list, and then
      only print functions that were part of filename ".*foo:".  In
      contrast, the command:

         print_stats('foo:', .1)

      would limit the list to all functions having file names
      ".*foo:", and then proceed to only print the first 10% of them.

   print_callers(*restrictions)

      This method for the "Stats" class prints a list of all functions
      that called each function in the profiled database.  The
      ordering is identical to that provided by "print_stats()", and
      the definition of the restricting argument is also identical.
      Each caller is reported on its own line.  The format differs
      slightly depending on the profiler that produced the stats:

      * With "profile", a number is shown in parentheses after each
        caller to show how many times this specific call was made.
        For convenience, a second non-parenthesized number repeats the
        cumulative time spent in the function at the right.

      * With "cProfile", each caller is preceded by three numbers: the
        number of times this specific call was made, and the total and
        cumulative times spent in the current function while it was
        invoked by this specific caller.

   print_callees(*restrictions)

      This method for the "Stats" class prints a list of all function
      that were called by the indicated function.  Aside from this
      reversal of direction of calls (re: called vs was called by),
      the arguments and ordering are identical to the
      "print_callers()" method.

   get_stats_profile()

      This method returns an instance of StatsProfile, which contains
      a mapping of function names to instances of FunctionProfile.
      Each FunctionProfile instance holds information related to the
      function's profile such as how long the function took to run,
      how many times it was called, etc...

      Added in version 3.9: Added the following dataclasses:
      StatsProfile, FunctionProfile. Added the following function:
      get_stats_profile.


What Is Deterministic Profiling?
================================

*Deterministic profiling* is meant to reflect the fact that all
*function call*, *function return*, and *exception* events are
monitored, and precise timings are made for the intervals between
these events (during which time the user's code is executing).  In
contrast, *statistical profiling* (which is not done by this module)
randomly samples the effective instruction pointer, and deduces where
time is being spent.  The latter technique traditionally involves less
overhead (as the code does not need to be instrumented), but provides
only relative indications of where time is being spent.

在 Python 中，由于在执行过程中总有一个活动的解释器，因此执行确定性评测
不需要插入指令的代码。Python 自动为每个事件提供一个 *钩子* （可选回调
）。此外，Python 的解释特性往往会给执行增加太多开销，以至于在典型的应
用程序中，确定性分析往往只会增加很小的处理开销。结果是，确定性分析并没
有那么代价高昂，但是它提供了有关 Python 程序执行的大量运行时统计信息。

调用计数统计信息可用于识别代码中的错误（意外计数），并识别可能的内联扩
展点（高频调用）。内部时间统计可用于识别应仔细优化的 "热循环" 。累积时
间统计可用于识别算法选择上的高级别错误。请注意，该分析器中对累积时间的
异常处理，允许直接比较算法的递归实现与迭代实现的统计信息。


局限性
======

一个限制是关于时间信息的准确性。确定性性能分析存在一个涉及精度的基本问
题。最明显的限制是，底层的 “时钟” 周期大约为 0.001 秒（通常）。因此，
没有什么测量会比底层时钟更精确。如果进行了足够的测量，那么 “误差” 将趋
于平均。不幸的是，消除第一个误差会引入第二个误差来源。

第二个问题是，从调度事件到分析器调用获取时间函数实际 *获取* 时钟状态，
这需要 "一段时间" 。类似地，从获取时钟值（然后保存）开始，直到再次执行
用户代码为止，退出分析器事件句柄时也存在一定的延迟。因此，多次调用单个
函数或调用多个函数通常会累积此错误。尽管这种方式的误差通常小于时钟的精
度（小于一个时钟周期），但它 *可以* 累积并变得非常可观。

The problem is more important with "profile" than with the lower-
overhead "cProfile".  For this reason, "profile" provides a means of
calibrating itself for a given platform so that this error can be
probabilistically (on the average) removed. After the profiler is
calibrated, it will be more accurate (in a least square sense), but it
will sometimes produce negative numbers (when call counts are
exceptionally low, and the gods of probability work against you :-). )
Do *not* be alarmed by negative numbers in the profile.  They should
*only* appear if you have calibrated your profiler, and the results
are actually better than without calibration.


校准
====

The profiler of the "profile" module subtracts a constant from each
event handling time to compensate for the overhead of calling the time
function, and socking away the results.  By default, the constant is
0. The following procedure can be used to obtain a better constant for
a given platform (see 局限性).

   import profile
   pr = profile.Profile()
   for i in range(5):
       print(pr.calibrate(10000))

此方法将执行由参数所给定次数的 Python 调用，在性能分析器之下直接和再次
地执行，并对两次执行计时。 它将随后计算每个性能分析器事件的隐藏开销，
并将其以浮点数的形式返回。例如，在一台运行 macOS 的 1.8Ghz Intel Core
i5 上，使用 Python 的 time.process_time() 作为计时器，魔数大约为
4.04e-6。

此操作的目标是获得一个相当稳定的结果。如果你的计算机 *非常* 快速，或者
你的计时器函数的分辨率很差，你可能必须传入 100000，甚至 1000000，才能
得到稳定的结果。

当你有一个一致的答案时，有三种方法可以使用:

   import profile

   # 1. 将计算出的偏差应用于此后创建的所有 Profile 实例。
   profile.Profile.bias = your_computed_bias

   # 2. 将计算出的偏差应用于特定的 Profile 实例。
   pr = profile.Profile()
   pr.bias = your_computed_bias

   # 3. 在实例构造函数中指定计算出的偏差。
   pr = profile.Profile(bias=your_computed_bias)

如果你可以选择，那么选择更小的常量会更好，这样你的结果将“更不容易”在性
能分析统计中显示负值。


使用自定义计时器
================

如果你想要改变当前时间的确定方式（例如，强制使用时钟时间或进程持续时间
），请向 "Profile" 类构造器传入你想要的计时函数:

   pr = profile.Profile(your_time_func)

The resulting profiler will then call "your_time_func". Depending on
whether you are using "profile.Profile" or "cProfile.Profile",
"your_time_func"'s return value will be interpreted differently:

"profile.Profile"
   "your_time_func" 应当返回一个数字，或一个总和为当前时间的数字列表（
   如同 "os.times()" 所返回的内容）。 如果该函数返回一个数字，或所返回
   的数字列表长度为 2，则你将得到一个特别快速的调度例程版本。

   请注意你应当为你选择的计时器函数校准性能分析器类 (参见 校准)。 对于
   大多数机器来说，一个返回长整数值的计时器在性能分析期间将提供在低开
   销方面的最佳结果。 ("os.times()" 是 *相当* 糟糕的，因为它返回一个浮
   点数值的元组)。 如果你想以最干净的方式替换一个更好的计时器，请派生
   一个类并硬连线一个能最佳地处理计时器调用的替换调度方法，并使用适当
   的校准常量。

"cProfile.Profile"
   "your_time_func" 应当返回一个数字。如果它返回整数，你还可以通过第二
   个参数指定一个单位时间的实际持续长度来唤起类构造器。 举例来说，如果
   "your_integer_time_func" 返回以千秒为单位的时间，则你应当以如下方式
   构造 "Profile" 实例:

      pr = cProfile.Profile(your_integer_time_func, 0.001)

   As the "cProfile.Profile" class cannot be calibrated, custom timer
   functions should be used with care and should be as fast as
   possible.  For the best results with a custom timer, it might be
   necessary to hard-code it in the C source of the internal "_lsprof"
   module.

Python 3.3 在 "time" 中添加了几个可被用来精确测量进程或时钟时间的新函
数。例如，参见 "time.perf_counter()".
