1 <chapter id="profiling">
2 <title>Profiling</Title>
3 <indexterm><primary>profiling</primary>
5 <indexterm><primary>cost-centre profiling</primary></indexterm>
7 <para> Glasgow Haskell comes with a time and space profiling
8 system. Its purpose is to help you improve your understanding of
9 your program's execution behaviour, so you can improve it.</para>
11 <para> Any comments, suggestions and/or improvements you have are
12 welcome. Recommended “profiling tricks” would be
13 especially cool! </para>
15 <para>Profiling a program is a three-step process:</para>
19 <para> Re-compile your program for profiling with the
20 <literal>-prof</literal> option, and probably one of the
21 <literal>-auto</literal> or <literal>-auto-all</literal>
22 options. These options are described in more detail in <xref
23 linkend="prof-compiler-options"> </para>
24 <indexterm><primary><literal>-prof</literal></primary>
26 <indexterm><primary><literal>-auto</literal></primary>
28 <indexterm><primary><literal>-auto-all</literal></primary>
33 <para> Run your program with one of the profiling options, eg.
34 <literal>+RTS -p -RTS</literal>. This generates a file of
35 profiling information.</para>
36 <indexterm><primary><option>-p</option></primary><secondary>RTS
37 option</secondary></indexterm>
41 <para> Examine the generated profiling information, using one of
42 GHC's profiling tools. The tool to use will depend on the kind
43 of profiling information generated.</para>
48 <sect1 id="cost-centres">
49 <title>Cost centres and cost-centre stacks</title>
51 <para>GHC's profiling system assigns <firstterm>costs</firstterm>
52 to <firstterm>cost centres</firstterm>. A cost is simply the time
53 or space required to evaluate an expression. Cost centres are
54 program annotations around expressions; all costs incurred by the
55 annotated expression are assigned to the enclosing cost centre.
56 Furthermore, GHC will remember the stack of enclosing cost centres
57 for any given expression at run-time and generate a call-graph of
58 cost attributions.</para>
60 <para>Let's take a look at an example:</para>
63 main = print (nfib 25)
64 nfib n = if n < 2 then 1 else nfib (n-1) + nfib (n-2)
67 <para>Compile and run this program as follows:</para>
70 $ ghc -prof -auto-all -o Main Main.hs
76 <para>When a GHC-compiled program is run with the
77 <option>-p</option> RTS option, it generates a file called
78 <filename><prog>.prof</filename>. In this case, the file
79 will contain something like this:</para>
82 Fri May 12 14:06 2000 Time and Allocation Profiling Report (Final)
86 total time = 0.14 secs (7 ticks @ 20 ms)
87 total alloc = 8,741,204 bytes (excludes profiling overheads)
89 COST CENTRE MODULE %time %alloc
95 COST CENTRE MODULE entries %time %alloc %time %alloc
97 MAIN MAIN 0 0.0 0.0 100.0 100.0
98 main Main 0 0.0 0.0 0.0 0.0
99 CAF PrelHandle 3 0.0 0.0 0.0 0.0
100 CAF PrelAddr 1 0.0 0.0 0.0 0.0
101 CAF Main 6 0.0 0.0 100.0 100.0
102 main Main 1 0.0 0.0 100.0 100.0
103 nfib Main 242785 100.0 100.0 100.0 100.0
107 <para>The first part of the file gives the program name and
108 options, and the total time and total memory allocation measured
109 during the run of the program (note that the total memory
110 allocation figure isn't the same as the amount of
111 <emphasis>live</emphasis> memory needed by the program at any one
112 time; the latter can be determined using heap profiling, which we
113 will describe shortly).</para>
115 <para>The second part of the file is a break-down by cost centre
116 of the most costly functions in the program. In this case, there
117 was only one significant function in the program, namely
118 <function>nfib</function>, and it was responsible for 100%
119 of both the time and allocation costs of the program.</para>
121 <para>The third and final section of the file gives a profile
122 break-down by cost-centre stack. This is roughly a call-graph
123 profile of the program. In the example above, it is clear that
124 the costly call to <function>nfib</function> came from
125 <function>main</function>.</para>
127 <para>The time and allocation incurred by a given part of the
128 program is displayed in two ways: “individual”, which
129 are the costs incurred by the code covered by this cost centre
130 stack alone, and “inherited”, which includes the costs
131 incurred by all the children of this node.</para>
133 <para>The usefulness of cost-centre stacks is better demonstrated
134 by modifying the example slightly:</para>
137 main = print (f 25 + g 25)
139 g n = nfib (n `div` 2)
140 nfib n = if n < 2 then 1 else nfib (n-1) + nfib (n-2)
143 <para>Compile and run this program as before, and take a look at
144 the new profiling results:</para>
147 COST CENTRE MODULE scc %time %alloc %time %alloc
149 MAIN MAIN 0 0.0 0.0 100.0 100.0
150 main Main 0 0.0 0.0 0.0 0.0
151 CAF PrelHandle 3 0.0 0.0 0.0 0.0
152 CAF PrelAddr 1 0.0 0.0 0.0 0.0
153 CAF Main 9 0.0 0.0 100.0 100.0
154 main Main 1 0.0 0.0 100.0 100.0
155 g Main 1 0.0 0.0 0.0 0.2
156 nfib Main 465 0.0 0.2 0.0 0.2
157 f Main 1 0.0 0.0 100.0 99.8
158 nfib Main 242785 100.0 99.8 100.0 99.8
161 <para>Now although we had two calls to <function>nfib</function>
162 in the program, it is immediately clear that it was the call from
163 <function>f</function> which took all the time.</para>
165 <para>The actual meaning of the various columns in the output is:</para>
171 <para>The number of times this particular point in the call
172 graph was entered.</para>
177 <term>individual %time</term>
179 <para>The percentage of the total run time of the program
180 spent at this point in the call graph.</para>
185 <term>individual %alloc</term>
187 <para>The percentage of the total memory allocations
188 (excluding profiling overheads) of the program made by this
194 <term>inherited %time</term>
196 <para>The percentage of the total run time of the program
197 spent below this point in the call graph.</para>
202 <term>inherited %alloc</term>
204 <para>The percentage of the total memory allocations
205 (excluding profiling overheads) of the program made by this
206 call and all of its sub-calls.</para>
211 <para>In addition you can use the <Option>-P</Option> RTS option
212 <indexterm><primary><option>-P</option></primary></indexterm> to
213 get the following additional information:</para>
217 <term><literal>ticks</literal></term>
219 <para>The raw number of time “ticks” which were
220 attributed to this cost-centre; from this, we get the
221 <literal>%time</literal> figure mentioned
227 <term><literal>bytes</literal></term>
229 <para>Number of bytes allocated in the heap while in this
230 cost-centre; again, this is the raw number from which we get
231 the <literal>%alloc</literal> figure mentioned
237 <para>What about recursive functions, and mutually recursive
238 groups of functions? Where are the costs attributed? Well,
239 although GHC does keep information about which groups of functions
240 called each other recursively, this information isn't displayed in
241 the basic time and allocation profile, instead the call-graph is
242 flattened into a tree. The XML profiling tool (described in <xref
243 linkend="prof-xml-tool">) will be able to display real loops in
244 the call-graph.</para>
246 <sect2><title>Inserting cost centres by hand</title>
248 <para>Cost centres are just program annotations. When you say
249 <option>-auto-all</option> to the compiler, it automatically
250 inserts a cost centre annotation around every top-level function
251 in your program, but you are entirely free to add the cost
252 centre annotations yourself.</para>
254 <para>The syntax of a cost centre annotation is</para>
257 {-# SCC "name" #-} <expression>
260 <para>where <literal>"name"</literal> is an aribrary string,
261 that will become the name of your cost centre as it appears
262 in the profiling output, and
263 <literal><expression></literal> is any Haskell
264 expression. An <literal>SCC</literal> annotation extends as
265 far to the right as possible when parsing.</para>
269 <sect2 id="prof-rules">
270 <title>Rules for attributing costs</title>
272 <para>The cost of evaluating any expression in your program is
273 attributed to a cost-centre stack using the following rules:</para>
277 <para>If the expression is part of the
278 <firstterm>one-off</firstterm> costs of evaluating the
279 enclosing top-level definition, then costs are attributed to
280 the stack of lexically enclosing <literal>SCC</literal>
281 annotations on top of the special <literal>CAF</literal>
286 <para>Otherwise, costs are attributed to the stack of
287 lexically-enclosing <literal>SCC</literal> annotations,
288 appended to the cost-centre stack in effect at the
289 <firstterm>call site</firstterm> of the current top-level
290 definition<footnote> <para>The call-site is just the place
291 in the source code which mentions the particular function or
292 variable.</para></footnote>. Notice that this is a recursive
297 <para>Time spent in foreign code (see <xref linkend="ffi">)
298 is always attributed to the cost centre in force at the
299 Haskell call-site of the foreign function.</para>
303 <para>What do we mean by one-off costs? Well, Haskell is a lazy
304 language, and certain expressions are only ever evaluated once.
305 For example, if we write:</para>
311 <para>then <varname>x</varname> will only be evaluated once (if
312 at all), and subsequent demands for <varname>x</varname> will
313 immediately get to see the cached result. The definition
314 <varname>x</varname> is called a CAF (Constant Applicative
315 Form), because it has no arguments.</para>
317 <para>For the purposes of profiling, we say that the expression
318 <literal>nfib 25</literal> belongs to the one-off costs of
319 evaluating <varname>x</varname>.</para>
321 <para>Since one-off costs aren't strictly speaking part of the
322 call-graph of the program, they are attributed to a special
323 top-level cost centre, <literal>CAF</literal>. There may be one
324 <literal>CAF</literal> cost centre for each module (the
325 default), or one for each top-level definition with any one-off
326 costs (this behaviour can be selected by giving GHC the
327 <option>-caf-all</option> flag).</para>
329 <indexterm><primary><literal>-caf-all</literal></primary>
332 <para>If you think you have a weird profile, or the call-graph
333 doesn't look like you expect it to, feel free to send it (and
334 your program) to us at
335 <email>glasgow-haskell-bugs@haskell.org</email>.</para>
339 <sect1 id="prof-compiler-options">
340 <title>Compiler options for profiling</title>
342 <indexterm><primary>profiling</primary><secondary>options</secondary></indexterm>
343 <indexterm><primary>options</primary><secondary>for profiling</secondary></indexterm>
347 <term><Option>-prof</Option>:</Term>
348 <indexterm><primary><option>-prof</option></primary></indexterm>
350 <para> To make use of the profiling system
351 <emphasis>all</emphasis> modules must be compiled and linked
352 with the <option>-prof</option> option. Any
353 <literal>SCC</literal> annotations you've put in your source
354 will spring to life.</para>
356 <para> Without a <option>-prof</option> option, your
357 <literal>SCC</literal>s are ignored; so you can compile
358 <literal>SCC</literal>-laden code without changing
364 <para>There are a few other profiling-related compilation options.
365 Use them <emphasis>in addition to</emphasis>
366 <option>-prof</option>. These do not have to be used consistently
367 for all modules in a program.</para>
371 <term><option>-auto</option>:</Term>
372 <indexterm><primary><option>-auto</option></primary></indexterm>
373 <indexterm><primary>cost centres</primary><secondary>automatically inserting</secondary></indexterm>
375 <para> GHC will automatically add
376 <Function>_scc_</Function> constructs for all
377 top-level, exported functions.</para>
382 <term><option>-auto-all</option>:</Term>
383 <indexterm><primary><option>-auto-all</option></primary></indexterm>
385 <para> <Emphasis>All</Emphasis> top-level functions,
386 exported or not, will be automatically
387 <Function>_scc_</Function>'d.</para>
392 <term><option>-caf-all</option>:</Term>
393 <indexterm><primary><option>-caf-all</option></primary></indexterm>
395 <para> The costs of all CAFs in a module are usually
396 attributed to one “big” CAF cost-centre. With
397 this option, all CAFs get their own cost-centre. An
398 “if all else fails” option…</para>
403 <term><option>-ignore-scc</option>:</Term>
404 <indexterm><primary><option>-ignore-scc</option></primary></indexterm>
406 <para>Ignore any <Function>_scc_</Function>
407 constructs, so a module which already has
408 <Function>_scc_</Function>s can be compiled
409 for profiling with the annotations ignored.</para>
417 <sect1 id="prof-time-options">
418 <title>Time and allocation profiling</Title>
420 <para>To generate a time and allocation profile, give one of the
421 following RTS options to the compiled program when you run it (RTS
422 options should be enclosed between <literal>+RTS...-RTS</literal>
427 <term><Option>-p</Option> or <Option>-P</Option>:</Term>
428 <indexterm><primary><option>-p</option></primary></indexterm>
429 <indexterm><primary><option>-P</option></primary></indexterm>
430 <indexterm><primary>time profile</primary></indexterm>
432 <para>The <Option>-p</Option> option produces a standard
433 <Emphasis>time profile</Emphasis> report. It is written
435 <Filename><replaceable>program</replaceable>.prof</Filename>.</para>
437 <para>The <Option>-P</Option> option produces a more
438 detailed report containing the actual time and allocation
439 data as well. (Not used much.)</para>
444 <term><option>-px</option>:</term>
445 <indexterm><primary><option>-px</option></primary></indexterm>
447 <para>The <option>-px</option> option generates profiling
448 information in the XML format understood by our new
449 profiling tool, see <xref linkend="prof-xml-tool">.</para>
454 <term><option>-xc</option></term>
455 <indexterm><primary><option>-xc</option></primary><secondary>RTS
456 option</secondary></indexterm>
458 <para>This option makes use of the extra information
459 maintained by the cost-centre-stack profiler to provide
460 useful information about the location of runtime errors.
461 See <xref linkend="rts-options-debugging">.</para>
469 <sect1 id="prof-heap">
470 <title>Profiling memory usage</title>
472 <para>In addition to profiling the time and allocation behaviour
473 of your program, you can also generate a graph of its memory usage
474 over time. This is useful for detecting the causes of
475 <firstterm>space leaks</firstterm>, when your program holds on to
476 more memory at run-time that it needs to. Space leaks lead to
477 longer run-times due to heavy garbage collector ativity, and may
478 even cause the program to run out of memory altogether.</para>
480 <para>To generate a heap profile from your program:</para>
484 <para>Compile the program for profiling (<xref
485 linkend="prof-compiler-options">).</para>
488 <para>Run it with one of the heap profiling options described
489 below (eg. <option>-hc</option> for a basic producer profile).
490 This generates the file
491 <filename><replaceable>prog</replaceable>.hp</filename>.</para>
494 <para>Run <command>hp2ps</command> to produce a Postscript
496 <filename><replaceable>prog</replaceable>.ps</filename>. The
497 <command>hp2ps</command> utility is described in detail in
498 <xref linkend="hp2ps">.</para>
501 <para>Display the heap profile using a postscript viewer such
502 as <application>Ghostview</application>, or print it out on a
503 Postscript-capable printer.</para>
507 <sect2 id="rts-options-heap-prof">
508 <title>RTS options for heap profiling</title>
510 <para>There are several different kinds of heap profile that can
511 be generated. All the different profile types yield a graph of
512 live heap against time, but they differ in how the live heap is
513 broken down into bands. The following RTS options select which
514 break-down to use:</para>
518 <term><option>-hc</option></term>
519 <indexterm><primary><option>-hc</option></primary><secondary>RTS
520 option</secondary></indexterm>
522 <para>Breaks down the graph by the cost-centre stack which
523 produced the data.</para>
528 <term><option>-hm</option></term>
529 <indexterm><primary><option>-hm</option></primary><secondary>RTS
530 option</secondary></indexterm>
532 <para>Break down the live heap by the module containing
533 the code which produced the data.</para>
538 <term><option>-hd</option></term>
539 <indexterm><primary><option>-hd</option></primary><secondary>RTS
540 option</secondary></indexterm>
542 <para>Breaks down the graph by <firstterm>closure
543 description</firstterm>. For actual data, the description
544 is just the constructor name, for other closures it is a
545 compiler-generated string identifying the closure.</para>
550 <term><option>-hy</option></term>
551 <indexterm><primary><option>-hy</option></primary><secondary>RTS
552 option</secondary></indexterm>
554 <para>Breaks down the graph by
555 <firstterm>type</firstterm>. For closures which have
556 function type or unknown/polymorphic type, the string will
557 represent an approximation to the actual type.</para>
562 <term><option>-hr</option></term>
563 <indexterm><primary><option>-hr</option></primary><secondary>RTS
564 option</secondary></indexterm>
566 <para>Break down the graph by <firstterm>retainer
567 set</firstterm>. Retainer profiling is described in more
568 detail below (<xref linkend="retainer-prof">).</para>
573 <term><option>-hb</option></term>
574 <indexterm><primary><option>-hb</option></primary><secondary>RTS
575 option</secondary></indexterm>
577 <para>Break down the graph by
578 <firstterm>biography</firstterm>. Biographical profiling
579 is described in more detail below (<xref
580 linkend="biography-prof">).</para>
585 <para>In addition, the profile can be restricted to heap data
586 which satisfies certain criteria - for example, you might want
587 to display a profile by type but only for data produced by a
588 certain module, or a profile by retainer for a certain type of
589 data. Restrictions are specified as follows:</para>
593 <term><option>-hc</option><replaceable>name</replaceable>,...</term>
594 <indexterm><primary><option>-hc</option></primary><secondary>RTS
595 option</secondary></indexterm>
597 <para>Restrict the profile to closures produced by
598 cost-centre stacks with one of the specified cost centres
604 <term><option>-hC</option><replaceable>name</replaceable>,...</term>
605 <indexterm><primary><option>-hC</option></primary><secondary>RTS
606 option</secondary></indexterm>
608 <para>Restrict the profile to closures produced by
609 cost-centre stacks with one of the specified cost centres
610 anywhere in the stack.</para>
615 <term><option>-hm</option><replaceable>module</replaceable>,...</term>
616 <indexterm><primary><option>-hm</option></primary><secondary>RTS
617 option</secondary></indexterm>
619 <para>Restrict the profile to closures produced by the
620 specified modules.</para>
625 <term><option>-hd</option><replaceable>desc</replaceable>,...</term>
626 <indexterm><primary><option>-hd</option></primary><secondary>RTS
627 option</secondary></indexterm>
629 <para>Restrict the profile to closures with the specified
630 description strings.</para>
635 <term><option>-hy</option><replaceable>type</replaceable>,...</term>
636 <indexterm><primary><option>-hy</option></primary><secondary>RTS
637 option</secondary></indexterm>
639 <para>Restrict the profile to closures with the specified
645 <term><option>-hr</option><replaceable>cc</replaceable>,...</term>
646 <indexterm><primary><option>-hr</option></primary><secondary>RTS
647 option</secondary></indexterm>
649 <para>Restrict the profile to closures with retainer sets
650 containing cost-centre stacks with one of the specified
651 cost centres at the top.</para>
656 <term><option>-hb</option><replaceable>bio</replaceable>,...</term>
657 <indexterm><primary><option>-hb</option></primary><secondary>RTS
658 option</secondary></indexterm>
660 <para>Restrict the profile to closures with one of the
661 specified biographies, where
662 <replaceable>bio</replaceable> is one of
663 <literal>lag</literal>, <literal>drag</literal>,
664 <literal>void</literal>, or <literal>use</literal>.</para>
669 <para>For example, the following options will generate a
670 retainer profile restricted to <literal>Branch</literal> and
671 <literal>Leaf</literal> constructors:</para>
674 <replaceable>prog</replaceable> +RTS -hr -hdBranch,Leaf
677 <para>There can only be one "break-down" option
678 (eg. <option>-hr</option> in the example above), but there is no
679 limit on the number of further restrictions that may be applied.
680 All the options may be combined, with one exception: GHC doesn't
681 currently support mixing the <option>-hr</option> and
682 <option>-hb</option> options.</para>
684 <para>There are two more options which relate to heap
689 <term><Option>-i<replaceable>secs</replaceable></Option>:</Term>
690 <indexterm><primary><option>-i</option></primary></indexterm>
692 <para>Set the profiling (sampling) interval to
693 <replaceable>secs</replaceable> seconds (the default is
694 0.1 second). Fractions are allowed: for example
695 <Option>-i0.2</Option> will get 5 samples per second.
696 This only affects heap profiling; time profiles are always
697 sampled on a 1/50 second frequency.</para>
702 <term><option>-xt</option></term>
703 <indexterm><primary><option>-xt</option></primary><secondary>RTS option</secondary>
706 <para>Include the memory occupied by threads in a heap
707 profile. Each thread takes up a small area for its thread
708 state in addition to the space allocated for its stack
709 (stacks normally start small and then grow as
712 <para>This includes the main thread, so using
713 <option>-xt</option> is a good way to see how much stack
714 space the program is using.</para>
716 <para>Memory occupied by threads and their stacks is
717 labelled as “TSO” when displaying the profile
718 by closure description or type description.</para>
725 <sect2 id="retainer-prof">
726 <title>Retainer Profiling</title>
728 <para>Retainer profiling is designed to help answer questions
729 like <quote>why is this data being retained?</quote>. We start
730 by defining what we mean by a retainer:</para>
733 <para>A retainer is either the system stack, or an unevaluated
734 closure (thunk).</para>
737 <para>In particular, constructors are <emphasis>not</emphasis>
740 <para>An object A is retained by an object B if object A can be
741 reached by recursively following pointers starting from object
742 B but not meeting any other retainers on the way. Each object
743 has one or more retainers, collectively called its
744 <firstterm>retainer set</firstterm>.</para>
746 <para>When retainer profiling is requested by giving the program
747 the <option>-hr</option> option, a graph is generated which is
748 broken down by retainer set. A retainer set is displayed as a
749 set of cost-centre stacks; because this is usually too large to
750 fit on the profile graph, each retainer set is numbered and
751 shown abbreviated on the graph along with its number, and the
752 full list of retainer sets is dumped into the file
753 <filename><replaceable>prog</replaceable>.prof</filename>.</para>
755 <para>Retainer profiling requires multiple passes over the live
756 heap in order to discover the full retainer set for each
757 object, which can be quite slow. So we set a limit on the
758 maximum size of a retainer set, where all retainer sets larger
759 than the maximum retainer set size are replaced by the special
760 set <literal>MANY</literal>. The maximum set size defaults to 8
761 and can be altered with the <option>-R</option> RTS
766 <term><option>-R</option><replaceable>size</replaceable></term>
768 <para>Restrict the number of elements in a retainer set to
769 <replaceable>size</replaceable> (default 8).</para>
775 <title>Hints for using retainer profiling</title>
777 <para>The definition of retainers is designed to reflect a
778 common cause of space leaks: a large structure is retained by
779 an unevaluated computation, and will be released once the
780 compuation is forced. A good example is looking up a value in
781 a finite map, where unless the lookup is forced in a timely
782 manner the unevaluated lookup will cause the whole mapping to
783 be retained. These kind of space leaks can often be
784 eliminated by forcing the relevant computations to be
785 performed eagerly, using <literal>seq</literal> or strictness
786 annotations on data constructor fields.</para>
788 <para>Often a particular data structure is being retained by a
789 chain of unevaluated closures, only the nearest of which will
790 be reported by retainer profiling - for example A retains B, B
791 retains C, and C retains a large structure. There might be a
792 large number of Bs but only a single A, so A is really the one
793 we're interested in eliminating. However, retainer profiling
794 will in this case report B as the retainer of the large
795 structure. To move further up the chain of retainers, we can
796 ask for another retainer profile but this time restrict the
797 profile to B objects, so we get a profile of the retainers of
801 <replaceable>prog</replaceable> +RTS -hr -hcB
804 <para>This trick isn't foolproof, because there might be other
805 B closures in the heap which aren't the retainers we are
806 interested in, but we've found this to be a useful technique
807 in most cases.</para>
811 <sect2 id="biography-prof">
812 <title>Biographical Profiling</title>
814 <para>A typical heap object may be in one of the following four
815 states at each point in its lifetime:</para>
819 <para>The <firstterm>lag</firstterm> stage, which is the
820 time between creation and the first use of the
824 <para>the <firstterm>use</firstterm> stage, which lasts from
825 the first use until the last use of the object, and</para>
828 <para>The <firstterm>drag</firstterm> stage, which lasts
829 from the final use until the last reference to the object
833 <para>An object which is never used is said to be in the
834 <firstterm>void</firstterm> state for its whole
839 <para>A biographical heap profile displays the portion of the
840 live heap in each of the four states listed above. Usually the
841 most interesting states are the void and drag states: live heap
842 in these states is more likely to be wasted space than heap in
843 the lag or use states.</para>
845 <para>It is also possible to break down the heap in one or more
846 of these states by a different criteria, by restricting a
847 profile by biography. For example, to show the portion of the
848 heap in the drag or void state by producer: </para>
851 <replaceable>prog</replaceable> +RTS -hc -hbdrag,void
854 <para>Once you know the producer or the type of the heap in the
855 drag or void states, the next step is usually to find the
859 <replaceable>prog</replaceable> +RTS -hr -hc<replaceable>cc</replaceable>...
862 <para>NOTE: this two stage process is required because GHC
863 cannot currently profile using both biographical and retainer
864 information simultaneously.</para>
873 <sect1 id="prof-xml-tool">
874 <title>Graphical time/allocation profile</title>
876 <para>You can view the time and allocation profiling graph of your
877 program graphically, using <command>ghcprof</command>. This is a
878 new tool with GHC 4.08, and will eventually be the de-facto
879 standard way of viewing GHC profiles<footnote><para>Actually this
880 isn't true any more, we are working on a new tool for
881 displaying heap profiles using Gtk+HS, so
882 <command>ghcprof</command> may go away at some point in the future.</para>
885 <para>To run <command>ghcprof</command>, you need
886 <productname>daVinci</productname> installed, which can be
888 url="http://www.informatik.uni-bremen.de/daVinci/"><citetitle>The Graph
889 Visualisation Tool daVinci</citetitle></ulink>. Install one of
891 distributions<footnote><para><productname>daVinci</productname> is
892 sadly not open-source :-(.</para></footnote>, and set your
893 <envar>DAVINCIHOME</envar> environment variable to point to the
894 installation directory.</para>
896 <para><command>ghcprof</command> uses an XML-based profiling log
897 format, and you therefore need to run your program with a
898 different option: <option>-px</option>. The file generated is
899 still called <filename><prog>.prof</filename>. To see the
900 profile, run <command>ghcprof</command> like this:</para>
902 <indexterm><primary><option>-px</option></primary></indexterm>
905 $ ghcprof <prog>.prof
908 <para>which should pop up a window showing the call-graph of your
909 program in glorious detail. More information on using
910 <command>ghcprof</command> can be found at <ulink
911 url="http://www.dcs.warwick.ac.uk/people/academic/Stephen.Jarvis/profiler/index.html"><citetitle>The
912 Cost-Centre Stack Profiling Tool for
913 GHC</citetitle></ulink>.</para>
918 <title><command>hp2ps</command>––heap profile to PostScript</title>
920 <indexterm><primary><command>hp2ps</command></primary></indexterm>
921 <indexterm><primary>heap profiles</primary></indexterm>
922 <indexterm><primary>postscript, from heap profiles</primary></indexterm>
923 <indexterm><primary><option>-h<break-down></option></primary></indexterm>
928 hp2ps [flags] [<file>[.hp]]
932 <command>hp2ps</command><indexterm><primary>hp2ps
933 program</primary></indexterm> converts a heap profile as produced
934 by the <Option>-h<break-down></Option> runtime option into a
935 PostScript graph of the heap profile. By convention, the file to
936 be processed by <command>hp2ps</command> has a
937 <filename>.hp</filename> extension. The PostScript output is
938 written to <filename><file>@.ps</filename>. If
939 <filename><file></filename> is omitted entirely, then the
940 program behaves as a filter.</para>
942 <para><command>hp2ps</command> is distributed in
943 <filename>ghc/utils/hp2ps</filename> in a GHC source
944 distribution. It was originally developed by Dave Wakeling as part
945 of the HBC/LML heap profiler.</para>
947 <para>The flags are:</para>
952 <term><Option>-d</Option></Term>
954 <para>In order to make graphs more readable,
955 <command>hp2ps</command> sorts the shaded bands for each
956 identifier. The default sort ordering is for the bands with
957 the largest area to be stacked on top of the smaller ones.
958 The <Option>-d</Option> option causes rougher bands (those
959 representing series of values with the largest standard
960 deviations) to be stacked on top of smoother ones.</para>
965 <term><Option>-b</Option></Term>
967 <para>Normally, <command>hp2ps</command> puts the title of
968 the graph in a small box at the top of the page. However, if
969 the JOB string is too long to fit in a small box (more than
970 35 characters), then <command>hp2ps</command> will choose to
971 use a big box instead. The <Option>-b</Option> option
972 forces <command>hp2ps</command> to use a big box.</para>
977 <term><Option>-e<float>[in|mm|pt]</Option></Term>
979 <para>Generate encapsulated PostScript suitable for
980 inclusion in LaTeX documents. Usually, the PostScript graph
981 is drawn in landscape mode in an area 9 inches wide by 6
982 inches high, and <command>hp2ps</command> arranges for this
983 area to be approximately centred on a sheet of a4 paper.
984 This format is convenient of studying the graph in detail,
985 but it is unsuitable for inclusion in LaTeX documents. The
986 <Option>-e</Option> option causes the graph to be drawn in
987 portrait mode, with float specifying the width in inches,
988 millimetres or points (the default). The resulting
989 PostScript file conforms to the Encapsulated PostScript
990 (EPS) convention, and it can be included in a LaTeX document
991 using Rokicki's dvi-to-PostScript converter
992 <command>dvips</command>.</para>
997 <term><Option>-g</Option></Term>
999 <para>Create output suitable for the <command>gs</command>
1000 PostScript previewer (or similar). In this case the graph is
1001 printed in portrait mode without scaling. The output is
1002 unsuitable for a laser printer.</para>
1007 <term><Option>-l</Option></Term>
1009 <para>Normally a profile is limited to 20 bands with
1010 additional identifiers being grouped into an
1011 <literal>OTHER</literal> band. The <Option>-l</Option> flag
1012 removes this 20 band and limit, producing as many bands as
1013 necessary. No key is produced as it won't fit!. It is useful
1014 for creation time profiles with many bands.</para>
1019 <term><Option>-m<int></Option></Term>
1021 <para>Normally a profile is limited to 20 bands with
1022 additional identifiers being grouped into an
1023 <literal>OTHER</literal> band. The <Option>-m</Option> flag
1024 specifies an alternative band limit (the maximum is
1027 <para><Option>-m0</Option> requests the band limit to be
1028 removed. As many bands as necessary are produced. However no
1029 key is produced as it won't fit! It is useful for displaying
1030 creation time profiles with many bands.</para>
1035 <term><Option>-p</Option></Term>
1037 <para>Use previous parameters. By default, the PostScript
1038 graph is automatically scaled both horizontally and
1039 vertically so that it fills the page. However, when
1040 preparing a series of graphs for use in a presentation, it
1041 is often useful to draw a new graph using the same scale,
1042 shading and ordering as a previous one. The
1043 <Option>-p</Option> flag causes the graph to be drawn using
1044 the parameters determined by a previous run of
1045 <command>hp2ps</command> on <filename>file</filename>. These
1046 are extracted from <filename>file@.aux</filename>.</para>
1051 <term><Option>-s</Option></Term>
1053 <para>Use a small box for the title.</para>
1058 <term><Option>-t<float></Option></Term>
1060 <para>Normally trace elements which sum to a total of less
1061 than 1% of the profile are removed from the
1062 profile. The <option>-t</option> option allows this
1063 percentage to be modified (maximum 5%).</para>
1065 <para><Option>-t0</Option> requests no trace elements to be
1066 removed from the profile, ensuring that all the data will be
1072 <term><Option>-c</Option></Term>
1074 <para>Generate colour output.</para>
1079 <term><Option>-y</Option></Term>
1081 <para>Ignore marks.</para>
1086 <term><Option>-?</Option></Term>
1088 <para>Print out usage information.</para>
1094 <sect2 id="manipulating-hp">
1095 <title>Manipulating the hp file</title>
1097 <para>(Notes kindly offered by Jan-Willhem Maessen.)</para>
1100 The <filename>FOO.hp</filename> file produced when you ask for the
1101 heap profile of a program <filename>FOO</filename> is a text file with a particularly
1102 simple structure. Here's a representative example, with much of the
1103 actual data omitted:
1106 DATE "Thu Dec 26 18:17 2002"
1107 SAMPLE_UNIT "seconds"
1118 BEGIN_SAMPLE 11695.47
1121 The first four lines (<literal>JOB</literal>, <literal>DATE</literal>, <literal>SAMPLE_UNIT</literal>, <literal>VALUE_UNIT</literal>) form a
1122 header. Each block of lines starting with <literal>BEGIN_SAMPLE</literal> and ending
1123 with <literal>END_SAMPLE</literal> forms a single sample (you can think of this as a
1124 vertical slice of your heap profile). The hp2ps utility should accept
1125 any input with a properly-formatted header followed by a series of
1131 <title>Zooming in on regions of your profile</title>
1134 You can look at particular regions of your profile simply by loading a
1135 copy of the <filename>.hp</filename> file into a text editor and deleting the unwanted
1136 samples. The resulting <filename>.hp</filename> file can be run through <command>hp2ps</command> and viewed
1142 <title>Viewing the heap profile of a running program</title>
1145 The <filename>.hp</filename> file is generated incrementally as your
1146 program runs. In principle, running <command>hp2ps</command> on the incomplete file
1147 should produce a snapshot of your program's heap usage. However, the
1148 last sample in the file may be incomplete, causing <command>hp2ps</command> to fail. If
1149 you are using a machine with UNIX utilities installed, it's not too
1150 hard to work around this problem (though the resulting command line
1151 looks rather Byzantine):
1153 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1157 The command <command>fgrep -n END_SAMPLE FOO.hp</command> finds the
1158 end of every complete sample in <filename>FOO.hp</filename>, and labels each sample with
1159 its ending line number. We then select the line number of the last
1160 complete sample using <command>tail</command> and <command>cut</command>. This is used as a
1161 parameter to <command>head</command>; the result is as if we deleted the final
1162 incomplete sample from <filename>FOO.hp</filename>. This results in a properly-formatted
1163 .hp file which we feed directly to <command>hp2ps</command>.
1167 <title>Viewing a heap profile in real time</title>
1170 The <command>gv</command> and <command>ghostview</command> programs
1171 have a "watch file" option can be used to view an up-to-date heap
1172 profile of your program as it runs. Simply generate an incremental
1173 heap profile as described in the previous section. Run <command>gv</command> on your
1176 gv -watch -seascape FOO.ps
1178 If you forget the <literal>-watch</literal> flag you can still select
1179 "Watch file" from the "State" menu. Now each time you generate a new
1180 profile <filename>FOO.ps</filename> the view will update automatically.
1184 This can all be encapsulated in a little script:
1187 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1189 gv -watch -seascape FOO.ps &
1191 sleep 10 # We generate a new profile every 10 seconds.
1192 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1196 Occasionally <command>gv</command> will choke as it tries to read an incomplete copy of
1197 <filename>FOO.ps</filename> (because <command>hp2ps</command> is still running as an update
1198 occurs). A slightly more complicated script works around this
1199 problem, by using the fact that sending a SIGHUP to gv will cause it
1200 to re-read its input file:
1203 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1209 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1220 <sect1 id="ticky-ticky">
1221 <title>Using “ticky-ticky” profiling (for implementors)</Title>
1222 <indexterm><primary>ticky-ticky profiling</primary></indexterm>
1224 <para>(ToDo: document properly.)</para>
1226 <para>It is possible to compile Glasgow Haskell programs so that
1227 they will count lots and lots of interesting things, e.g., number
1228 of updates, number of data constructors entered, etc., etc. We
1229 call this “ticky-ticky”
1230 profiling,<indexterm><primary>ticky-ticky
1231 profiling</primary></indexterm> <indexterm><primary>profiling,
1232 ticky-ticky</primary></indexterm> because that's the sound a Sun4
1233 makes when it is running up all those counters
1234 (<Emphasis>slowly</Emphasis>).</para>
1236 <para>Ticky-ticky profiling is mainly intended for implementors;
1237 it is quite separate from the main “cost-centre”
1238 profiling system, intended for all users everywhere.</para>
1240 <para>To be able to use ticky-ticky profiling, you will need to
1241 have built appropriate libraries and things when you made the
1242 system. See “Customising what libraries to build,” in
1243 the installation guide.</para>
1245 <para>To get your compiled program to spit out the ticky-ticky
1246 numbers, use a <Option>-r</Option> RTS
1247 option<indexterm><primary>-r RTS option</primary></indexterm>.
1248 See <XRef LinkEnd="runtime-control">.</para>
1250 <para>Compiling your program with the <Option>-ticky</Option>
1251 switch yields an executable that performs these counts. Here is a
1252 sample ticky-ticky statistics file, generated by the invocation
1253 <command>foo +RTS -rfoo.ticky</command>.</para>
1256 foo +RTS -rfoo.ticky
1259 ALLOCATIONS: 3964631 (11330900 words total: 3999476 admin, 6098829 goods, 1232595 slop)
1260 total words: 2 3 4 5 6+
1261 69647 ( 1.8%) function values 50.0 50.0 0.0 0.0 0.0
1262 2382937 ( 60.1%) thunks 0.0 83.9 16.1 0.0 0.0
1263 1477218 ( 37.3%) data values 66.8 33.2 0.0 0.0 0.0
1264 0 ( 0.0%) big tuples
1265 2 ( 0.0%) black holes 0.0 100.0 0.0 0.0 0.0
1266 0 ( 0.0%) prim things
1267 34825 ( 0.9%) partial applications 0.0 0.0 0.0 100.0 0.0
1268 2 ( 0.0%) thread state objects 0.0 0.0 0.0 0.0 100.0
1270 Total storage-manager allocations: 3647137 (11882004 words)
1271 [551104 words lost to speculative heap-checks]
1275 ENTERS: 9400092 of which 2005772 (21.3%) direct to the entry code
1276 [the rest indirected via Node's info ptr]
1277 1860318 ( 19.8%) thunks
1278 3733184 ( 39.7%) data values
1279 3149544 ( 33.5%) function values
1280 [of which 1999880 (63.5%) bypassed arg-satisfaction chk]
1281 348140 ( 3.7%) partial applications
1282 308906 ( 3.3%) normal indirections
1283 0 ( 0.0%) permanent indirections
1286 2137257 ( 36.4%) from entering a new constructor
1287 [the rest from entering an existing constructor]
1288 2349219 ( 40.0%) vectored [the rest unvectored]
1290 RET_NEW: 2137257: 32.5% 46.2% 21.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1291 RET_OLD: 3733184: 2.8% 67.9% 29.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1292 RET_UNBOXED_TUP: 2: 0.0% 0.0%100.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1294 RET_VEC_RETURN : 2349219: 0.0% 0.0%100.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1296 UPDATE FRAMES: 2241725 (0 omitted from thunks)
1300 0 ( 0.0%) data values
1301 34827 ( 1.6%) partial applications
1302 [2 in place, 34825 allocated new space]
1303 2206898 ( 98.4%) updates to existing heap objects (46 by squeezing)
1304 UPD_CON_IN_NEW: 0: 0 0 0 0 0 0 0 0 0
1305 UPD_PAP_IN_NEW: 34825: 0 0 0 34825 0 0 0 0 0
1307 NEW GEN UPDATES: 2274700 ( 99.9%)
1309 OLD GEN UPDATES: 1852 ( 0.1%)
1311 Total bytes copied during GC: 190096
1313 **************************************************
1314 3647137 ALLOC_HEAP_ctr
1315 11882004 ALLOC_HEAP_tot
1320 34831 ALLOC_FUN_hst_0
1321 34816 ALLOC_FUN_hst_1
1325 2382937 ALLOC_UP_THK_ctr
1328 0 E!NT_PERM_IND_ctr requires +RTS -Z
1329 [... lots more info omitted ...]
1330 0 GC_SEL_ABANDONED_ctr
1333 0 GC_FAILED_PROMOTION_ctr
1334 47524 GC_WORDS_COPIED_ctr
1337 <para>The formatting of the information above the row of asterisks
1338 is subject to change, but hopefully provides a useful
1339 human-readable summary. Below the asterisks <Emphasis>all
1340 counters</Emphasis> maintained by the ticky-ticky system are
1341 dumped, in a format intended to be machine-readable: zero or more
1342 spaces, an integer, a space, the counter name, and a newline.</para>
1344 <para>In fact, not <Emphasis>all</Emphasis> counters are
1345 necessarily dumped; compile- or run-time flags can render certain
1346 counters invalid. In this case, either the counter will simply
1347 not appear, or it will appear with a modified counter name,
1348 possibly along with an explanation for the omission (notice
1349 <literal>ENT_PERM_IND_ctr</literal> appears
1350 with an inserted <literal>!</literal> above). Software analysing
1351 this output should always check that it has the counters it
1352 expects. Also, beware: some of the counters can have
1353 <Emphasis>large</Emphasis> values!</para>
1360 ;;; Local Variables: ***
1362 ;;; sgml-parent-document: ("users_guide.sgml" "book" "chapter") ***