[ghc-hetmet.git] / ghc / docs / users_guide / separate_compilation.sgml

  <sect1 id="separate-compilation">
    <title>Separate compilation</title>
    
    <indexterm><primary>separate compilation</primary></indexterm>
    <indexterm><primary>recompilation checker</primary></indexterm>
    <indexterm><primary>make and recompilation</primary></indexterm>
    
    <para>This section describes how GHC supports separate
    compilation.</para>

    <sect2 id="hi-files">
      <title>Interface files</title>
      
      <indexterm><primary>interface files</primary></indexterm>
      <indexterm><primary><literal>.hi</literal> files</primary></indexterm>
      
      <para>When GHC compiles a source file <filename>A.hs</filename>
      which contains a module <literal>A</literal>, say, it generates
      an object <filename>A.o</filename>, <emphasis>and</emphasis> a
      companion <emphasis>interface file</emphasis>
      <filename>A.hi</filename>.  The interface file is not intended
      for human consumption, as you'll see if you take a look at one.
      It's merely there to help the compiler compile other modules in
      the same program.</para>

      <para>NOTE: In general, the name of a file containing module
      <literal>M</literal> should be named <filename>M.hs</filename>
      or <literal>M.lhs</literal>.  The only exception to this rule is
      module <literal>Main</literal>, which can be placed in any
      file.<indexterm><primary>filenames</primary><secondary>for
      modules</secondary> </indexterm></para>
      
      <para>The interface file for <literal>A</literal> contains
      information needed by the compiler when it compiles any module
      <literal>B</literal> that imports <literal>A</literal>, whether
      directly or indirectly.  When compiling <literal>B</literal>,
      GHC will read <filename>A.hi</filename> to find the details that
      it needs to know about things defined in
      <literal>A</literal>.</para>

      <para>The interface file may contain all sorts of things that
      aren't explicitly exported from <literal>A</literal> by the
      programmer.  For example, even though a data type is exported
      abstractly, <filename>A.hi</filename> will contain the full data
      type definition.  For small function definitions,
      <filename>A.hi</filename> will contain the complete definition
      of the function.  For bigger functions,
      <filename>A.hi</filename> will contain strictness information
      about the function.  And so on.  GHC puts much more information
      into <filename>.hi</filename> files when optimisation is turned
      on with the <option>-O</option> flag (see <xref
      linkend="options-optimise">).  Without <option>-O</option> it
      puts in just the minimum; with <option>-O</option> it lobs in a
      whole pile of stuff.  <indexterm><primary>optimsation, effect on
      .hi files</primary></indexterm></para>

      <para><filename>A.hi</filename> should really be thought of as a
      compiler-readable version of <filename>A.o</filename>.  If you
      use a <filename>.hi</filename> file that wasn't generated by the
      same compilation run that generates the <filename>.o</filename>
      file the compiler may assume all sorts of incorrect things about
      <literal>A</literal>, resulting in core dumps and other
      unpleasant happenings.</para>

    </sect2>

    <sect2 id="options-finding-imports">
      <title>Finding interface files</title>

      <indexterm><primary>interface files, finding them</primary></indexterm>
      <indexterm><primary>finding interface files</primary></indexterm>

      <para>In your program, you import a module
      <literal>Foo</literal> by saying <literal>import Foo</literal>.
      GHC goes looking for an interface file,
      <filename>Foo.hi</filename>.  It has a builtin list of
      directories (notably including <filename>.</filename>) where it
      looks.</para>

      <variablelist>

        <varlistentry>
          <term><option>-i&lt;dirs&gt;</option></term>
          <listitem>
            <para><indexterm><primary><option>-i&lt;dirs&gt;</option>
            </primary></indexterm>This flag appends a colon-separated
            list of <filename>dirs</filename> to the &ldquo;import
            directories&rdquo; list, which initially contains a single
            entry: <quote>.</quote>.</para>

            <para>This list is scanned before any package directories
            (see <xref linkend="packages">) when looking for imports,
            but note that if you have a home module with the same name
            as a package module then this is likely to cause trouble
            in other ways, with link errors being the least nasty
            thing that can go wrong...</para>

            <para>See also <XRef LinkEnd="recomp"> for the
            significance of using relative and absolute pathnames in
            the <option>-i</option> list.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>-i</option></term>
          <listitem>
            <para>resets the &ldquo;import directories&rdquo; list
            back to nothing.</para>
          </listitem>
        </varlistentry>

      </variablelist>

      <para>See also the section on packages (<xref
      linkend="packages">), which describes how to use installed
      libraries.</para>

    </sect2>

    <Sect2 id="hi-options">
      <title>Other options related to interface files</title>
      <indexterm><primary>interface files, options</primary></indexterm>

      <variablelist>
        <varlistentry>
          <term><option>-ddump-hi</option></term>
          <indexterm><primary><option>-ddump-hi</option></primary>
          </indexterm>
          <listitem>
            <para>Dumps the new interface to standard output.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>-ddump-hi-diffs</option></term>
          <indexterm><primary><option>-ddump-hi-diffs</option></primary>
          </indexterm>
          <listitem>
            <para>The compiler does not overwrite an existing
            <filename>.hi</filename> interface file if the new one is
            the same as the old one; this is friendly to
            <command>make</command>.  When an interface does change,
            it is often enlightening to be informed.  The
            <option>-ddump-hi-diffs</option> option will make GHC run
            <command>diff</command> on the old and new
            <filename>.hi</filename> files.</para>
          </listitem>
        </varlistentry>

        <varlistentry>
          <term><option>-ddump-minimal-imports</option></term>
          <indexterm><primary><option>-ddump-minimal-imports</option></primary>
          </indexterm>
          <listitem>
            <para>Dump to the file "M.imports" (where M is the module
            being compiled) a "minimal" set of import declarations.
            You can safely replace all the import declarations in
            "M.hs" with those found in "M.imports".  Why would you
            want to do that?  Because the "minimal" imports (a) import
            everything explicitly, by name, and (b) import nothing
            that is not required.  It can be quite painful to maintain
            this property by hand, so this flag is intended to reduce
            the labour.</para>
          </listitem>
        </varlistentry>
      </variablelist>
        
    </sect2>

    <sect2 id="recomp">
      <title>The recompilation checker</title>

      <indexterm><primary>recompilation checker</primary></indexterm>

      <variablelist>
        <varlistentry>
          <term><option>-no-recomp</option></term>
          <indexterm><primary><option>-recomp</option></primary></indexterm>
          <indexterm><primary><option>-no-recomp</option></primary></indexterm>
          <listitem>
            <para>Turn off recompilation checking (which is on by
            default).  Recompilation checking normally stops
            compilation early, leaving an existing
            <filename>.o</filename> file in place, if it can be
            determined that the module does not need to be
            recompiled.</para>
          </listitem>
        </varlistentry>
      </variablelist>
      
      <para>In the olden days, GHC compared the newly-generated
      <filename>.hi</filename> file with the previous version; if they
      were identical, it left the old one alone and didn't change its
      modification date.  In consequence, importers of a module with
      an unchanged output <filename>.hi</filename> file were not
      recompiled.</para>

      <para>This doesn't work any more.  Suppose module
      <literal>C</literal> imports module <literal>B</literal>, and
      <literal>B</literal> imports module <literal>A</literal>.  So
      changes to <filename>A.hi</filename> should force a
      recompilation of <literal>C</literal>.  And some changes to
      <literal>A</literal> (changing the definition of a function that
      appears in an inlining of a function exported by
      <literal>B</literal>, say) may conceivably not change
      <filename>B.hi</filename> one jot.  So now&hellip;</para>

      <para>GHC keeps a version number on each interface file, and on
      each type signature within the interface file.  It also keeps in
      every interface file a list of the version numbers of everything
      it used when it last compiled the file.  If the source file's
      modification date is earlier than the <filename>.o</filename>
      file's date (i.e. the source hasn't changed since the file was
      last compiled), and the reompilation checking is on, GHC will be
      clever.  It compares the version numbers on the things it needs
      this time with the version numbers on the things it needed last
      time (gleaned from the interface file of the module being
      compiled); if they are all the same it stops compiling rather
      early in the process saying &ldquo;Compilation IS NOT
      required&rdquo;.  What a beautiful sight!</para>

      <para>Patrick Sansom had a workshop paper about how all this is
      done (though the details have changed quite a bit). <ULink
      URL="mailto:sansom@dcs.gla.ac.uk">Ask him</ULink> if you want a
      copy.</para>

    </sect2>

    <sect2 id="using-make">
      <title>Using <command>make</command></title>

      <indexterm><primary><literal>make</literal></primary></indexterm>

      <para>It is reasonably straightforward to set up a
      <filename>Makefile</filename> to use with GHC, assuming you name
      your source files the same as your modules.  Thus:</para>

<ProgramListing>
HC      = ghc
HC_OPTS = -cpp $(EXTRA_HC_OPTS)

SRCS = Main.lhs Foo.lhs Bar.lhs
OBJS = Main.o   Foo.o   Bar.o

.SUFFIXES : .o .hs .hi .lhs .hc .s

cool_pgm : $(OBJS)
        rm $@
        $(HC) -o $@ $(HC_OPTS) $(OBJS)

# Standard suffix rules
.o.hi:
        @:

.lhs.o:
        $(HC) -c $&#60; $(HC_OPTS)

.hs.o:
        $(HC) -c $&#60; $(HC_OPTS)

# Inter-module dependencies
Foo.o Foo.hc Foo.s    : Baz.hi          # Foo imports Baz
Main.o Main.hc Main.s : Foo.hi Baz.hi   # Main imports Foo and Baz
</ProgramListing>

      <para>(Sophisticated <command>make</command> variants may
      achieve some of the above more elegantly.  Notably,
      <command>gmake</command>'s pattern rules let you write the more
      comprehensible:</para>

<ProgramListing>
%.o : %.lhs
        $(HC) -c $&#60; $(HC_OPTS)
</ProgramListing>

      <para>What we've shown should work with any
      <command>make</command>.)</para>

      <para>Note the cheesy <literal>.o.hi</literal> rule: It records
      the dependency of the interface (<filename>.hi</filename>) file
      on the source.  The rule says a <filename>.hi</filename> file
      can be made from a <filename>.o</filename> file by
      doing&hellip;nothing.  Which is true.</para>

      <para>Note the inter-module dependencies at the end of the
      Makefile, which take the form</para>

<ProgramListing>
Foo.o Foo.hc Foo.s    : Baz.hi          # Foo imports Baz
</ProgramListing>

      <para>They tell <command>make</command> that if any of
      <literal>Foo.o</literal>, <literal>Foo.hc</literal> or
      <literal>Foo.s</literal> have an earlier modification date than
      <literal>Baz.hi</literal>, then the out-of-date file must be
      brought up to date.  To bring it up to date,
      <literal>make</literal> looks for a rule to do so; one of the
      preceding suffix rules does the job nicely.</para>

      <sect3 id="sec-makefile-dependencies">
        <title>Dependency generation</title>
        <indexterm><primary>dependencies in Makefiles</primary></indexterm>
        <indexterm><primary>Makefile dependencies</primary></indexterm>

        <para>Putting inter-dependencies of the form <literal>Foo.o :
        Bar.hi</literal> into your <filename>Makefile</filename> by
        hand is rather error-prone.  Don't worry, GHC has support for
        automatically generating the required dependencies.  Add the
        following to your <filename>Makefile</filename>:</para>

<ProgramListing>
depend :
        ghc -M $(HC_OPTS) $(SRCS)
</ProgramListing>

        <para>Now, before you start compiling, and any time you change
        the <literal>imports</literal> in your program, do
        <command>make depend</command> before you do <command>make
        cool&lowbar;pgm</command>.  <command>ghc -M</command> will
        append the needed dependencies to your
        <filename>Makefile</filename>.</para>

        <para>In general, if module <literal>A</literal> contains the
        line

<programlisting>
import B ...blah...
</programlisting>

        then <command>ghc -M</command> will generate a dependency line
        of the form:

<programlisting>
A.o : B.hi
</programlisting>

        If module <literal>A</literal> contains the line

<programlisting>
import {-# SOURCE #-} B ...blah...
</programlisting>

        then <command>ghc -M</command> will generate a dependency
        line of the form:

<programlisting>
A.o : B.hi-boot
</programlisting>

       (See <xref linkend="mutual-recursion"> for details of
       <literal>hi-boot</literal> style interface files.)  If
       <literal>A</literal> imports multiple modules, then there will
       be multiple lines with <filename>A.o</filename> as the
       target.</para>

        <para>By default, <command>ghc -M</command> generates all the
        dependencies, and then concatenates them onto the end of
        <filename>makefile</filename> (or
        <filename>Makefile</filename> if <filename>makefile</filename>
        doesn't exist) bracketed by the lines "<literal>&num; DO NOT
        DELETE: Beginning of Haskell dependencies</literal>" and
        "<literal>&num; DO NOT DELETE: End of Haskell
        dependencies</literal>".  If these lines already exist in the
        <filename>makefile</filename>, then the old dependencies are
        deleted first.</para>

        <para>The dependency generation phase of GHC can take some
        additional options, which you may find useful.  For historical
        reasons, each option passed to the dependency generator from
        the GHC command line must be preceded by
        <literal>-optdep</literal>.  For example, to pass <literal>-f
        .depend</literal> to the dependency generator, you say

<screen>
ghc -M -optdep-f -optdep.depend ...
</screen>
      
        The options which affect dependency generation are:</para>
        
        <variablelist>
          <varlistentry>
            <term><option>-w</option></term>
            <listitem>
              <para>Turn off warnings about interface file shadowing.</para>
            </listitem>
          </varlistentry>
          
          <varlistentry>
            <term><option>-f</option> <replaceable>file</replaceable></term>
            <listitem>
              <para>Use <replaceable>file</replaceable> as the makefile,
              rather than <filename>makefile</filename> or
              <filename>Makefile</filename>.  If
              <replaceable>file</replaceable> doesn't exist,
              <command>mkdependHS</command> creates it.  We often use
              <option>-f .depend</option> to put the dependencies in
              <filename>.depend</filename> and then
              <command>include</command> the file
              <filename>.depend</filename> into
              <filename>Makefile</filename>.</para>
            </listitem>
          </varlistentry>

          <varlistentry>
            <term><option>-o &lt;osuf&gt;</option></term>
            <listitem>
              <para>Use <filename>.&lt;osuf&gt;</filename> as the
              "target file" suffix ( default: <literal>o</literal>).
              Multiple <option>-o</option> flags are permitted
              (GHC2.05 onwards).  Thus "<option>-o hc -o o</option>"
              will generate dependencies for <filename>.hc</filename>
              and <filename>.o</filename> files.</para>
            </listitem>
          </varlistentry>

          <varlistentry>
            <term><option>-s &lt;suf&gt;</option></term>
            <listitem>
              <para>Make extra dependencies that declare that files
              with suffix
              <filename>.&lt;suf&gt;&lowbar;&lt;osuf&gt;</filename>
              depend on interface files with suffix
              <filename>.&lt;suf&gt;&lowbar;hi</filename>, or (for
              <literal>&lcub;-&num; SOURCE &num;-&rcub;</literal>
              imports) on <filename>.hi-boot</filename>.  Multiple
              <option>-s</option> flags are permitted.  For example,
              <option>-o hc -s a -s b</option> will make dependencies
              for <filename>.hc</filename> on
              <filename>.hi</filename>,
              <filename>.a&lowbar;hc</filename> on
              <filename>.a&lowbar;hi</filename>, and
              <filename>.b&lowbar;hc</filename> on
              <filename>.b&lowbar;hi</filename>.  (Useful in
              conjunction with NoFib "ways".)</para>
            </listitem>
          </varlistentry>

          <varlistentry>
            <term><option>--exclude-module=&lt;file&gt;</option></term>
            <listitem>
              <para>Regard <filename>&lt;file&gt;</filename> as
              "stable"; i.e., exclude it from having dependencies on
              it.</para>
            </listitem>
          </varlistentry>

          <varlistentry>
            <term><option>-x</option></term>
            <listitem>
              <para>same as <option>--exclude-module</option></para>
            </listitem>
          </varlistentry>

          <varlistentry>
            <term><option>--exclude-directory=&lt;dirs&gt;</option></term>
            <listitem>
              <para>Regard the colon-separated list of directories
              <filename>&lt;dirs&gt;</filename> as containing stable,
              don't generate any dependencies on modules
              therein.</para>
            </listitem>
          </varlistentry>

          <varlistentry>
            <term><option>--include-module=&lt;file&gt;</option></term>
            <listitem>
              <para>Regard <filename>&lt;file&gt;</filename> as not
              "stable"; i.e., generate dependencies on it (if
              any). This option is normally used in conjunction with
              the <option>--exclude-directory</option> option.</para>
            </listitem>
          </varlistentry>

          <varlistentry>
            <term><option>--include-prelude</option></term>
            <listitem>
              <para>Regard prelude libraries as unstable, i.e.,
              generate dependencies on the prelude modules used
              (including <literal>Prelude</literal>).  This option is
              normally only used by the various system libraries. If a
              <option>-package</option> option is used, dependencies
              will also be generated on the library's
              interfaces.</para>
            </listitem>
          </varlistentry>
        </variablelist>

      </sect3>
    </sect2>

    <sect2 id="mutual-recursion">
      <title>How to compile mutually recursive modules</title>

      <indexterm><primary>module system, recursion</primary></indexterm>
      <indexterm><primary>recursion, between modules</primary></indexterm>

      <para>Currently, the compiler does not have proper support for
      dealing with mutually recursive modules:</para>

<ProgramListing>
module A where

import B

newtype TA = MkTA Int

f :: TB -&#62; TA
f (MkTB x) = MkTA x
--------
module B where

import A

data TB = MkTB !Int

g :: TA -&#62; TB
g (MkTA x) = MkTB x
</ProgramListing>

      <para>When compiling either module A and B, the compiler will
      try (in vain) to look for the interface file of the other. So,
      to get mutually recursive modules off the ground, you need to
      hand write an interface file for A or B, so as to break the
      loop.  These hand-written interface files are called
      <literal>hi-boot</literal> files, and are placed in a file
      called <filename>&lt;module&gt;.hi-boot</filename>.  To import
      from an <literal>hi-boot</literal> file instead of the standard
      <filename>.hi</filename> file, use the following syntax in the
      importing module: <indexterm><primary><literal>hi-boot</literal>
      files</primary></indexterm> <indexterm><primary>importing,
      <literal>hi-boot</literal> files</primary></indexterm></para>

<ProgramListing>
import {-# SOURCE #-} A
</ProgramListing>

      <para>The hand-written interface need only contain the bare
      minimum of information needed to get the bootstrapping process
      started.  For example, it doesn't need to contain declarations
      for <emphasis>everything</emphasis> that module
      <literal>A</literal> exports, only the things required by the
      module that imports <literal>A</literal> recursively.</para>

      <para>For the example at hand, the boot interface file for A
      would look like the following:</para>

<ProgramListing>
__interface A 1 0 where
__export A TA{MkTA} ;
1 newtype TA = MkTA PrelBase.Int ;
</ProgramListing>

      <para>The syntax is essentially the same as a normal
      <filename>.hi</filename> file (unfortunately), so you can
      usually tailor an existing <filename>.hi</filename> file to make
      a <filename>.hi-boot</filename> file.</para>

      <para>Notice that we only put the declaration for the newtype
      <literal>TA</literal> in the <literal>hi-boot</literal> file,
      not the signature for <Function>f</Function>, since
      <Function>f</Function> isn't used by <literal>B</literal>.</para>

      <para>The number &ldquo;1&rdquo; after
      &ldquo;&lowbar;&lowbar;interface A&rdquo; gives the version
      number of module A; it is incremented whenever anything in A's
      interface file changes.  In a normal interface file, the
      &ldquo;0&rdquo; is the version number of the compiler which
      generated the interface file; it is used to ensure that we don't
      mix-and-match interface files between compiler versions.
      Leaving it as zero in an <literal>hi-boot</literal> file turns
      off this check.</para>

      <para>The number &ldquo;1&rdquo; at the beginning of a
      declaration is the <emphasis>version number</emphasis> of that
      declaration: for the purposes of <filename>.hi-boot</filename>
      files these can all be set to 1.  All names must be fully
      qualified with the <emphasis>original</emphasis> module that an
      object comes from: for example, the reference to
      <literal>Int</literal> in the interface for <literal>A</literal>
      comes from <literal>PrelBase</literal>, which is a module
      internal to GHC's prelude.  It's a pain, but that's the way it
      is.</para>

      <para>If you want an <literal>hi-boot</literal> file to export a
      data type, but you don't want to give its constructors (because
      the constructors aren't used by the SOURCE-importing module),
      you can write simply:</para>

<ProgramListing>
__interface A 1 0 where
__export A TA;
1 data TA
</ProgramListing>

      <para>(You must write all the type parameters, but leave out the
      '=' and everything that follows it.)</para>

      <para><emphasis>Note:</emphasis> This is all a temporary
      solution, a version of the compiler that handles mutually
      recursive modules properly without the manual construction of
      interface files, is (allegedly) in the works.</para>
    </sect2>


    <sect2 id="orphan-modules">
      <title>Orphan modules and instance declarations</title>

<para> Haskell specifies that when compiling module M, any instance
declaration in any module "below" M is visible.  (Module A is "below"
M if A is imported directly by M, or if A is below a module that M imports directly.)
In principle, GHC must therefore read the interface files of every module below M,
just in case they contain an instance declaration that matters to M.  This would
be a disaster in practice, so GHC tries to be clever. </para>

<para>In particular, if an instance declaration is in the same module as the definition
of any type or class mentioned in the head of the instance declaration, then
GHC has to visit that interface file anyway.  Example:</para>
<ProgramListing>
  module A where
    instance C a =&gt; D (T a) where ...
    data T a = ...
</ProgramListing>
<para> The instance declaration is only relevant if the type T is in use, and if
so, GHC will have visited A's interface file to find T's definition. </para>

<para> The only problem comes when a module contains an instance declaration
and GHC has no other reason for visiting the module.  Example:
<ProgramListing>
  module Orphan where
    instance C a =&gt; D (T a) where ...
    class C a where ...
</ProgramListing>
Here, neither D nor T is declared in module Orphan.
We call such modules ``orphan modules'',
defined thus:</para>
<itemizedlist>
  <listitem> <para> An <emphasis>orphan module</emphasis> 
  <indexterm><primary>orphan module</primary></indexterm>
  contains at least one <emphasis>orphan instance</emphasis> or at 
  least one <emphasis>orphan rule</emphasis>.</para> </listitem>

  <listitem><para> An instance declaration in a module M is an <emphasis>orphan instance</emphasis> if
  <indexterm><primary>orphan instance</primary></indexterm>
  none of the type constructors
  or classes mentioned in the instance head (the part after the ``<literal>=&gt;</literal>'') are declared
  in M.</para> 

  <para> Only the instance head counts.  In the example above, it is not good enough for C's declaration 
  to be in module A; it must be the declaration of D or T.</para>
  </listitem>

  <listitem><para> A rewrite rule in a module M is an <emphasis>orphan rule</emphasis>
  <indexterm><primary>orphan rule</primary></indexterm>
  if none of the variables, type constructors,
  or classes that are free in the left hand side of the rule are declared in M.
  </para> </listitem>
 </itemizedlist>


<para> GHC identifies orphan modules, and visits the interface file of
every orphan module below the module being compiled.  This is usually
wasted work, but there is no avoiding it.  You should therefore do
your best to have as few orphan modules as possible.

</para><para>
You can identify an orphan module by looking in its interface file, M.hi.  If there is a ``!'' on
the first line, GHC considers it an orphan module. 
</para>
</sect2>

  </sect1>

<!-- Emacs stuff:
     ;;; Local Variables: ***
     ;;; mode: sgml ***
     ;;; sgml-parent-document: ("using.sgml" "book" "chapter") ***
     ;;; End: ***
 -->