summaryrefslogtreecommitdiff
path: root/doc/user/caching.in
diff options
context:
space:
mode:
authorLuca Falavigna <dktrkranz@debian.org>2014-04-26 15:11:58 +0200
committerLuca Falavigna <dktrkranz@debian.org>2014-04-26 15:11:58 +0200
commit140d836e9cd54fb67b969fd82ef7ed19ba574d40 (patch)
tree0df3e32ee39603d43f9b90fd2f2e1f7cce4249d4 /doc/user/caching.in
parentcb3425abe0bc2d05caf401ca24b82a25a81f009d (diff)
Imported Upstream version 2.3.1upstream/2.3.1
Diffstat (limited to 'doc/user/caching.in')
-rw-r--r--doc/user/caching.in502
1 files changed, 0 insertions, 502 deletions
diff --git a/doc/user/caching.in b/doc/user/caching.in
deleted file mode 100644
index 8611962..0000000
--- a/doc/user/caching.in
+++ /dev/null
@@ -1,502 +0,0 @@
-<!--
-
- Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013 The SCons Foundation
-
- Permission is hereby granted, free of charge, to any person obtaining
- a copy of this software and associated documentation files (the
- "Software"), to deal in the Software without restriction, including
- without limitation the rights to use, copy, modify, merge, publish,
- distribute, sublicense, and/or sell copies of the Software, and to
- permit persons to whom the Software is furnished to do so, subject to
- the following conditions:
-
- The above copyright notice and this permission notice shall be included
- in all copies or substantial portions of the Software.
-
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
- KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
- WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
- NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
- LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
- OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
- WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-
--->
-
- <para>
-
- On multi-developer software projects,
- you can sometimes speed up every developer's builds a lot by
- allowing them to share the derived files that they build.
- &SCons; makes this easy, as well as reliable.
-
- </para>
-
- <section>
- <title>Specifying the Shared Cache Directory</title>
-
- <para>
-
- To enable sharing of derived files,
- use the &CacheDir; function
- in any &SConscript; file:
-
- </para>
-
- <scons_example name="ex1">
- <file name="SConstruct">
- env = Environment()
- env.Program('hello.c')
- CacheDir('cache')
- </file>
- <file name="hello.c">
- hello.c
- </file>
- <directory name="cache">
- </directory>
- <file name="not_used" printme="1">
- CacheDir('/usr/local/build_cache')
- </file>
- </scons_example>
-
- <para>
-
- Note that the directory you specify must already exist
- and be readable and writable by all developers
- who will be sharing derived files.
- It should also be in some central location
- that all builds will be able to access.
- In environments where developers are using separate systems
- (like individual workstations) for builds,
- this directory would typically be
- on a shared or NFS-mounted file system.
-
- </para>
-
- <para>
-
- Here's what happens:
- When a build has a &CacheDir; specified,
- every time a file is built,
- it is stored in the shared cache directory
- along with its MD5 build signature.
- <footnote>
- <para>
- Actually, the MD5 signature is used as the name of the file
- in the shared cache directory in which the contents are stored.
- </para>
- </footnote>
- On subsequent builds,
- before an action is invoked to build a file,
- &SCons; will check the shared cache directory
- to see if a file with the exact same build
- signature already exists.
- If so, the derived file will not be built locally,
- but will be copied into the local build directory
- from the shared cache directory,
- like so:
-
- </para>
-
- <scons_output example="ex1">
- <scons_output_command>scons -Q</scons_output_command>
- <scons_output_command>scons -Q -c</scons_output_command>
- <scons_output_command>scons -Q</scons_output_command>
- </scons_output>
-
- <para>
-
- Note that the &CacheDir; feature still calculates
- MD5 build sigantures for the shared cache file names
- even if you configure &SCons; to use timestamps
- to decide if files are up to date.
- (See the <xref linkend="chap-depends"></xref>
- chapter for information about the &Decider; function.)
- Consequently, using &CacheDir; may reduce or eliminate any
- potential performance improvements
- from using timestamps for up-to-date decisions.
-
- </para>
-
- </section>
-
- <section>
- <title>Keeping Build Output Consistent</title>
-
- <para>
-
- One potential drawback to using a shared cache
- is that the output printed by &SCons;
- can be inconsistent from invocation to invocation,
- because any given file may be rebuilt one time
- and retrieved from the shared cache the next time.
- This can make analyzing build output more difficult,
- especially for automated scripts that
- expect consistent output each time.
-
- </para>
-
- <para>
-
- If, however, you use the <literal>--cache-show</literal> option,
- &SCons; will print the command line that it
- <emphasis>would</emphasis> have executed
- to build the file,
- even when it is retrieving the file from the shared cache.
- This makes the build output consistent
- every time the build is run:
-
- </para>
-
- <scons_output example="ex1">
- <scons_output_command>scons -Q</scons_output_command>
- <scons_output_command>scons -Q -c</scons_output_command>
- <scons_output_command>scons -Q --cache-show</scons_output_command>
- </scons_output>
-
- <para>
-
- The trade-off, of course, is that you no longer
- know whether or not &SCons;
- has retrieved a derived file from cache
- or has rebuilt it locally.
-
- </para>
-
- </section>
-
- <section>
- <title>Not Using the Shared Cache for Specific Files</title>
-
- <para>
-
- You may want to disable caching for certain
- specific files in your configuration.
- For example, if you only want to put
- executable files in a central cache,
- but not the intermediate object files,
- you can use the &NoCache;
- function to specify that the
- object files should not be cached:
-
- </para>
-
- <scons_example name="ex-NoCache">
- <file name="SConstruct" printme="1">
- env = Environment()
- obj = env.Object('hello.c')
- env.Program('hello.c')
- CacheDir('cache')
- NoCache('hello.o')
- </file>
- <file name="hello.c">
- hello.c
- </file>
- <directory name="cache">
- </directory>
- </scons_example>
-
- <para>
-
- Then when you run &scons; after cleaning
- the built targets,
- it will recompile the object file locally
- (since it doesn't exist in the shared cache directory),
- but still realize that the shared cache directory
- contains an up-to-date executable program
- that can be retrieved instead of re-linking:
-
- </para>
-
- <!--
-
- <scons_output example="ex1">
- <scons_output_command>scons -Q</scons_output_command>
- <scons_output_command>scons -Q -c</scons_output_command>
- <scons_output_command>scons -Q</scons_output_command>
- </scons_output>
-
- -->
-
- <screen>
- % <userinput>scons -Q</userinput>
- cc -o hello.o -c hello.c
- cc -o hello hello.o
- % <userinput>scons -Q -c</userinput>
- Removed hello.o
- Removed hello
- % <userinput>scons -Q</userinput>
- cc -o hello.o -c hello.c
- Retrieved `hello' from cache
- </screen>
-
- </section>
-
- <section>
- <title>Disabling the Shared Cache</title>
-
- <para>
-
- Retrieving an already-built file
- from the shared cache
- is usually a significant time-savings
- over rebuilding the file,
- but how much of a savings
- (or even whether it saves time at all)
- can depend a great deal on your
- system or network configuration.
- For example, retrieving cached files
- from a busy server over a busy network
- might end up being slower than
- rebuilding the files locally.
-
- </para>
-
- <para>
-
- In these cases, you can specify
- the <literal>--cache-disable</literal>
- command-line option to tell &SCons;
- to not retrieve already-built files from the
- shared cache directory:
-
- </para>
-
- <scons_output example="ex1">
- <scons_output_command>scons -Q</scons_output_command>
- <scons_output_command>scons -Q -c</scons_output_command>
- <scons_output_command>scons -Q</scons_output_command>
- <scons_output_command>scons -Q -c</scons_output_command>
- <scons_output_command>scons -Q --cache-disable</scons_output_command>
- </scons_output>
-
- </section>
-
- <section>
- <title>Populating a Shared Cache With Already-Built Files</title>
-
- <para>
-
- Sometimes, you may have one or more derived files
- already built in your local build tree
- that you wish to make available to other people doing builds.
- For example, you may find it more effective to perform
- integration builds with the cache disabled
- (per the previous section)
- and only populate the shared cache directory
- with the built files after the integration build
- has completed successfully.
- This way, the cache will only get filled up
- with derived files that are part of a complete, successful build
- not with files that might be later overwritten
- while you debug integration problems.
-
- </para>
-
- <para>
-
- In this case, you can use the
- the <literal>--cache-force</literal> option
- to tell &SCons; to put all derived files in the cache,
- even if the files already exist in your local tree
- from having been built by a previous invocation:
-
- </para>
-
- <scons_output example="ex1">
- <scons_output_command>scons -Q --cache-disable</scons_output_command>
- <scons_output_command>scons -Q -c</scons_output_command>
- <scons_output_command>scons -Q --cache-disable</scons_output_command>
- <scons_output_command>scons -Q --cache-force</scons_output_command>
- <scons_output_command>scons -Q</scons_output_command>
- </scons_output>
-
- <para>
-
- Notice how the above sample run
- demonstrates that the <literal>--cache-disable</literal>
- option avoids putting the built
- <filename>hello.o</filename>
- and
- <filename>hello</filename> files in the cache,
- but after using the <literal>--cache-force</literal> option,
- the files have been put in the cache
- for the next invocation to retrieve.
-
- </para>
-
- </section>
-
- <section>
- <title>Minimizing Cache Contention: the <literal>--random</literal> Option</title>
-
- <para>
-
- If you allow multiple builds to update the
- shared cache directory simultaneously,
- two builds that occur at the same time
- can sometimes start "racing"
- with one another to build the same files
- in the same order.
- If, for example,
- you are linking multiple files into an executable program:
-
- </para>
-
- <scons_example name="ex-random">
- <file name="SConstruct" printme="1">
- Program('prog',
- ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
- </file>
- <file name="f1.c">f1.c</file>
- <file name="f2.c">f2.c</file>
- <file name="f3.c">f3.c</file>
- <file name="f4.c">f4.c</file>
- <file name="f5.c">f5.c</file>
- <file name="f6.c">f6.c</file>
- </scons_example>
-
- <para>
-
- &SCons; will normally build the input object files
- on which the program depends in their normal, sorted order:
-
- </para>
-
- <scons_output example="ex-random">
- <scons_output_command>scons -Q</scons_output_command>
- </scons_output>
-
- <para>
-
- But if two such builds take place simultaneously,
- they may each look in the cache at nearly the same
- time and both decide that <filename>f1.o</filename>
- must be rebuilt and pushed into the shared cache directory,
- then both decide that <filename>f2.o</filename>
- must be rebuilt (and pushed into the shared cache directory),
- then both decide that <filename>f3.o</filename>
- must be rebuilt...
- This won't cause any actual build problems--both
- builds will succeed,
- generate correct output files,
- and populate the cache--but
- it does represent wasted effort.
-
- </para>
-
- <para>
-
- To alleviate such contention for the cache,
- you can use the <literal>--random</literal> command-line option
- to tell &SCons; to build dependencies
- in a random order:
-
- </para>
-
- <!--
-
- The following <screen> output was generated by this:
-
- <scons_output example="ex-random">
- <scons_output_command>scons -Q - -random</scons_output_command>
- </scons_output>
-
- We captured it directly here to guarantee a "random" order,
- guarding against the potential for - -random to happen
- to return things in the original sorted order.
-
- -->
-
- <screen>
- % <userinput>scons -Q --random</userinput>
- cc -o f3.o -c f3.c
- cc -o f1.o -c f1.c
- cc -o f5.o -c f5.c
- cc -o f2.o -c f2.c
- cc -o f4.o -c f4.c
- cc -o prog f1.o f2.o f3.o f4.o f5.o
- </screen>
-
- <para>
-
- Multiple builds using the <literal>--random</literal> option
- will usually build their dependencies in different,
- random orders,
- which minimizes the chances for a lot of
- contention for same-named files
- in the shared cache directory.
- Multiple simultaneous builds might still race to try to build
- the same target file on occasion,
- but long sequences of inefficient contention
- should be rare.
-
- </para>
-
- <para>
-
- Note, of course,
- the <literal>--random</literal> option
- will cause the output that &SCons; prints
- to be inconsistent from invocation to invocation,
- which may be an issue when
- trying to compare output from different build runs.
-
- </para>
-
- <para>
-
- If you want to make sure dependencies will be built
- in a random order without having to specify
- the <literal>--random</literal> on very command line,
- you can use the &SetOption; function to
- set the <literal>random</literal> option
- within any &SConscript; file:
-
- </para>
-
- <scons_example name="ex-random">
- <file name="SConstruct" printme="1">
- SetOption('random', 1)
- Program('prog',
- ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])
- </file>
- <file name="f1.c">f1.c</file>
- <file name="f2.c">f2.c</file>
- <file name="f3.c">f3.c</file>
- <file name="f4.c">f4.c</file>
- <file name="f5.c">f5.c</file>
- <file name="f6.c">f6.c</file>
- </scons_example>
-
- </section>
-
- <!--
-
- <section>
- <title>Troubleshooting Shared Caching: the &cache-debug; Option</title>
-
- <para>
-
- XXX describe the - - cache-debug option
- XXX maybe point to the troubleshooting appendix?
-
- </para>
-
- </section>
-
- -->
-
- <!--
-
- <section>
-
- <para>
-
- XXX describe CacheDir management: monitoring, deleting, etc.
-
- </para>
-
- </section>
-
- -->