diff options
Diffstat (limited to 'doc/user/caching.in')
| -rw-r--r-- | doc/user/caching.in | 502 |
1 files changed, 0 insertions, 502 deletions
diff --git a/doc/user/caching.in b/doc/user/caching.in deleted file mode 100644 index 8611962..0000000 --- a/doc/user/caching.in +++ /dev/null @@ -1,502 +0,0 @@ -<!-- - - Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013 The SCons Foundation - - Permission is hereby granted, free of charge, to any person obtaining - a copy of this software and associated documentation files (the - "Software"), to deal in the Software without restriction, including - without limitation the rights to use, copy, modify, merge, publish, - distribute, sublicense, and/or sell copies of the Software, and to - permit persons to whom the Software is furnished to do so, subject to - the following conditions: - - The above copyright notice and this permission notice shall be included - in all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY - KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE - WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND - NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE - LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION - OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION - WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - ---> - - <para> - - On multi-developer software projects, - you can sometimes speed up every developer's builds a lot by - allowing them to share the derived files that they build. - &SCons; makes this easy, as well as reliable. - - </para> - - <section> - <title>Specifying the Shared Cache Directory</title> - - <para> - - To enable sharing of derived files, - use the &CacheDir; function - in any &SConscript; file: - - </para> - - <scons_example name="ex1"> - <file name="SConstruct"> - env = Environment() - env.Program('hello.c') - CacheDir('cache') - </file> - <file name="hello.c"> - hello.c - </file> - <directory name="cache"> - </directory> - <file name="not_used" printme="1"> - CacheDir('/usr/local/build_cache') - </file> - </scons_example> - - <para> - - Note that the directory you specify must already exist - and be readable and writable by all developers - who will be sharing derived files. - It should also be in some central location - that all builds will be able to access. - In environments where developers are using separate systems - (like individual workstations) for builds, - this directory would typically be - on a shared or NFS-mounted file system. - - </para> - - <para> - - Here's what happens: - When a build has a &CacheDir; specified, - every time a file is built, - it is stored in the shared cache directory - along with its MD5 build signature. - <footnote> - <para> - Actually, the MD5 signature is used as the name of the file - in the shared cache directory in which the contents are stored. - </para> - </footnote> - On subsequent builds, - before an action is invoked to build a file, - &SCons; will check the shared cache directory - to see if a file with the exact same build - signature already exists. - If so, the derived file will not be built locally, - but will be copied into the local build directory - from the shared cache directory, - like so: - - </para> - - <scons_output example="ex1"> - <scons_output_command>scons -Q</scons_output_command> - <scons_output_command>scons -Q -c</scons_output_command> - <scons_output_command>scons -Q</scons_output_command> - </scons_output> - - <para> - - Note that the &CacheDir; feature still calculates - MD5 build sigantures for the shared cache file names - even if you configure &SCons; to use timestamps - to decide if files are up to date. - (See the <xref linkend="chap-depends"></xref> - chapter for information about the &Decider; function.) - Consequently, using &CacheDir; may reduce or eliminate any - potential performance improvements - from using timestamps for up-to-date decisions. - - </para> - - </section> - - <section> - <title>Keeping Build Output Consistent</title> - - <para> - - One potential drawback to using a shared cache - is that the output printed by &SCons; - can be inconsistent from invocation to invocation, - because any given file may be rebuilt one time - and retrieved from the shared cache the next time. - This can make analyzing build output more difficult, - especially for automated scripts that - expect consistent output each time. - - </para> - - <para> - - If, however, you use the <literal>--cache-show</literal> option, - &SCons; will print the command line that it - <emphasis>would</emphasis> have executed - to build the file, - even when it is retrieving the file from the shared cache. - This makes the build output consistent - every time the build is run: - - </para> - - <scons_output example="ex1"> - <scons_output_command>scons -Q</scons_output_command> - <scons_output_command>scons -Q -c</scons_output_command> - <scons_output_command>scons -Q --cache-show</scons_output_command> - </scons_output> - - <para> - - The trade-off, of course, is that you no longer - know whether or not &SCons; - has retrieved a derived file from cache - or has rebuilt it locally. - - </para> - - </section> - - <section> - <title>Not Using the Shared Cache for Specific Files</title> - - <para> - - You may want to disable caching for certain - specific files in your configuration. - For example, if you only want to put - executable files in a central cache, - but not the intermediate object files, - you can use the &NoCache; - function to specify that the - object files should not be cached: - - </para> - - <scons_example name="ex-NoCache"> - <file name="SConstruct" printme="1"> - env = Environment() - obj = env.Object('hello.c') - env.Program('hello.c') - CacheDir('cache') - NoCache('hello.o') - </file> - <file name="hello.c"> - hello.c - </file> - <directory name="cache"> - </directory> - </scons_example> - - <para> - - Then when you run &scons; after cleaning - the built targets, - it will recompile the object file locally - (since it doesn't exist in the shared cache directory), - but still realize that the shared cache directory - contains an up-to-date executable program - that can be retrieved instead of re-linking: - - </para> - - <!-- - - <scons_output example="ex1"> - <scons_output_command>scons -Q</scons_output_command> - <scons_output_command>scons -Q -c</scons_output_command> - <scons_output_command>scons -Q</scons_output_command> - </scons_output> - - --> - - <screen> - % <userinput>scons -Q</userinput> - cc -o hello.o -c hello.c - cc -o hello hello.o - % <userinput>scons -Q -c</userinput> - Removed hello.o - Removed hello - % <userinput>scons -Q</userinput> - cc -o hello.o -c hello.c - Retrieved `hello' from cache - </screen> - - </section> - - <section> - <title>Disabling the Shared Cache</title> - - <para> - - Retrieving an already-built file - from the shared cache - is usually a significant time-savings - over rebuilding the file, - but how much of a savings - (or even whether it saves time at all) - can depend a great deal on your - system or network configuration. - For example, retrieving cached files - from a busy server over a busy network - might end up being slower than - rebuilding the files locally. - - </para> - - <para> - - In these cases, you can specify - the <literal>--cache-disable</literal> - command-line option to tell &SCons; - to not retrieve already-built files from the - shared cache directory: - - </para> - - <scons_output example="ex1"> - <scons_output_command>scons -Q</scons_output_command> - <scons_output_command>scons -Q -c</scons_output_command> - <scons_output_command>scons -Q</scons_output_command> - <scons_output_command>scons -Q -c</scons_output_command> - <scons_output_command>scons -Q --cache-disable</scons_output_command> - </scons_output> - - </section> - - <section> - <title>Populating a Shared Cache With Already-Built Files</title> - - <para> - - Sometimes, you may have one or more derived files - already built in your local build tree - that you wish to make available to other people doing builds. - For example, you may find it more effective to perform - integration builds with the cache disabled - (per the previous section) - and only populate the shared cache directory - with the built files after the integration build - has completed successfully. - This way, the cache will only get filled up - with derived files that are part of a complete, successful build - not with files that might be later overwritten - while you debug integration problems. - - </para> - - <para> - - In this case, you can use the - the <literal>--cache-force</literal> option - to tell &SCons; to put all derived files in the cache, - even if the files already exist in your local tree - from having been built by a previous invocation: - - </para> - - <scons_output example="ex1"> - <scons_output_command>scons -Q --cache-disable</scons_output_command> - <scons_output_command>scons -Q -c</scons_output_command> - <scons_output_command>scons -Q --cache-disable</scons_output_command> - <scons_output_command>scons -Q --cache-force</scons_output_command> - <scons_output_command>scons -Q</scons_output_command> - </scons_output> - - <para> - - Notice how the above sample run - demonstrates that the <literal>--cache-disable</literal> - option avoids putting the built - <filename>hello.o</filename> - and - <filename>hello</filename> files in the cache, - but after using the <literal>--cache-force</literal> option, - the files have been put in the cache - for the next invocation to retrieve. - - </para> - - </section> - - <section> - <title>Minimizing Cache Contention: the <literal>--random</literal> Option</title> - - <para> - - If you allow multiple builds to update the - shared cache directory simultaneously, - two builds that occur at the same time - can sometimes start "racing" - with one another to build the same files - in the same order. - If, for example, - you are linking multiple files into an executable program: - - </para> - - <scons_example name="ex-random"> - <file name="SConstruct" printme="1"> - Program('prog', - ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c']) - </file> - <file name="f1.c">f1.c</file> - <file name="f2.c">f2.c</file> - <file name="f3.c">f3.c</file> - <file name="f4.c">f4.c</file> - <file name="f5.c">f5.c</file> - <file name="f6.c">f6.c</file> - </scons_example> - - <para> - - &SCons; will normally build the input object files - on which the program depends in their normal, sorted order: - - </para> - - <scons_output example="ex-random"> - <scons_output_command>scons -Q</scons_output_command> - </scons_output> - - <para> - - But if two such builds take place simultaneously, - they may each look in the cache at nearly the same - time and both decide that <filename>f1.o</filename> - must be rebuilt and pushed into the shared cache directory, - then both decide that <filename>f2.o</filename> - must be rebuilt (and pushed into the shared cache directory), - then both decide that <filename>f3.o</filename> - must be rebuilt... - This won't cause any actual build problems--both - builds will succeed, - generate correct output files, - and populate the cache--but - it does represent wasted effort. - - </para> - - <para> - - To alleviate such contention for the cache, - you can use the <literal>--random</literal> command-line option - to tell &SCons; to build dependencies - in a random order: - - </para> - - <!-- - - The following <screen> output was generated by this: - - <scons_output example="ex-random"> - <scons_output_command>scons -Q - -random</scons_output_command> - </scons_output> - - We captured it directly here to guarantee a "random" order, - guarding against the potential for - -random to happen - to return things in the original sorted order. - - --> - - <screen> - % <userinput>scons -Q --random</userinput> - cc -o f3.o -c f3.c - cc -o f1.o -c f1.c - cc -o f5.o -c f5.c - cc -o f2.o -c f2.c - cc -o f4.o -c f4.c - cc -o prog f1.o f2.o f3.o f4.o f5.o - </screen> - - <para> - - Multiple builds using the <literal>--random</literal> option - will usually build their dependencies in different, - random orders, - which minimizes the chances for a lot of - contention for same-named files - in the shared cache directory. - Multiple simultaneous builds might still race to try to build - the same target file on occasion, - but long sequences of inefficient contention - should be rare. - - </para> - - <para> - - Note, of course, - the <literal>--random</literal> option - will cause the output that &SCons; prints - to be inconsistent from invocation to invocation, - which may be an issue when - trying to compare output from different build runs. - - </para> - - <para> - - If you want to make sure dependencies will be built - in a random order without having to specify - the <literal>--random</literal> on very command line, - you can use the &SetOption; function to - set the <literal>random</literal> option - within any &SConscript; file: - - </para> - - <scons_example name="ex-random"> - <file name="SConstruct" printme="1"> - SetOption('random', 1) - Program('prog', - ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c']) - </file> - <file name="f1.c">f1.c</file> - <file name="f2.c">f2.c</file> - <file name="f3.c">f3.c</file> - <file name="f4.c">f4.c</file> - <file name="f5.c">f5.c</file> - <file name="f6.c">f6.c</file> - </scons_example> - - </section> - - <!-- - - <section> - <title>Troubleshooting Shared Caching: the &cache-debug; Option</title> - - <para> - - XXX describe the - - cache-debug option - XXX maybe point to the troubleshooting appendix? - - </para> - - </section> - - --> - - <!-- - - <section> - - <para> - - XXX describe CacheDir management: monitoring, deleting, etc. - - </para> - - </section> - - --> |