From 72c578fd4b0b4a5a43e18594339ac4ff26c376dc Mon Sep 17 00:00:00 2001 From: Luca Falavigna Date: Sat, 2 Jan 2010 20:56:27 +0100 Subject: Imported Upstream version 1.2.0.d20091224 --- doc/python10/design.xml | 898 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 898 insertions(+) create mode 100644 doc/python10/design.xml (limited to 'doc/python10/design.xml') diff --git a/doc/python10/design.xml b/doc/python10/design.xml new file mode 100644 index 0000000..cb58af9 --- /dev/null +++ b/doc/python10/design.xml @@ -0,0 +1,898 @@ + + + The &SCons; architecture consists of three layers: + + + + + + + + + + + + + + + + + + + The &SCons; Build Engine, a package of Python + modules that handle dependency management and updating out-of-date + objects. + + + + + + + + The &SCons; API (applications programming + interface) between the Build Engine + and the user interface. + + + + + + + + The &scons; script itself (note lower case + sc), which is the pre-provided interface to + the Build Engine. + + + + + + + + + Notice that this architecture separates the internal workings of + &SCons; (the Build Engine) from the + external user interface. The benefit is that the &SCons; Build Engine + can be imported into any other software package written in Python + to support a variety of user interfaces—or, to look at it + in reverse, other software interfaces can use the &SCons; Build + Engine to manage dependencies between their objects. + + + + + + Because the + &SCons; package itself is modular, only those parts of the package + relevant to the embedding interface need be imported; for example, + a utility that wants to use only file timestamps for checking + whether a file is up-to-date + need not import the MD5 signature module. + + + +
+ The &SCons; Build Engine + + + + The Build Engine is a package of Python modules that + form the heart of &SCons;. + + The Build Engine can be broadly divided into five + architectural subsystems, each responsible + for a crucial part of &SCons; functionality: + + + + + + + + + A node subsystem, responsible for managing + the files (or other objects) to be built, and the dependency + relationships between them. + + + + + + + + A scanner subsystem, responsible for + scanning various file types for implicit dependencies. + + + + + + + + A signature subsystem, responsible for + deciding whether a given file (or other object) requires + rebuilding. + + + + + + + + A builder subsystem, responsible for + actually executing the necessary command or function to + build a file (or other object). + + + + + + + + A job/task subsystem, responsible for + handling parallelization of builds. + + + + + + + + + The rest of this section will provide a high-level overview of the + class structure of each of these Build Engine subsystems. + + + +
+ Node Subsystem + + + + The node subsystem of the Build Engine is + responsible for managing the knowledge in &SCons; of + the relationships among the external objects + (files) it is responsible for updating. + The most important of these relationships is + the dependency relationship between various &Node; objects, + which &SCons; uses to determine the order + in which builds should be performed. + + + + + + + + + + + + + + + + The &scons; script (or other + user interface) + tells the Build Engine + about dependencies + through its &consenv; API. + The Build Engine also discovers + dependencies automatically through the use of &Scanner; objects. + + + + + + Subclasses of the &Node; class maintain additional + relationships that reflect the real-world + existence of these objects. + For example, the &Node_FS; subclass + is responsible for managing a + representation of the directory hierarchy + of a file system. + + + + + + A &Walker; class is used by other subsystems + to walk the dependency tree maintained by the &Node; class. + The &Walker; class maintains a stack of &Node; objects + visited during its depth-first traversal of the + dependency tree, + and uses an intermediate node &Wrapper; class + to maintain state information about a + &Node; object's dependencies. + + + +
+ +
+ Scanner Subsystem + + + + The scanner subsystem is responsible for maintaining + objects that can scan the contents of a &Node;'s + for implicit dependencies. + + + + + + + + + + + + + + + + In practice, a given &Scanner; subclass object + functions as a prototype, + returning clones of itself + depending on the &consenv; + values governing how the &Node; + should be scanned. + + + +
+ +
+ Signature Subsystem + + + + The signature subsystem is responsible for computing + signature information for &Node; objects. + The signature subsystem in &SCons; + supports multiple ways to + determine whether a &Node is up-to-date + by using an abstract &Sig; class + as a strategy wrapper: + + + + + + + + + + + + + + + + By default, &SCons; tracks dependencies by computing and + maintaining MD5 signatures for the contents of each source file + (or other object). The signature of a derived + file consists of the aggregate of the signatures of all the source + files plus the command-line string used to + build the file. These signatures are stored in a &sconsign; file + in each directory. + + + + + + If the contents of any of the source files changes, the change to its + MD5 signature is propogated to the signature of the derived file(s). The + simple fact that the new signature does not match the stored signature + indicates that the derived file is not up to date and must be rebuilt. + + + + + + A separate &TimeStamp; subclass of the &Sig; class supports + the use of traditional file timestamps for + deciding whether files are up-to-date. + + + +
+ +
+ Builder Subsystem + + + + The &SCons; Build Engine records how out-of-date files + (or other objects) should be rebuilt in &Builder; objects, + maintained by the builder subsystem: + + + + + + + + + + + + + + + + The actual underlying class name is &BuilderBase;, + and there are subclasses that can encapsulate + multiple &Builder; objects for special purposes. + One subclass + (&CompositeBuilder;) + selects an appropriate encapsulated &Builder; + based on the file suffix of the target object. + The other + (&MultiStepBuilder;). + can chain together multiple + &Builder; objects, + for example, + to build an executable program from a source file + through an implicit intermediate object file. + + + + + + A &BuilderBase; object has an associated + &ActionBase; object + responsible for actually executing + the appropriate steps + to update the target file. + There are three subclasses, + one for externally executable commands + (&CommandAction;), + one for Python functions + (&FunctionAction;), + and one for lists of + multiple &Action; objects + (&ListAction;). + + + +
+ +
+ Job/Task Subsystem + + + + &SCons; supports parallel builds with a thread-based tasking + model, managed by the job/task subsystem. + + + + + + + + + + + + + + + + Instead of performing an outer-loop recursive descent + of the dependency tree and then forking a task when it finds a + file that needs updating, &SCons; starts as many threads as are + requested, each thread managed by the &Jobs; class. + As a performance optimization, + the &Jobs; class maintains an internal + distinction between + &Serial; and &Parallel; + build jobs, + so that serial builds + don't pay any performance penalty + by using a multi-threaded implementation + written for &Parallel; builds. + + + + + + Each &Jobs; object, running in its own thread, + then requests a &Task; from a central &Taskmaster;, + which is responsible + for handing out available &Task; objects for (re-)building + out-of-date nodes. A condition variable + makes sure that the &Jobs; objects + query the &Taskmaster; one at a time. + + + + + + The &Taskmaster uses the node subsystem's + &Walker; class to walk the dependency tree, + and the &Sig; class to use the + appropriate method + of deciding if a &Node; is up-to-date. + + + + + + This scheme has many advantages over the standard &Make; + implementation of . + Effective use of is difficult + with the usual recursive use of Make, + because the number of jobs started by multiply + at each level of the source tree. + This makes the actual number of jobs + executed at any moment very dependent on the size and layout of + the tree. &SCons;, in contrast, starts only as many jobs as are + requested, and keeps them constantly busy (excepting jobs that + block waiting for their dependency files to finish building). + + + +
+ +
+ +
+ The &SCons; API + + + + This section provides an overview of the &SCons; interface. The + complete interface specification is both more detailed and flexible + than this overview. + + + +
+ &ConsVars; + + + + In &SCons;, a &consenv; is an object through which an external + interface (such as the &scons; script) communicates dependency + information to the &SCons; Build Engine. + + + + + + A construction environment is implemented as a dictionary + containing: + + + + + + + + + construction variables, string values that are substituted + into command lines or used by builder functions; + + + + + + + + one or more &Builder; objects that can be invoked to update a + file or other object; + + + + + + + + one or more &Scanner; objects that can be used to + scan a file automatically for dependencies (such as + files specified on #include lines). + + + + + + + + + &Consenvs; are instantiated as follows: + + + + + env = Environment() + env_debug = Environment(CCFLAGS = '-g') + + +
+ +
+ &Builder; Objects + + + + An &SCons; &Builder; object encapsulates information about how to + build a specific type of file: an executable program, an object + file, a library, etc. A &Builder; object is associated with a + file through an associated &consenv; method and later invoked to + actually build the file. The &Builder; object will typically use + construction variables (such as &CCFLAGS;, &LIBPATH;) to influence + the specific build execution. + + + + + + &Builder; objects are instantiated as follows: + + + + + bld = Builder(name = 'Program', action = "$CC -o $TARGET $SOURCES") + + + + + In the above example, the action is a + command-line string in which the Build Engine will + interpolate the values of construction + variables before execution. The actual + action specified, though, + may be a function: + + + + + def update(dest): + # [code to update the object] + return 0 + + bld = Builder(name = 'Program', function = update) + + + + + Or a callable Python object (or class): + + + + + class class_a: + def __call__(self, kw): + # build the desired object + return 0 + + builder = SCons.Builder.Builder(action = class_a()) + + + + + A &Builder; object may have the prefix and + suffix of its target file type specified + as keyword arguments at instantiation. Additionally, the + suffix of the source files used by this + &Builder; to build its target files may be specified using the + src_suffix keyword argument: + + + + + bld_lib = Builder(name = 'Library', action = "$AR r $TARGET $SOURCES", + prefix = 'lib', suffix = '.a', src_suffix = '.o') + + + + + The specified prefix and + suffix will be appended to the name of any + target file built by this &Builder; object, if they are not + already part of the file name. The src_suffix + is used by the &SCons; Build Engine to chain together + multiple &Builder; objects to create, + for example, a library from the original source + files without having to specify the + intermediate .o files. + + + + + + &Builder; objects are associated with a &consenv; through a + &consvar; named &BUILDERS;, a list of the &Builder objects that + will be available for execution through the &consenv: + + + + + env = Environment(BUILDERS = [ Object, Library, WebPage, Program ]) + + +
+ +
+ &Scanner; Objects + + + + &Scanner; objects perform automatic checking for dependencies + by scanning the contents of files. The canonical + example is scanning a C source file or header file for + files specified on #include lines. + + + + + + A &Scanner; object is instantiated as follows: + + + + + def c_scan(contents): + # scan contents of file + return # list of files found + + c_scanner = Scanner(name = 'CScan', function = c_scan, + argument = None, + skeys = ['.c', '.C', '.h', '.H') + + + + + The skeys argument specifies a list of file + suffixes for file types that this &Scanner; knows how to scan. + + + + + + &Scanner; objects are associated with a &consenv; through a + &consvar; named &SCANNERS;, a list of the &Scanner; objects that + will be available through the &consenv: + + + + + env = Environment(SCANNERS = [ CScan, M4Scan ]) + + + + + For utilities that will build files with a variety of file + suffixes, or which require unusual scanning rules, a &Scanner; + object may be associated explicitly with a &Builder; object as + follows: + + + + + def tool_scan(contents): + # scan contents of file + return # list of files found + + tool_scanner = Scanner(name = 'TScan', function = tool_scan) + + bld = Builder(name = 'Tool', scanner = tool_scanner) + + +
+ +
+ &BuildDir; + + + + &SCons; supports a flexible mechanism for building target + files in a separate build directory from the source files. + The &BuildDir; syntax is straightforward: + + + + + BuildDir(source = 'src', build = 'bld') + + + + + By + default, source files are linked or copied into the build + directory, because exactly replicating the source directory + is sometimes necessary for certain combinations of use of + #include "..." and search + paths. + + An option exists to specify that only output files should be placed in + the build directory: + + + + + BuildDir(source = 'src', build = 'bld', no_sources = 1) + + +
+ +
+ &Repository; + + + + &SCons; supports the ability to search a list of code repositories + for source files and derived files. This works much like + &Make;'s VPATH feature, as implemented in + recent versions of GNU &Make;. + (The POSIX standard for &Make; specifies slightly + different behavior for VPATH.) + The syntax is: + + + + + Repository('/home/source/1.1', '/home/source/1.0') + + + + + A command-line option exists to allow + repositories to be specified on the command line, or in the + &SCONSFLAGS; environment variable (not construction variable!). + This avoids a chicken-and-egg situation and allows the top-level + &SConstruct; file to be found in a repository as well. + + + +
+ +
+ &Cache; + + + + &SCons; supports a way for developers to share derived files. Again, the + syntax is straightforward: + + + + + Cache('/var/build.cache/i386') + + + + + Copies of any derived files built will be placed in the specified + directory with their MD5 signature. If another build results in an + out-of-date derived file with the same signature, the derived file + will be copied from the cache instead of being rebuilt. + + + +
+ +
+ +
+ The &scons; Script + + + + The &scons; script provides an interface + that looks roughly equivalent to the + classic &Make; utility—that is, execution from the command + line, and dependency information read from configuration files. + + + + + + The most noticeable difference between &scons; and &Make;, or most + other build tools, is that the configuration files are actually + Python scripts, generically called "SConscripts" (although the + top-level "Makefile" is named &SConstruct). Users do not have to + learn a new language syntax, but instead configure dependency + information by making direct calls to the Python API of the + &SCons; Build Engine. Here is an example &SConstruct file which + builds a program in side-by-side normal and debug versions: + + + + + env = Environment() + debug = env.Copy(CCFLAGS = '-g') + + source_files = ['f1.c', 'f2.c', 'f3.c'] + + env.Program(target = 'foo', sources = source_files) + debug.Program(target = 'foo-debug', sources = source_files) + + + + + Notice the fact that this file is a Python script, which allows us + to define and re-use an array that lists the source files. + + + + + + Because quoting individul strings in long + lists of files can get tedious and error-prone, the &SCons; + methods support a short-cut of listing multiple files in a single + string, separated by white space. + This would change + the assignment in the above example to a more easily-readable: + + + + + source_files = 'f1.c f2.c f3.c' + + + + + The mechanism to establish hierarchical builds is to "include" any + subsidiary configuration files in the build by listing them explicitly + in a call to the &SConscript; function: + + + + + SConscript('src/SConscript', 'lib/SConscript') + + + + + By convention, configuration files in subdirectories are named + &SConscript;. + + + + + + The &scons; script has intentionally been made to look, from + the outside, as much like &Make; as is practical. To this + end, the &scons; script supports all of the same command-line + options supported by GNU &Make;: FILE, + , , , + etc. For compatibility, &scons; ignores those GNU &Make; options + that don't make sense for the &SCons; architecture, such as + , , , + and . The + intention is that, given an equivalent &SConstruct; file for a + &Makefile;, a user could use &SCons; as a drop-in replacement for + &Make;. Additional command-line options are, where possible, taken + from the Perl &Cons; utility on which the &SCons; design is based. + + + +
-- cgit v1.2.3