Synopsis 0.5 - User Manual

The powerful way: new project file format

The Project file provides a more powerful, more integrated method of driving Synopsis than a Makefile. However, it is harder to work with. The intended usage is from a GUI, however a GUI is still forthcoming (see the next section).

The project file is still a Python script, but does away with the methods and nested classes, instead being one class called Project which has a number of attributes. Some of these attributes are quite complex, but are stored in a readable 'repr' type format. Configuration for modules is stored as an object instance rather than as a class, and is represented in the file as an object instantiation, like so:

Example 2.1. How object instantiation works

# The 'struct' class is included at the top of the config file. It
# basically stores attributes given to the constructor
class struct:
  def __init__(self, **args):
    for key, value in args.items:
      setattr(self, key, value)

class Project:
  # Instantiate an object:
  my_object = struct(name = "foo", magic_number = 42)
  
  # And another one, this time formatted nicely:
  some_object = struct(
    name = "hi", 
    some_list = [1, 2, 3],
    # Here an attribute is a nested object:
    nested_object = struct(
      type = "foo",
      fred = "baz"
    )
  )

The new Project structure defines not only configuration for the individual modules, but the relationships between different stages of the document generation process. The process is split into "Actions", each of which generally has inputs and outputs, with some exceptions:

SourceAction

Selects the source files to use in the project. It has no input, and its output is just the names of the individual files. The SourceAction has a number of rules which it follows in turn to decide which files to use. The rules are set by the configuration, and can select simple file names, use recursive searches and glob expressions, or exclude previously selected files. The ability to exclude certain files is useful for avoiding the appearance of temporary, backup, or cvs-related files in the documentation. See the next section for details on the available rules.

ParserAction

Takes a list of files from one or more SourceActions at input, and applies a Parser to each one in turn. The output is a number of AST's, one for each input file. The ParserAction contains a config object with options for the parser.

CacherAction

Can take as input any number of ASTs (from eg: ParserAction, LinkerAction). Outputs the same ASTs, but stores the ASTs on disk so that they don't need to be regenerated each time if the timestamps have not changed. For C++ files all dependencies (#included files) will have their timestamps checked. For other languages only the actual source files will be checked.

LinkerAction

Can take one or more ASTs as input, and outputs a single AST. The name comes from the process of linking a number of ASTs, typically from different source files, into a single cohesive AST. The Linker is more powerful than that however, allowing complex manipulations of the AST. These manipulations can be based on configuration only, or rely on comments extracted from the source files. The LinkerAction contains a config object to contain all the linker options available.

FormatAction

The FormatAction is the tail end of the chain, and takes a single AST which it converts into some output format. A number of output formats are supported, with varying degrees of completeness and usefulness. The HTML formatter is the most developed, having an immense range of configuration options.

The configuration options for the config objects in the Parser, Linker and Formatter objects are described later in this manual, and are the same as the old config format just with a different syntax.

Project class

class Project:
    # attributes go here. Must be indented!
    name = 'project name'       # string - name of project
    verbose = 0                 # boolean
    data_dir = './'             # string - path (currently unused)
    default_formatter = 'name'  # string - the name of the default formatter to
                                # use if none is specified on the command line.
    channels = [ <list of channels> ]
    actions = [ <list of actions> ]

The main element of the config file is the "class Project". It has a number of attributes, including a string name, verbose flag, a list of actions, a list of channels, and a default formatter to use if none is specified. The channels are the connections between actions, and are simply tuple pairs of names of actions, e.g.:

    channels = [
        ('C++ Parser', 'File Cacher'),
        ('File Cacher', 'Linker')
    ]

Be careful that the action names in the channels list match the names of the actual Actions in the actions list.

Action object

An action object, part of the list of actions in Project, is a list of attributes (not a struct!). The first 4 attributes are always the same, but different Actions have different uses for the remaining attributes (remember, a list has variable length). The following example shows all action types:

    actions = [
        # string type, X/Y coords, string name, extra attributes...
        ['SourceAction', 100, 100, 'Boost Sources',  [ <list of SourceAction Rules> ] ],
        ['ParserAction', 200, 100, 'C++ Parser',     struct(key=val, key=val, ...) ],
        ['CacherAction', 300, 100, 'File Cacher',    'directory' or None, 'file' or None ],
        ['LinkerAction', 400, 100, 'Linker',         struct(key=val, key=val, ...) ],
        ['FormatAction', 500, 100, 'HTML Formatter', struct(key=val, key=val, ...) ]
    ]

As you can see, all types use the same format for the first four attributes. The first is the type of action. The 2nd and 3rd are the X and Y coordinates to use to display the action in the GUI. The 4th is the name of the Action.

For SourceActions the 5th attribute is a list of SourceAction Rules (see the next section).

For CacherActions the 5th and 6th attributes are directory name and file name respectively. One and only one of these must be set (the other must be None). If directory is set, then the inputs to the action will be cached in the given directory and the outputs will be the same as the inputs. If the file is set, then the action will have no inputs, and one output which is loaded from the given file. Note that this second mode of operation is not really caching but more like loading, and can be used to load an AST from a file which you don't have or want to use source code for.

For all other Action types the 5th attribute is a Config object. For information on these see the documentation for the relevant chapters of this manual (Parsers, Linker and Formatters).

SourceAction Rules

The rules used to select files in the SourceAction are stored in a list, where each rule is a tuple. The first element of the tuple is a string denoting the type of rule, and the rest depends on the type. Available types of SourceAction rules are:

Simple

A 'simple' rule that contains a list of filenames as the second tuple element. The filenames may be absolute or relative. For backwards compatibility, a single filename may be used instead of a list.

('Simple', ['fred.py', 'baz.py'])
          
Glob

Allows easy selection of a number of files in a project or subdirectory, without having to update the config when extra files are added or removed. There are three extra tuple elements: a list of directories, a glob string, and an integer recursion flag. The directory names are just strings, and may be absolute or relative. The glob string is a glob expression as used by unix shells: the special characters are '*' which matches to 0 or more characters, and ? which matches any one character. The recursion flag can be 0 or 1. If 1, then subdirectories of the listed directories are also searched.

('Glob', ['src/', 'tools/'], '*.cc', 1)
          
Exclude

Allows files matched by previous rules to be excluded from the list. Remember that the rules are processed in order, so later rules can add more files again. There is only one extra field which is a glob expression to match the whole pathname. In order to make matching subdirectories easier, the filenames are always considered to have a '/' prepended before comparing to the glob expression. This means that if a CVS directory is in the current directory and its files' relative pathnames resemble 'CVS/Root', you can still match it with '/CVS/'.

('Exclude', '/CVS/')
          
Dir

Backwards compatibility for an old version of non-recursive Glob: Has a single directory and a glob expression.

('Dir', 'src/', '*.cc')
          
Base

Backwards compatibility for an old version of recursive Glob: Has a single directory and a glob expression.

('Base', 'src/', '*.cc')
          

All rules are processed in the order that they appear in the list. The effect of a rule is to add remove files to/from a list. Therefore to explicitly exclude a file, the exclude rule should be at the end.

Note that this is the opposite to permission type rules you might be used to, since in this case we don't know what files there are until a rule is processed. With permissions you know what user/file you are checking permissions for so it's usually the first matching rule which is used - hence exclusions are usually placed first for permissions.

Another special case is when you are using the 'multiple_files' feature of the C++ parser - in this case the first rule must be a Simple rule with the main filename (e.g.: Python.h). This is documented at the end of the C++ Parser section.

Command Line Usage

To use the project file and generate output for the default formatter (specified in the config), use a command line like so:

$ synopsis -P project.synopsis

You can specify a formatter to use other than the default formatter like so:

$ synopsis -P project.synopsis -Wc,formatter=MyFormatter

Synopsis will ensure that all Actions connected to the formatter are also executed.

See the Boost demo for an example of how to use the project file format.