This manual documents Application Development with Semantic.

Copyright © 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2007 Eric M. Ludlam Copyright © 2001, 2002, 2003, 2004 David Ponce Copyright © 2002, 2003 Richard Y. Kim

     Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sections being list their titles, with the Front-Cover Texts being list, and with the Back-Cover Texts being list.  A copy of the license is included in the section entitled ``GNU Free Documentation License''.

This file documents Application Development with Semantic. Infrastructure for parser based text analysis in Emacs

Copyright © 1999, 2000, 2001, 2002, 2003, 2004 Eric M. Ludlam, David Ponce, and Richard Y. Kim

     [file semanticheader.texi not included]

Semantic Application Development Manual
***************************************

A semantic application takes the semantic tags generated by semantic parsers then provides useful services to the user. For a list of such applications, see the Semantic User's Guide (semantic-user).

An application developer needs to know

  * when to invoke the parser to generate or regenerate the tag lists.

  * how to access the tag lists.

  * how to access information about each tag.

This chapter describes semantic fuctions and concepts an application developer needs to know to perform all of the tasks just listed.

Semantic Tags
*************

     [file tags.texi not included]

Searching Tag Tables
********************

These functions take some key, and returns information found in a tag table.  Some will return one tag (the first matching item found.) Others will return a list of all items matching a given criterion. Most of these functions work regardless of a buffer being in memory or not.

Any specialty search routine that claims to use a function that is an overload method will need to execute in a buffer of the same mode as the tags being searched.

Breadth Search
==============

Searching the breadth of a list of tags means that only one level of the tags will be searched.  If one of the tags is a datatype with additional members, those members are not searched.

 -- Function: semantic-find-first-tag-by-name name &optional table
     Find the first tag with NAME in TABLE. NAME is a string. TABLE is a semantic tags table.  See semantic-something-to-tag-table. This routine uses assoc to quickly find the first matching entry.

 -- Function: semantic-find-tags-by-name name &optional table
     Find all tags with NAME in TABLE. NAME is a string. TABLE is a tag table.  See semantic-something-to-tag-table.

 -- Function: semantic-find-tags-for-completion prefix &optional table
     Find all tags whos name begins with PREFIX in TABLE. PREFIX is a string. TABLE is a tag table.  See semantic-something-to-tag-table. While it would be nice to use try-completion or all-completions, those functions do not return the tags, only a string. Uses compare-strings for fast comparison.

 -- Function: semantic-find-tags-by-name-regexp regexp &optional table
     Find all tags with name matching REGEXP in TABLE. REGEXP is a string containing a regular expression, TABLE is a tag table.  See semantic-something-to-tag-table. Consider using semantic-find-tags-for-completion if you are attempting to do completions.

 -- Function: semantic-find-tags-by-class class &optional table
     Find all tags of class CLASS in TABLE. CLASS is a symbol representing the class of the token, such as 'variable, of 'function.. TABLE is a tag table.  See semantic-something-to-tag-table.

 -- Function: semantic-find-tags-by-type type &optional table
     Find all tags of with a type TYPE in TABLE. TYPE is a string or tag representing a data type as defined in the language the tags were parsed from, such as ``int'', or perhaps a tag whose name is that of a struct or class. TABLE is a tag table.  See semantic-something-to-tag-table.

 -- Function: semantic-find-tags-included &optional table
     Find all tags in TABLE that are of the 'include class. TABLE is a tag table.  See semantic-something-to-tag-table.

Deep Search
===========

 -- Function: semantic-brute-find-first-tag-by-name name streamorbuffer &optional search-parts search-include
     Find a tag NAME within STREAMORBUFFER.  NAME is a string. If SEARCH-PARTS is non-nil, search children of tags. If SEARCH-INCLUDE is non-nil, search include files.

     Use semantic-find-first-tag-by-name instead.

     semantic-find-nonterminal-by-name,semantic-brute-find-first-tag-by-name

 -- Function: semantic-brute-find-tag-by-property property value streamorbuffer &optional search-parts search-includes
     Find all tags with PROPERTY equal to VALUE in STREAMORBUFFER. Optional argument SEARCH-PARTS and SEARCH-INCLUDES are passed to semantic-brute-find-tag-by-function.

     semantic-find-nonterminal-by-property,semantic-brute-find-tag-by-property

 -- Function: semantic-brute-find-tag-by-attribute attr streamorbuffer &optional search-parts search-includes
     Find all tags with a given ATTR in STREAMORBUFFER. ATTR is a symbol key into the attributes list. Optional argument SEARCH-PARTS and SEARCH-INCLUDES are passed to semantic-brute-find-tag-by-function.

     semantic-find-nonterminal-by-extra-spec,semantic-brute-find-tag-by-attribute

 -- Function: semantic-brute-find-tag-by-attribute-value attr value streamorbuffer &optional search-parts search-includes
     Find all tags with a given ATTR equal to VALUE in STREAMORBUFFER. ATTR is a symbol key into the attributes list. VALUE is the value that ATTR should match. Optional argument SEARCH-PARTS and SEARCH-INCLUDES are passed to semantic-brute-find-tag-by-function.

     semantic-find-nonterminal-by-extra-spec-value,semantic-brute-find-tag-by-attribute-value

 -- Function: semantic-brute-find-tag-by-position position streamorbuffer &optional nomedian
     Find a nonterminal covering POSITION within STREAMORBUFFER. POSITION is a number, or marker.  If NOMEDIAN is non-nil, don't do the median calculation, and return nil.

     semantic-find-nonterminal-by-position,semantic-brute-find-tag-by-position

 -- Function: semantic-brute-find-innermost-tag-by-position position streamorbuffer &optional nomedian
     Find a list of tags covering POSITION within STREAMORBUFFER. POSITION is a number, or marker.  If NOMEDIAN is non-nil, don't do the median calculation, and return nil. This function will find the topmost item, and recurse until no more details are available of findable.

     semantic-find-innermost-nonterminal-by-position,semantic-brute-find-innermost-tag-by-position

 -- Function: semantic-brute-find-tag-by-class class streamorbuffer &optional search-parts search-includes
     Find all tags with a class CLASS within STREAMORBUFFER. CLASS is a symbol representing the class of the tags to find. See semantic-tag-class. Optional argument SEARCH-PARTS and SEARCH-INCLUDES are passed to semantic-brute-find-tag-by-function.

     Use semantic-find-tag-by-class instead.

     semantic-find-nonterminal-by-token,semantic-brute-find-tag-by-class

 -- Function: semantic-brute-find-tag-standard streamorbuffer &optional search-parts search-includes
     Find all tags in STREAMORBUFFER which define simple class types. See semantic-tag-class. Optional argument SEARCH-PARTS and SEARCH-INCLUDES are passed to semantic-brute-find-tag-by-function.

     semantic-find-nonterminal-standard,semantic-brute-find-tag-standard

 -- Function: semantic-brute-find-tag-by-type type streamorbuffer &optional search-parts search-includes
     Find all tags with type TYPE within STREAMORBUFFER. TYPE is a string which is the name of the type of the tags returned. See semantic-tag-type. Optional argument SEARCH-PARTS and SEARCH-INCLUDES are passed to semantic-brute-find-tag-by-function.

     semantic-find-nonterminal-by-type,semantic-brute-find-tag-by-type

 -- Function: semantic-brute-find-tag-by-function function streamorbuffer &optional search-parts search-includes
     Find all tags for which FUNCTION's value is non-nil within STREAMORBUFFER. FUNCTION must return non-nil if an element of STREAM will be included in the new list.

     If optional argument SEARCH-PARTS is non-nil, all sub-parts of tags are searched.  The overloadable function semantic-tag-componenets is used for the searching child lists.  If SEARCH-PARTS is the symbol 'positiononly, then only children that have positional information are searched.

     If SEARCH-INCLUDES is non-nil, then all include files are also searched for matches.  This parameter hasn't be active for a while and is obsolete.

     semantic-find-nonterminal-by-function,semantic-brute-find-tag-by-function

 -- Function: semantic-brute-find-first-tag-by-function function streamorbuffer &optional search-parts search-includes
     Find the first nonterminal which FUNCTION match within STREAMORBUFFER. FUNCTION must return non-nil if an element of STREAM will be included in the new list.

     The following parameters were never implemented.

     If optional argument SEARCH-PARTS, all sub-parts of tags are searched. The overloadable function semantic-tag-components is used for searching. If SEARCH-INCLUDES is non-nil, then all include files are also searched for matches.

     semantic-find-nonterminal-by-function-first-match,semantic-brute-find-first-tag-by-function

Specialty Search
================

There are some specialty searches needed by some semantic tools that could prove useful.  These specialty searches often do not match against some single attribute as most breadth searches do.

 -- Function: semantic-find-tags-of-compound-type &optional table
     Find all tags which are a compound type in TABLE. Compound types are structures, or other data type which is not of a primitive nature, such as int or double. Used in completion.

 -- Function: semantic-find-tags-by-scope-protection scopeprotection parent &optional table
     Find all tags accessable by SCOPEPROTECTION. SCOPEPROTECTION is a symbol which can be returned by the method semantic-tag-protection.  A hard-coded order is used to determine a match. PARENT is a tag representing the PARENT slot needed for semantic-tag-protection. TABLE is a list of tags (a subset of PARENT members) to scan.  If TABLE is nil, the type members of PARENT are used. See semantic-tag-protected-p for details on which tags are returned.

 -- Function: semantic-find-tags-external-children-of-type type &optional table
     Find all tags in whose parent is TYPE in TABLE. These tags are defined outside the scope of the original TYPE declaration. TABLE is a tag table.  See semantic-something-to-tag-table.

Custom Search
=============

The searching framework for semantic for tag tables has two basic root methods.  One is a function and the other is a macro.  The functional version is needed if some sort of macro conflict arrises. The macro version is useful because it eliminates a level of function call, and is faster.

 -- Function: semantic--find-tags-by-function predicate &optional table
     Find tags for which PREDICATE is non-nil in TABLE. PREDICATE is a lambda expression which accepts on TAG. TABLE is a semantic tags table.  See semantic-something-to-tag-table.

 -- Function: semantic--find-tags-by-macro form &optional table
     Find tags for which FORM is non-nil in TABLE. TABLE is a semantic tags table.  See semantic-something-to-tag-table.

Tags at Point
*************

When you need to get the tag the cursor is on, there is a more efficient mechanism than using semantic-brute-find-tag-by-position.  This mechanism directly queries the overlays the parsing step leaves in the buffer.  This provides for very rapid retrieval of what function or variable the cursor is currently in.

These functions query the current buffer's overlay system for tags.

 -- Function: semantic-find-tag-by-overlay &optional positionormarker buffer
     Find all tags covering POSITIONORMARKER by using overlays. If POSITIONORMARKER is nil, use the current point. Optional BUFFER is used if POSITIONORMARKER is a number, otherwise the current buffer is used.  This finds all tags covering the specified position by checking for all overlays covering the current spot.  They are then sorted from largest to smallest via the start location.

     semantic-find-nonterminal-by-overlay,semantic-find-tag-by-overlay

 -- Function: semantic-find-tag-by-overlay-in-region start end &optional buffer
     Find all tags which exist in whole or in part between START and END. Uses overlays to determine positin. Optional BUFFER argument specifies the buffer to use.

     semantic-find-nonterminal-by-overlay-in-region,semantic-find-tag-by-overlay-in-region

 -- Function: semantic-find-tag-by-overlay-next &optional start buffer
     Find the next tag after START in BUFFER. If START is in an overlay, find the tag which starts next, not the current tag.

 -- Function: semantic-find-tag-by-overlay-prev &optional start buffer
     Find the next tag before START in BUFFER. If START is in an overlay, find the tag which starts next, not the current tag.

 -- Function: semantic-current-tag
     Return the current tag in the current buffer. If there are more than one in the same location, return the smallest tag.  Return nil if there is no tag here.

     semantic-current-nonterminal,semantic-current-tag

 -- Function: semantic-current-tag-parent
     Return the current tags parent in the current buffer. A tag's parent would be a containing structure, such as a type containing a field.  Return nil if there is no parent.

     semantic-current-nonterminal-parent,semantic-current-tag-parent

Tag Decoration
**************

Tags can be decorated in different ways.  One way a user can control it is through semantic-decoration-mode. See semantic-user (Tag Decoration Mode)

Applications can use the same routines to decorate tags as well.

Highlighting
============

 -- Function: semantic-highlight-tag tag &optional face
     Specify that TAG should be highlighted. Optional FACE specifies the face to use. semantic-highlight-token,semantic-highlight-tag

 -- Function: semantic-unhighlight-tag tag
     Unhighlight TAG, restoring it's previous face. semantic-unhighlight-token,semantic-unhighlight-tag

 -- Function: semantic-momentary-highlight-tag tag &optional face
     Highlight TAG, removing highlighting when the user hits a key. Optional argument FACE is the face to use for highlighting. If FACE is not specified, then highlight will be used. semantic-momentary-highlight-token,semantic-momentary-highlight-tag

The highlighting routines do their work by changing the face property of the tag overlay.  The raw routine is:

 -- Function: semantic-set-tag-face tag face
     Specify that TAG should use FACE for display. semantic-set-token-face,semantic-set-tag-face

Changing a tag's visible properties
===================================

You can give a tag other properties as well, such as making it invisible or intangible.  You can control how code is edited programatically through these routines.

 -- Function: semantic-set-tag-invisible tag &optional visible
     Enable the text in TAG to be made invisible. If VISIBLE is non-nil, make the text visible. semantic-set-token-invisible,semantic-set-tag-invisible

 -- Function: semantic-tag-invisible-p tag
     Return non-nil if TAG is invisible. semantic-token-invisible-p,semantic-tag-invisible-p

 -- Function: semantic-set-tag-intangible tag &optional tangible
     Enable the text in TAG to be made intangible. If TANGIBLE is non-nil, make the text visible. This function does not have meaning in XEmacs because it seems that the extent 'intangible' property does not exist. semantic-set-token-intangible,semantic-set-tag-intangible

 -- Function: semantic-tag-intangible-p tag
     Return non-nil if TAG is intangible. This function does not have meaning in XEmacs because it seems that the extent 'intangible' property does not exist. semantic-token-intangible-p,semantic-tag-intangible-p

 -- Function: semantic-set-tag-read-only tag &optional writable
     Enable the text in TAG to be made read-only. Optional argument WRITABLE should be non-nil to make the text writable instead of read-only. semantic-set-token-read-only,semantic-set-tag-read-only

 -- Function: semantic-tag-read-only-p tag
     Return non-nil if the current TAG is marked read only. semantic-token-read-only-p,semantic-tag-read-only-p

Secondary Overlays
==================

Each tag which is being visited in a buffer has one overlay.  This overlay is used to track where the tag is while a user is editing, and can also be used for fast tag identification, and some simple decoration techniques.

A secondary overlay associates a new overlay object with a tag which does not cover the entire body of a tag.  You can then put visible features on that overlay to decorate portions of a tag.  This is how tag boundaries are drawn.

Creation and Deletion
---------------------

 -- Function: semantic-tag-create-secondary-overlay tag &optional link-hook
     Create a secondary overlay for TAG. Returns an overlay.  The overlay is also saved in TAG. LINK-HOOK is a function called whenever TAG is to be linked into a buffer.  It should take TAG and OVERLAY as arguments. The LINK-HOOK should be used to position and set properties on the generated secondary overlay.

 -- Function: semantic-tag-delete-secondary-overlay tag overlay-or-property
     Delete from TAG the secondary overlay OVERLAY-OR-PROPERTY. If OVERLAY-OR-PROPERTY is an overlay, delete that overlay. If OVERLAY-OR-PROPERTY is a symbol, find the overlay with that property.

Queries
-------

 -- Function: semantic-tag-get-secondary-overlay tag property
     Return secondary overlays from TAG with PROPERTY. PROPERTY is a symbol and all overlays with that symbol are returned..

 -- Function: semantic-tag-secondary-overlays tag
     Return a list of secondary overlays active on TAG.

Linking and Unlinking
---------------------

When a tag's file is put in a buffer, that tag is ``linked'' into the buffer.  When the buffer is deleted, the tag is ``unlinked''.  This process adds and removes the default overlay on the tag.  Secondary overlays use semantic-tag-add-hook and semantic-tag-remove-hook too apply link and unlink hooks.

This allows the secondary overlays to be automatically removed or added by the seconary overlay system whenever a tag's file goes in or out of a buffer.

Folding
=======

Using secondary overlays, a set of tag folding routines are made available.  These routines are similar to the tag visible properties.

 -- Function: semantic-set-tag-folded tag &optional folded
     Fold TAG, such that only the first line of text is shown. Optional argument FOLDED should be non-nil to fold the tag. nil implies the tag should be fully shown.

 -- Function: semantic-tag-folded-p tag
     Non-nil if TAG is currently folded.

Tag Sorting
***********

Sometimes it is important to reorganize a tag stream into a form that is better for display to a user.  It is important to not use functions with side effects that could effect the tag cache.

There are some existing utility functions which will reorganize the tag list for you.

 -- Function: semantic-unique-tag-table tags
     Scan a list of TAGS, removing duplicates. This must first sort the tags by position ascending. TAGS are removed only if they are equivalent, as can happen when multiple tag sources are scanned.

 -- Function: semantic-unique-tag-table-by-name tags
     Scan a list of TAGS, removing duplicate names. This must first sort the tags by name alphabetically ascending.

 -- Function: semantic-bucketize tags &optional parent filter
     Sort TAGS into a group of buckets based on tag class. Unknown classes are placed in a Misc bucket. Type bucket names are defined by either semantic-symbol->name-assoc-list. If PARENT is specified, then TAGS belong to this PARENT in some way. This will use semantic-symbol->name-assoc-list-for-type-parts to generate bucket names. Optional argument FILTER is a filter function to be applied to each bucket. The filter function will take one argument, which is a list of tokens, and may re-organize the list with side-effects.

 -- Variable: semantic-bucketize-tag-class
     Function used to get a symbol describing the class of a tag. This function must take one argument of a semantic tag. It should return a symbol found in semantic-symbol->name-assoc-list which semantic-bucketize uses to bin up tokens. To create new bins for an application augment semantic-symbol->name-assoc-list, and semantic-symbol->name-assoc-list-for-type-parts in addition to setting this variable (locally in your function).

 -- Function: semantic-adopt-external-members tags
     Rebuild TAGS so that externally defined members are regrouped. Some languages such as C++ and CLOS permit the declaration of member functions outside the definition of the class.  It is easier to study the structure of a program when such methods are grouped together more logically.

     This function uses semantic-tag-external-member-p to determine when a potential child is an externally defined member.

     Note: Applications which use this function must account for token types which do not have a position, but have children which *do* have positions.

     Applications should use semantic-mark-external-member-function to modify all tags which are found as externally defined to some type.  For example, changing the token type for generating extra buckets with the bucket function.

 -- Variable: semantic-orphaned-member-metaparent-type
     In semantic-adopt-external-members, the type of 'type for metaparents. A metaparent is a made-up type semantic token used to hold the child list of orphaned members of a named type.

 -- Variable: semantic-mark-external-member-function
     Function called when an externally defined orphan is found. By default, the token is always marked with the adopted property. This function should be locally bound by a program that needs to add additional behaviors into the token list. This function is called with two arguments.  The first is TOKEN which is a shallow copy of the token to be modified.  The second is the PARENT which is adopting TOKEN.  This function should return TOKEN (or a copy of it) which is then integrated into the revised token list.

Tag Completion
**************

Often time, it is useful to ask the user for the name of some tag. This can be as simple as just prompting for a name, but often time, the semantics can be quite complex.  If two tags have the same name, which do you choose?

Semantic provides a completion engine for prompting for tags by name, and providing fancy ways to display completion lists that allow the user to choose a tag if several have the same name.

To use a completion function in your interactive function, you can augment the interactive command like this:

     (defun my-function (tag)
        "Do something to TAG."
        (interactive (list (my-completion-function "Tag: ")))
        ...)

Tag Completion Convenience Functions
====================================

There are some pre written completion functions that can be used in your programs.

 -- Function: semantic-complete-read-tag-buffer-deep prompt &optional default-tag initial-input history
     Ask for a tag by name from the current buffer. Available tags are from the current buffer, at any level. Completion options are presented in a traditional way, with highlighting to resolve same-name collisions. PROMPT is a string to prompt with. DEFAULT-TAG is a semantic tag or string to use as the default value. If INITIAL-INPUT is non-nil, insert it in the minibuffer initially. HISTORY is a symbol representing a variable to story the history in.

 -- Function: semantic-complete-read-tag-project prompt &optional default-tag initial-input history
     Ask for a tag by name from the current project. Available tags are from the current project, at the top level. Completion options are presented in a traditional way, with highlighting to resolve same-name collisions. PROMPT is a string to prompt with. DEFAULT-TAG is a semantic tag or string to use as the default value. If INITIAL-INPUT is non-nil, insert it in the minibuffer initially. HISTORY is a symbol representing a variable to store the history in.

 -- Function: semantic-complete-read-tag-analyzer prompt &optional context history
     Ask for a tag by name based on the current context. The function semantic-analyze-current-context is used to calculate the context.  semantic-analyze-possible-completions is used to generate the list of possible completions. PROMPT is the first part of the prompt.  Additional prompt is added based on the contexts full prefix. CONTEXT is the semantic analyzer context to start with. HISTORY is a symbol representing a variable to stor the history in. usually a default-tag and initial-input are available for completion prompts.  these are calculated from the CONTEXT variable passed in.

 -- Function: semantic-complete-inline-analyzer context
     Complete a symbol name by name based on the current context. This is similar to semantic-complete-read-tag-analyze, except that the completion interaction is in the buffer where the context was calculated from. CONTEXT is the semantic analyzer context to start with. See semantic-complete-inline-tag-engine for details on how completion works.

Custom Tag Completion Functions
===============================

There aren't many built in completion functions, but there are many parts that can be put together into custom completion functions.

A completion function is built up of three important parts.

Tag Collection
     Something that selects tags, and provides some list of tags available, such as all functions, or all classes named ``bob''.

Typing and selecting
     The prompt where you can type in the name of a tag.

Displaying possible completion values
     A mechanism for displaying completion lists.

There is one typing and selecting routine that can be used to create your custom completion prompt.

 -- Function: semantic-complete-read-tag-engine collector displayor prompt default-tag initial-input history
     Read a semantic tag, and return a tag for the selection. Argument COLLECTOR is an object which can be used to to calculate a list of possible hits.  See semantic-completion-collector-engine for details on COLLECTOR. Argumeng DISPLAYOR is an object used to display a list of possible completions for a given prefix.  Seesemantic-completion-display-engine for details on DISPLAYOR. PROMPT is a string to prompt with. DEFAULT-TAG is a semantic tag or string to use as the default value. If INITIAL-INPUT is non-nil, insert it in the minibuffer initially. HISTORY is a symbol representing a variable to story the history in.

As you can see, this takes one collector, and one displayor.  These are objects created for this prompt at runtime.  The completion engine then uses to perform their tasks.

For example:

     (defun semantic-complete-read-tag-buffer-deep (prompt &optional
     						      default-tag initial-input history)
       "Ask for a tag by name from the current buffer.
     PROMPT is a string to prompt with.
     DEFAULT-TAG is a semantic tag or string to use as the default value.
     If INITIAL-INPUT is non-nil, insert it in the minibuffer initially.
     HISTORY is a symbol representing a variable to story the history in."
       (semantic-complete-read-tag-engine
        (semantic-collector-buffer-deep prompt :buffer (current-buffer))
        (semantic-displayor-traditional-with-focus-highlight "simple")
        ;;(semantic-displayor-tooltip "simple")
        prompt
        default-tag
        initial-input
        history)
       )

Tag Collectors
--------------

A tag collector is an object that inherits from semantic-collector-abstract.  A new collector is needed for each specialized type of prompt that draws from a different selection of tags.

You can have a collector that satisfies multiple purposes using slots filled in the initializer for that object.

Collectors inherit from some of the following:

 -- Type: semantic-collector-abstract
     Root class for completion engines. The baseclass provides basic functionality for interacting with a completion displayor object, and tracking the current progress of a completion.

 -- Type: semantic-collector-buffer-abstract
     Root class for per-buffer completion engines. These collectors track themselves on a per-buffer basis.

 -- Type: semantic-collector-project-abstract
     Root class for project wide completion engines. Uses semanticdb for searching all tags in the current project.

Available instantiable classes are:

 -- Type: semantic-collector-buffer-deep
     Completion engine for tags in the current buffer. When searching for a tag, uses semantic  deep searche functions. Basics search only in the current buffer.

 -- Type: semantic-collector-project
     Completion engine for tags in a project.

 -- Type: semantic-collector-project-brutish
     Completion engine for tags in a project.

 -- Type: semantic-collector-analyze-completions
     Completion engine that uses the context analyzer to provide options. The only options available for completion are those which can be logically inserted into the current context.

NOTE: Add sections for writing new collectors.

Tag Displayors
--------------

When a user is interacting with the prompt, and requests completion, those tags are drawn from the collector.  If the user asks for a list of completion by hitting a complete key twice, then the list of completions heeds to be displayed in some way.

Displayors can be used to manage the display of all tags currently available, AND often needs to be used to focus one one particular tag of many in a visible way.

All displayors inherit from the displayor baseclass that defines the default behaviors:

 -- Type: semantic-displayor-abstract
     Manages the display of some number of tags. Provides the basics for a displayor, including interacting with a collector, and tracking tables of completion to display.

 -- Type: semantic-displayor-focus-abstract
     A displayor which has the ability to focus in on one tag. Focusing is a way of differentiationg between multiple tags which have the same name.

Distinct implementations of displayors include:

 -- Type: semantic-displayor-traditional
     Traditional display mechanism for a list of possible completions. Completions are showin in a new buffer and listed with the ability to click on the items to aid in completion.

 -- Type: semantic-displayor-traditional-with-focus-highlight
     A traditional displayor which can focus on a tag by showing it. Same as semantic-displayor-traditional, but with selection between multiple tags with the same name done by 'focusing' on the source location of the different tags to differentiate them.

 -- Type: semantic-displayor-tooltip
     Display mechanism using tooltip for a list of possible completions.

NOTE: Add sections for writing new collectors.

Older Tag Completion functions
==============================

These are older completion functions.  They may still be useful.

 -- Function: semantic-read-symbol prompt &optional default stream filter
     Read a symbol name from the user for the current buffer. PROMPT is the prompt to use. Optional arguments: DEFAULT is the default choice.  If no default is given, one is read from under point. STREAM is the list of tokens to complete from. FILTER is provides a filter on the types of things to complete. FILTER must be a function to call on each element.

 -- Function: semantic-read-variable prompt &optional default stream
     Read a variable name from the user for the current buffer. PROMPT is the prompt to use. Optional arguments: DEFAULT is the default choice.  If no default is given, one is read from under point. STREAM is the list of tokens to complete from.

 -- Function: semantic-read-function prompt &optional default stream
     Read a function name from the user for the current buffer. PROMPT is the prompt to use. Optional arguments: DEFAULT is the default choice.  If no default is given, one is read from under point. STREAM is the list of tags to complete from.

 -- Function: semantic-read-type prompt &optional default stream
     Read a type name from the user for the current buffer. PROMPT is the prompt to use. Optional arguments: DEFAULT is the default choice.  If no default is given, one is read from under point. STREAM is the list of tags to complete from.

Override Methods
****************

These functions are called `override methods' because they provide generic behaviors, which a given language can override.  For example, finding a dependency file in Emacs lisp can be done with the `locate-library' command (which overrides the default behavior.)  In C, a dependency can be found by searching a generic search path which can be passed in via a variable.

If you plan to use one of these functions from a buffer that is not of the same major-mode as the original tag, you can use this form to make sure the correct action takes place:

 -- Function: semantic-with-mode-bindings mode &rest body
     Evaluate BODY with the local bindings of MODE. The current mode bindings are saved, BODY is evaluated, and the saved bindings are restored, even in case of an abnormal exit. Value is what BODY returns.

For more on override methods, see Semantic Overload Mechanism (lang-support-guide).

Format Tag
==========

Any given tag consists of Meta information which is best viewed in some textual form.  This could be as simple as the tag's name, or as a prototype to be added to header file in C.  Not only are there several default converters from a Tag into text, but there is also some convenient variables that can be used with them.  Use these variables to allow options on output forms when displaying tags in your programs.

 -- Variable: semantic-format-tag-functions
     List of functions which convert a tag to text. Each function must take the parameters TAG &optional PARENT COLOR. TAG is the tag to convert. PARENT is a parent tag or name which refers to the structure or class which contains TAG.  PARENT is NOT a class which a TAG would claim as a parent. COLOR indicates that the generated text should be colored using font-lock.

 -- Variable: semantic-format-tag-custom-list
     A List used by customizeable variables to choose a tag to text function. Use this variable in the :type field of a customizable variable.

Every tag to text conversion function must take the same parameters, which are TAG, the tag to be converted, PARENT, the containing parent (like a structure which contains a variable), and COLOR, which is a flag specifying that color should be applied to the returned string.

When creating, or using these strings, particularly with color, use concat to build up larger strings instead of format.  This will preserve text properties.

 -- Function: semantic-format-tag-name tag &optional parent color
     Return the name string describing TAG. The name is the shortest possible representation. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-name-nonterminal,semantic-format-tag-name

 -- Function: semantic-format-tag-abbreviate tag &optional parent color
     Return an abbreviated string describing TAG. The abbreviation is to be short, with possible symbols indicating the type of tag, or other information. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-abbreviate-nonterminal,semantic-format-tag-abbreviate

 -- Function: semantic-format-tag-summarize tag &optional parent color
     Summarize TAG in a reasonable way. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-summerize-nonterminal,semantic-format-tag-summarize

 -- Function: semantic-format-tag-prototype tag &optional parent color
     Return a prototype for TAG. This function should be overloaded, though it need not be used. This is because it can be used to create code by language independent tools. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-prototype-nonterminal,semantic-format-tag-prototype

 -- Function: semantic-format-tag-concise-prototype tag &optional parent color
     Return a concise prototype for TAG. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-concise-prototype-nonterminal,semantic-format-tag-concise-prototype

 -- Function: semantic-format-tag-uml-abbreviate tag &optional parent color
     Return a UML style abbreviation for TAG. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-uml-abbreviate-nonterminal,semantic-format-tag-uml-abbreviate

 -- Function: semantic-format-tag-uml-prototype tag &optional parent color
     Return a UML style prototype for TAG. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-uml-prototype-nonterminal,semantic-format-tag-uml-prototype

 -- Function: semantic-format-tag-uml-concise-prototype tag &optional parent color
     Return a UML style concise prototype for TAG. Optional argument PARENT is the parent type if TAG is a detail. Optional argument COLOR means highlight the prototype with font-lock colors. This function can be overloaded (see define-mode-local-override for details). semantic-uml-concise-prototype-nonterminal,semantic-format-tag-uml-concise-prototype

 -- Function: semantic-format-tag-prin1 tag &optional parent color
     Convert TAG to a string that is the print name for TAG. PARENT and COLOR are ignored. semantic-prin1-nonterminal,semantic-format-tag-prin1

An additional utility will return a string for just the data type of a tag.  This function is used in the above routines as well.

 -- Function: semantic-format-tag-type tag color
     Convert the data type of TAG to a string usable in tag formatting. It is presumed that TYPE is a string or semantic tag. This function can be overloaded (see define-mode-local-override for details).

Tag Members
===========

Tags are often in a hierarchical form, meaning that a tag found in the top-level list may itself contain additional members.

The following overridable functions can fetch those tags.

 -- Function: semantic-tag-components tag
     Return a list of components for TAG. A Component is a part of TAG which itself may be a TAG. Examples include the elements of a structure in a tag of class `type, or the list of arguments to a tag of class 'function. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-tag-components-with-overlays tag
     Return the list of top level components belonging to TAG. Children are any sub-tags which contain overlays.

     Default behavior is to get semantic-tag-components in addition to the components of an anonymous types (if applicable.)

     Note for language authors: If a mode defines a language tag that has tags in it with overlays you should still return them with this function. Ignoring this step will prevent several features from working correctly. This function can be overloaded (see define-mode-local-override for details).

Some languages can define parts of a tag outside the actual scope of the parent tag.  You can fetch information about them with these overload functions.

 -- Function: semantic-tag-external-member-p parent tag
     Return non-nil if PARENT is the parent of TAG. TAG is an external member of PARENT when it is somehow tagged as having PARENT as it's parent. PARENT and TAG must both be semantic tags.

     The default behavior, if not overridden with tag-external-member-p is to match :parent attribute in the name of TAG.

     If this function is overridden, use semantic-tag-external-member-children-p-default to also include the default behavior, and merely extend your own. This function can be overloaded (see define-mode-local-override for details). semantic-nonterminal-external-member-p,semantic-tag-external-member-p

 -- Function: semantic-tag-external-member-children tag &optional usedb
     Return the list of children which are not *in* TAG. If optional argument USEDB is non-nil, then also search files in the Semantic Database.  If USEDB is a list of databases, search those databases.

     Children in this case are functions or types which are members of TAG, such as the parts of a type, but which are not defined inside the class.  C++ and CLOS both permit methods of a class to be defined outside the bounds of the class' definition.

     The default behavior, if not overridden with tag-external-member-children is to search using semantic-tag-external-member-p in all top level definitions with a parent of TAG.

     If this function is overridden, use semantic-tag-external-member-children-default to also include the default behavior, and merely extend your own. This function can be overloaded (see define-mode-local-override for details). semantic-nonterminal-external-member-children,semantic-tag-external-member-children

Tag Details
===========

These functions help derive information about tags that may not be obvious for non-traditional languages with their own token types.

 -- Function: semantic-tag-protection tag &optional parent
     Return protection information about TAG with optional PARENT. This function returns on of the following symbols: nil        - No special protection.  Language dependent. 'public    - Anyone can access this TAG. 'private   - Only methods in the local scope can access TAG. 'protected - Like private for outside scopes, like public for child classes. Some languages may choose to provide additional return symbols specific to themselves.  Use of this function should allow for this.

     The default behavior (if not overridden with tag-protection is to return a symbol based on type modifiers. This function can be overloaded (see define-mode-local-override for details). It makes the overload semantic-nonterminal-protection obsolete. semantic-nonterminal-protection,semantic-tag-protection

 -- Function: semantic-tag-protected-p tag protection &optional parent
     Non-nil if TAG is is protected. PROTECTION is a symbol which can be returned by the method semantic-tag-protection. PARENT is the parent data type which contains TAG.

     For these PROTECTIONs, true is returned if TAG is:

     nil
          Always true

     private
          True if nil.

     protected
          True if private or nil.

     public
          True if private, protected, or nil.

 -- Function: semantic-tag-abstract-p tag &optional parent
     Return non nil if TAG is abstract. Optional PARENT is the parent tag of TAG. In UML, abstract methods and classes have special meaning and behavior in how methods are overridden.  In UML, abstract methods are italicized.

     The default behavior (if not overridden with tag-abstract-p is to return true if abstract is in the type modifiers. This function can be overloaded (see define-mode-local-override for details). It makes the overload semantic-nonterminal-abstract obsolete. semantic-nonterminal-abstract,semantic-tag-abstract-p

 -- Function: semantic-tag-leaf-p tag &optional parent
     Return non nil if TAG is leaf. Optional PARENT is the parent tag of TAG. In UML, leaf methods and classes have special meaning and behavior.

     The default behavior (if not overridden with tag-leaf-p is to return true if leaf is in the type modifiers. This function can be overloaded (see define-mode-local-override for details). It makes the overload semantic-nonterminal-leaf obsolete. semantic-tag-leaf,semantic-tag-leaf-p

 -- Function: semantic-tag-static-p tag &optional parent
     Return non nil if TAG is static. Optional PARENT is the parent tag of TAG. In UML, static methods and attributes mean that they are allocated in the parent class, and are not instance specific. UML notation specifies that STATIC entries are underlined. This function can be overloaded (see define-mode-local-override for details). semantic-tag-static,semantic-tag-static-p

 -- Variable: semantic-dependency-include-path
     Defines the include path used when searching for files. This should be a list of directories to search which is specific to the file being included.

     If semantic-dependency-tag-file is overridden for a given language, this path is most likely ignored.

     This function, reguardless of being overriden, caches the located dependency file location in the tag property dependency-file. If you override this function, you do not need to implement your own cache.  Each time the buffer is fully reparsed, the cache will be reset.

     TODO: use ffap.el to locate such items.

 -- Function: semantic-dependency-tag-file &optional tag
     Find the filename represented from TAG. Depends on semantic-dependency-include-path for searching.  Always searches . first, then searches additional paths. This function can be overloaded (see define-mode-local-override for details). It makes the overload semantic-find-dependency obsolete. semantic-find-dependency,semantic-dependency-tag-file

 -- Function: semantic-prototype-file buffer
     Return a file in which prototypes belonging to BUFFER should be placed. Default behavior (if not overridden) looks for a token specifying the prototype file, or the existence of an EDE variable indicating which file prototypes belong in. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-go-to-tag tag &optional parent
     Go to the location of TAG. TAG may be a stripped element, in which case PARENT specifies a parent tag that has position information. Different behaviors are provided depending on the type of tag. For example, dependencies (includes) will seek out the file that is depended on (see semantic-dependency-tag-file. This function can be overloaded (see define-mode-local-override for details). It makes the overload semantic-find-nonterminal obsolete. semantic-find-nonterminal,semantic-go-to-tag

 -- Function: semantic-texi-find-documentation name &optional type
     Find the function or variable NAME of TYPE in the texinfo source. NAME is a string representing some functional symbol. TYPE is a string, such as ``variable`` or ''Command'' used to find the correct definition in case NAME qualifies as several things. When this function exists, POINT is at the definition. If the doc was not found, an error is thrown. Note: TYPE not yet implemented.

Making New Methods
==================

To create your own application specific mode local function, you should use define-overload.  This function creates the framework needed for different mode-authors to customize your applications.

Once a function has been defined as mode-local, you should use define-mode-local-override to add a mode specific feature.

 -- Function: define-overload name args docstring &rest body
     Define a new function, as with defun which can be overloaded. NAME is the name of the function to create. ARGS are the arguments to the function. DOCSTRING is a documentation string to describe the function.  The docstring will automatically had details about its overload symbol appended to the end. BODY is code that would be run when there is no override defined.  The default is to call the function NAME-default with the appropriate arguments.

     BODY can also include an override form that specifies which part of BODY is specifically overridden.  This permits to specify common code run for both default and overridden implementations. An override form is one of:

     1. (:override [OVERBODY]) 2. (:override-with-args OVERARGS [OVERBODY])

     OVERBODY is the code that would be run when there is no override defined.  The default is to call the function NAME-default with the appropriate arguments deduced from ARGS. OVERARGS is a list of arguments passed to the override and NAME-default function, in place of those deduced from ARGS.

 -- Function: define-mode-local-override name mode args docstring &rest body
     Define a mode specific override of the function overload NAME. Has meaning only if NAME has been created with define-overload. MODE is the major mode this override is being defined for. ARGS are the function arguments, which should match those of the same named function created with define-overload. DOCSTRING is the documentation string. BODY is the implementation of this function. define-mode-overload-implementation,define-mode-local-override

Parser Features
***************

If you write a program that uses a tag table in a persistent display or database, it is necessary to know when tag tables change so that your displays can be updated.  This is especially important as tags can be replaced, changed, or deleted, and the associated overlays will then throw errors when you try to use them.  Complete integration with tag changes can be achieved via several hooks.

If you write an application that frequenly accesses the tags tables, it is important to make sure those tags are up to date, and to make sure you application does not adversely effect all the other minor modes that may be running.

The semantic parser has many features and hooks that applications can use to provide a good user experience.

Editing Buffers
===============

One interesting way to interact with the parser is to let it know that changes you are going to make will not require re-parsing.

 -- Variable: semantic-edits-are-safe
     When non-nil, modifications do not require a reparse. This prevents tokens from being marked dirty, and it prevents top level edits from causing a cache check. Use this when writing programs that could cause a full reparse, but will not change the tag structure, such as adding or updating top-level comments.

Parser State
============

It is sometimes useful to know what the current parsing state is.  These function can let you know what level of re-parsing may be needed.  Careful choices on when to reparse can make your program much faster.

 -- Function: semantic-parse-tree-needs-update-p
     Return non-nil if the current parse tree needs to be updated.

 -- Function: semantic-parse-tree-needs-rebuild-p
     Return non-nil if the current parse tree needs to be rebuilt.

 -- Function: semantic-parse-tree-unparseable-p
     Return non-nil if the current buffer has been marked unparseable.

 -- Function: semantic-parse-tree-up-to-date-p
     Return non-nil if the current parse tree is up to date.

Parser Hooks
============

If you just want to know when a buffer changes, use this hook.

 -- Variable: semantic-after-toplevel-cache-change-hook
     Hooks run after the buffer tag list has changed. This list will change when a buffer is reparsed, or when the tag list in a buffer is cleared.  It is *NOT* called if the current tag list is partially reparsed.

     Hook functions must take one argument, which is the new list of tags associated with this buffer.

     For language specific hooks, make sure you define this as a local hook.

If you want tighter interaction with how the user is editing different tags, you can use this hook instead.

 -- Variable: semantic-after-partial-cache-change-hook
     Hooks run after the buffer cache has been updated.

     This hook will run when the cache has been partially reparsed. Partial reparses are incurred when a user edits a buffer, and only the modified sections are rescanned.

     Hook functions must take one argument, which is the list of tokens updated in the current buffer.

     For language specific hooks, make sure you define this as a local hook.

It is also useful to clean up any data your application is using when semantic flushes its tags table.

 -- Variable: semantic-before-toplevel-cache-flush-hook
     Hooks run before the toplevel nonterminal cache is flushed. For language specific hooks, make sure you define this as a local hook.  This hook is called before a corresponding semantic-after-toplevel-cache-change-hook which is also called during a flush when the cache is given a new value of nil.

Lexical Safety
==============

If you application frequenly requests lists of tags upon user request, it may be important to avoid lexical problems that frequenly occur when the user has partially written an expression, such as starting a string, or argument list.

You can protect your code from lexical problems with this macro:

 -- Function: semantic-lex-catch-errors symbol &rest forms
     Using SYMBOL, execute FORMS catching lexical errors. If FORMS results in a call to the parser that throws a lexical error, the error will be caught here without the buffer's cache being thrown out of date. If there is an error, the syntax that failed is returned. If there is no error, then the last value of FORMS is returned.

It is important to provide a good SYMBOL so that these macros can nest correctly.

If you want your code to run anyway, even if there is a lexical error, using this macro like this:

     (semantic-lex-catch-errors
        (semantic-fetch-tags))

will put the parser into the 'unparseable' state, and allow other routines to request the tag table without incurring additional parser attempts.

Semantic Database
*****************

Semanticdb is a database mechanism for storing tags parsed by semantic.  The database operates in the background, saving tags as they are parsed between sessions.  When a file is read in, and there is a previously created set of tags available for it, sematnicdb will save time by not parsing the file, and using the cached copy.

In applications, semanticdb can provide access to the sum of all tags in a project or in the basic system.  This database can they be searched using a set of special routines.

Semanticdb in Programs::
========================

If you write a program using semanticdb, you will probably want to make sure it is active in your program.

 -- Function: semanticdb-minor-mode-p
     Return non-nil if semanticdb-minor-mode is active.

Since semanticdb is optional, it is best if a program can gracefully degrade service when semanticdb is not available, or to throw an error letting the user know it is required to be active.

At the simplest level, you can ask if a given file is in the database, recieving a tag table.  Semanticdb will give you an accurate set of tags just by asking.

 -- Function: semanticdb-file-stream file
     Return a list of tags belonging to FILE. If file has database tags available in the database, return them. If file does not have tags available, then load the file, and create them.

Alternately, you can get at the table object for a file by asking for it.

 -- Function: semanticdb-file-table-object file &optional dontload
     Return a semanticdb table belonging to FILE. If file has database tags available in the database, return it. If file does not have tags available, and DONTLOAD is nil, then load the tags for FILE, and create a new table object for it. DONTLOAD does not affect the creation of new database objects.

Semanticdb Tag Queries
======================

You can search for tags in the semantic database using the semanticdb-find API.  It is important to note that database search functions do not return a plain list of tags.  This is because some tags may not be loaded in a buffer, which means that the found tag would not have an overlay, and no way to determine where it came from.

As such, all search functions return a special Database Results list.

There are several types of searches, divided into groups by implementation.  While it is possible to add new types of searches, or write custom searches, the built in searches are usually the only ones available in system backends System Databases.

When the term brute or brutish is used as a search criteria, that is distinguishing between an include-path based search, and a search that scans everything available in a project.

Non-brute force searches assume that all symbols available in a given buffer are on the search path, or in some file that has been included, imported, or otherwise indicated in the source file itself.  While not always true for interpreted languages (such as Emacs Lisp), it is common among declaritive languages.

Sometimes a brute force approach is needed, scanning every file available to the database.  You may want to do this if your application is collecting data unrelated to a file currently being worked on.

DB Results
----------

The successful results of a search returns a special list of the following form:

        ( (DATABASE TAG1 TAG2 ...) (DATABASE2 TAG3 TAG4 ...) ...)

It should not be necessary to access the results in this way, however, as there are several routines that can be used to access this list.

To turn a semanticdb search result into a simple tag table, use:

 -- Function: semanticdb-strip-find-results results &optional find-file-match
     Strip a semanticdb search RESULTS to exclude objects. This makes it appear more like the results of a semantic-find- call. Optional FIND-FILE-MATCH loads all files associated with RESULTS into buffers.  This has the side effect of enabling semantic-tag-buffer to return a value.

To write a function that accepts a tag table, or a semanticdb search result, use this to test if it is a semanticdb search result:

 -- Function: semanticdb-find-results-p resultp
     Non-nil if RESULTP is in the form of a semanticdb search result. This query only really tests the first entry in the list that is RESULTP, but should be good enough for debugging assertions.

 -- Function: semanticdb-find-result-with-nil-p resultp
     Non-nil of RESULTP is in the form of a semanticdb search result. nil is a valid value where a TABLE usually is, but only if the TAG results include overlays. This query only really tests the first entry in the list that is RESULTP, but should be good enough for debugging assertions.

To operate on the search results as though it were a simple tags table, or plain list, use these routines.

 -- Function: semanticdb-find-result-length result
     Number of tags found in RESULT.

 -- Function: semanticdb-find-result-nth result n
     In RESULT, return the Nth search result. This is a 0 based search result, with the first match being element 0.

     The returned value is a cons cell: (TAG . TABLE) where TAG is the tag at the Nth position.  TABLE is the semanticdb table where the TAG was found.  Sometimes TABLE can be nil.

 -- Function: semanticdb-find-result-nth-in-buffer result n
     In RESULT, return the Nth search result. Like semanticdb-find-result-nth, except that only the TAG is returned, and the buffer it is found it will be made current. If the result tag has no position information, the originating buffer is still made current.

DB Search Paths
---------------

For searches based on an include path (non-brutish) a path of tables needs to be generated.  Searching a lot of tables is slow, which is why a brutish search is not always recommended.  An include-based approach can also generate a lot of tables, so you can control how detailed a search is with a throttle variable.

Ideally, each language mode will have a mode-specific value for this throttle value.  A user can also specify their own values if the default is not good enough.

 -- Variable: semanticdb-find-default-throttle
     The default throttle for semanticdb-find routines. The throttle controls how detailed the list of database tables is for a symbol lookup.  The value is a list with the following keys:

     file
          The file the search is being performed from.  This option is here for completeness only, and is assumed to always be on.

     local
          Tables from the same local directory are included.  This includes files directly referenced by a file name which might be in a different directory.

     project
          Tables from the same local project are included If project is specified, then local is assumed.

     unloaded
          If a table is not in memory, load it.  If it is not cached on disk either, get the source, parse it, and create the table.

     system
          Tables from system databases.  These are specifically tables from system header files, or language equivalent.

     recursive
          For include based searches, includes tables referenced by included files.

     omniscience
          Included system databases which are omniscience, or somehow know everything.  Omniscience databases are found in semanticdb-project-system-databases.  The Emacs Lisp system DB is an omniscience database.

You can use the command semanticdb-find-test-translate-path to interactively test out how the path translator is working.  The path translation routines are:

 -- Function: semanticdb-find-translate-path path brutish
     Translate PATH into a list of semantic tables. Path translation involves identifying the PATH input argument in one of the following ways: nil - Take the current buffer, and use it's include list buffer - Use that buffer's include list. filename - Use that file's include list.  If the file is not in a buffer, see of there is a semanticdb table for it.  If not, read that file into a buffer. tag - Get that tag's buffer of file file.  See above. table - Search that table, and it's include list. find result - Search the results of a previous find.

     In addition, once the base path is found, there is the possibility of each added table adding yet more tables to the path, so this routine can return a lengthy list.

     If argument BRUTISH is non-nil, then instead of using the include list, use all tables found in the parent project of the table identified by translating PATH.  Such searches use brute force to scan every available table.

     The return value is a list of objects of type semanticdb-table or it's children.  In the case of passing in a find result, the result is returned unchanged.

     This routine uses semanticdb-find-table-for-include to translate specific include tags into a semanticdb table. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semanticdb-find-table-for-include includetag &optional table
     For a single INCLUDETAG found in TABLE, find a semanticdb-table object INCLUDETAG is a semantic TAG of class 'include. TABLE as defined by semantic-something-to-tag-table to identify where the tag came from.  TABLE is optional if INCLUDETAG has an overlay of :filename attribute. This function can be overloaded (see define-mode-local-override for details).

DB Basic Name Search
--------------------

These searches scan a database table collection for tags based on name.  They are divided into normal and deep searches.  A deep search, as with in buffer tag scanning, implies that all entries are scanned, including those in type declarations.

Normal Searches:

 -- Function: semanticdb-find-tags-by-name name &optional path find-file-match
     Search for all tags matching NAME on PATH. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

 -- Function: semanticdb-find-tags-by-name-regexp regexp &optional path find-file-match
     Search for all tags matching REGEXP on PATH. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

 -- Function: semanticdb-find-tags-for-completion prefix &optional path find-file-match
     Search for all tags matching PREFIX on PATH. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

 -- Function: semanticdb-find-tags-by-class class &optional path find-file-match
     Search for all tags of CLASS on PATH. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

Deep Searches:

 -- Function: semanticdb-deep-find-tags-by-name name &optional path find-file-match
     Search for all tags matching NAME on PATH. Search also in all components of top level tags founds. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

 -- Function: semanticdb-deep-find-tags-by-name-regexp regexp &optional path find-file-match
     Search for all tags matching REGEXP on PATH. Search also in all components of top level tags founds. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

 -- Function: semanticdb-deep-find-tags-for-completion prefix &optional path find-file-match
     Search for all tags matching PREFIX on PATH. Search also in all components of top level tags founds. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

DB Basic Brute Search
---------------------

These searches allow searching on specific attributes of tags, such as name, type, or other attribute.

 -- Function: semanticdb-brute-deep-find-tags-by-name name &optional path find-file-match
     Search for all tags matching NAME on PATH. See semanticdb-find-translate-path for details on PATH. The argument BRUTISH will be set so that searching includes all tables in the current project. FIND-FILE-MATCH indicates that any time a matchi is found, the file associated wit that tag should be loaded into a buffer.

DB Advanced Search
------------------

These are searches that were needed to accomplish some specialized tasks as discovered in utilities.  Advanced searches include matching methods defined outside some parent class.

The reason for advanced searches are so that external repositories such as the Emacs obarray, or java .class files can quickly answer these needed questions without dumping the entire symbol list into Emacs for a regular semanticdb search.

 -- Function: semanticdb-find-tags-external-children-of-type type &optional path find-file-match
     Search for all tags defined outside of TYPE w/ TYPE as a parent. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer.

DB Generic Brute Search
-----------------------

The generic search, semanticdb-find-nonterminal-by-function will call your function with every tag available.  (Very slow for system databases.)

NOTE:  There is no such function.  Hopefully we can get away without implementing one, which will make system databases more useful.  If you are really stuck, you can use the following, though I don't recommend it.

 -- Function: semanticdb-find-tags-collector function &optional path find-file-match brutish
     Search for all tags returned by FUNCTION over PATH. See semanticdb-find-translate-path for details on PATH. FIND-FILE-MATCH indicates that any time a match is found, the file associated with that tag should be loaded into a buffer. If optional argument BRUTISH is non-nil, then ignore include statements, and search all tables in this project tree.

System Databases
================

System Databases are special implementations of the database and table API which make some external tag source appear as though it were a normal buffer with semantic parsed tags available.

Search routines for these databases return a special type of table not associated with a file.  It is important to be aware of this possible return value.

 -- Type: semanticdb-project-database-emacs-lisp
     Database representing Emacs core.

This Emacs database is loaded automatically.

 -- Type: semanticdb-project-database-ebrowse
     Semantic Database deriving tags using the EBROWSE tool. EBROWSE is a C/C++ parser for use with ebrowse Emacs program.

To create new EBROWSE project databases, use:

 -- Command: semanticdb-create-ebrowse-database dir
     Create an EBROSE database for directory DIR. The database file is stored in ~/.semanticdb, or whichever directory is specified by semanticdb-default-system-save-directory.

Semantic Parsed System Databases
--------------------------------

Another kind of system database is one created by the semantic tools, but saved in a special way.  These databases are created with a script, or at some other convenient time.  They can then be searched normally.  They will appear on the same list of system databases as back ends that refer to object files indirectly.

A simple way to create one from Emacs is with this function:

 -- Command: semanticdb-create-system-database path &optional class
     Create a system database starting at PATH. PATH should be a top level directory for a series of files containing declarations for SYSTEM files.  In C, this would be header filaes. CLASS is the class for the database to create.  Only child classes of symbol semanticdb-project-database-system are accepted.

Idle Scheduling
***************

The Semantic Idle Scheduler is a minor mode which performs semantic specific tasks in idle time. See Idle Scheduler (semantic-user.info).

It performs the following tasks in order:

 1. Reprarse the current buffer if needed

 2. Reparse other buffers that need it

 3. Execute other scheduled semantic related operations.

Care is take in the idle scheduler to exit immediatly if user input is detected, improving editing performance.

The reason for grouping these tasks together is so that the automatic reparsing code executes before other idle services.  This allows lexically broken buffers to be detected so that the other applications that follow do not accidentally reparse the buffer leaving unmatched syntax all over.

You can create new minor modes that are automatically scheduled by the semantic idle scheduler.  Create the new minor mode with:

 -- Function: define-semantic-idle-service name doc &rest forms
     Create a new idle services with NAME. DOC will be a documentation string describing FORMS. FORMS will be called during idle time after the current buffer's semantic tag information has been updated. This routines creates the following functions and variables:

User Input Handling
===================

When writing an idle service, it is important for tasks that can take a long time to correctly exit upon user input.

You can test for user input in your idle handler with the following routines:

 -- Function: semantic-throw-on-input from
     Exit with throw when in semantic-exit-on-input on user input. FROM is an indication of where this function is called from as a value to pass to throw.  It is recommended to use the name of the function calling this one.

If you need to carefully extract from your function, you can wrap just a section of your function to exit on user input by wrapping it with this macro:

 -- Function: semantic-exit-on-input symbol &rest forms
     Using SYMBOL as an argument to throw, execute FORMS. If FORMS includes a call to semantic-thow-on-input, then if a user presses any key during execution, this form macro will exit with the value passed to semantic-throw-on-input. If FORMS completes, then the return value is the same as progn.

Upon catching user input, you can try to detect if there was an exit from the return argument, and continue throwing with an additional call to semantic-throw-on-input.

Programming Examples
********************

*** NOTE ***  These examples are for semantic 1.4. Using the below functions will generate compile time warnings with advice on what functions to use in semantic 2.0.

Here are some simple examples that use different aspects of the semantic library APIs.  For fully functional example programs with lots of comments, see the file semantic-examples.el.

Interactively querying for a token name
=======================================

If you need a command that asks the user for a token name, you can get full range completion using the query functions Tag Completion.

     (interactive (list (semantic-read-symbol "Symbol: ")))

Finding a symbol in a buffer
============================

If you have the name of a function or variable, and need to find its location in a buffer, you need a search function.  There is a wide range of searches you can perform Searching Tag Tables.

     (semantic-find-nonterminal-by-name
      "some-name"
      (current-buffer)
      t    ;; look inside structures and classes for these symbols
      nil) ;; do not look inside header files.

Finding a symbol in a project
=============================

If you have the name of a function or variable, and need to find its location somewhere in a project, you need to use the Semantic Database see semanticdb.  There are many search functions similar to the ones found in Searching Tag Tables.

The Semantic Database is interesting in that the return structure is not

Locating a token in a buffer
============================

If you have a nonterminal token, or a list of them, you may want to find their position in a buffer.

     (semanticdb-find-nonterminal-by-name
      "symbol"
      nil   ;; Defaults to the current project's database list.
      t     ;; Search inside types
      nil   ;; Do not search include files
      nil   ;; Only search files in the same mode (all C files)
      t     ;; When a token is found, make sure it is loaded in a buffer.
      )

Of interesting note above, semanticdb can find symbols in files that are not loaded into an Emacs buffer.  These tokens do not have an associated overlay, and the function semantic-token-buffer will fail.

The last parameter's tells the search function to find-file-noselect any file in which a matching token was found.  This will allow you to merge all the tokens into a completion list, or other flat list needed by most functions that use association lists.

If you do not ask semanticdb to load those files, you will need to explicitly request the database object (found in the car of each sublist) get the file loaded.  It is useful to not auto find all files if you don't need to jump to that token.

Converting a token into a human readable string.
================================================

A tag is a rather unpleasant Lisp structure when trying to decipher what is going on.  As such, there is a wide range of functions available that can convert a token into a human readable, and colorful string Format Tag.

If you program interfaces with lots of users, you will probably want to have your program define a configurable variable that will let users change the visible portion of your program.

     (defcustom my-summary-function 'semantic-uml-prototype-nonterminal
       "*Function to use when showing info about my tag."
       :group 'my-program
       :type semantic-format-tag-custom-list)

Note the special type provided by Semantic.

Next, you can call this function to create a string.

     (funcall my-summary-function tag
                                  tag-parent
                                  t ; use color
                                  )

In this case, TAG-PARENT is an optional argument.  In many cases, parent is not used by the outputting function.  The parent must be a tag whose semantic-tag-componenets contains TAG, or nil for top-level definitions.  In particular, C++ needs the parent to correctly calculate the protection of each method.

Deriving the Current Context
****************************

This chapter deals with how to derive the current context, and also how a language maintainer can get the current context API to work with their language.

By default, the behavior will function in C like languages.  This means languages with parenthetical blocks, and type dereferencing which uses a similar form.

Blocks and Navigation
=====================

Source code is typically built up of control structures, and blocks of context, or lexical scope.  Semantic terms these lexical scopes as a ``context''.  The following functions can be used to navigate contexts. Some of them are override functions.  Language authors can override a subset of them to make them work for their language.

 -- Function: semantic-up-context &optional point bounds-type
     Move point up one context from POINT. Return non-nil if there are no more context levels. Overloaded functions using up-context take no parameters. BOUNDS-TYPE is a symbol representing a tag class to restrict movement to.  If this is nil, 'function is used. This will find the smallest tag of that class (function, variable, type, etc) and make sure non-nil is returned if you cannot go up past the bounds of that tag. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-beginning-of-context &optional point
     Move POINT to the beginning of the current context. Return non-nil if there is no upper context. The default behavior uses semantic-up-context. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-end-of-context &optional point
     Move POINT to the end of the current context. Return non-nil if there is no upper context. Be default, this uses semantic-up-context, and assumes parenthetical block delimiters. This function can be overloaded (see define-mode-local-override for details).

These next set of functions can be used to navigate across commands.

 -- Function: semantic-end-of-command
     Move to the end of the current command. Be default, uses semantic-command-separation-character. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-beginning-of-command
     Move to the beginning of the current command. Be default, uses semantic-command-separation-character. This function can be overloaded (see define-mode-local-override for details).

Deriving local variables
========================

Within a given context, or block of code, local variables are often defined.  These functions can be used to retrieve lists of locally scoped variables.

 -- Function: semantic-get-local-variables &optional point
     Get the local variables based on POINT's context. Local variables are returned in Semantic tag format. This can be overriden with get-local-variables. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-get-local-arguments &optional point
     Get arguments (variables) from the current context at POINT. Parameters are available if the point is in a function or method. Return a list of tags unlinked from the originating buffer. Arguments are obtained by overriding get-local-arguments, or by the default function semantic-get-local-arguments-default.  This, must return a list of tags, or a list of strings that will be converted to tags. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-get-all-local-variables &optional point
     Get all local variables for this context, and parent contexts. Local variables are returned in Semantic tag format. Be default, this gets local variables, and local arguments. Optional argument POINT is the location to start getting the variables from. This function can be overloaded (see define-mode-local-override for details).

Deriving the Current Context
============================

While a context has already been used to describe blocks of code, other context include more local details, such as the symbol the cursor is on, or the fact we are assigning into some other variable.

These context deriving functions can be overridden to provide language specific behavior.  By default, it assumes a C like language.

 -- Function: semantic-ctxt-current-symbol &optional point
     Return the current symbol the cursor is on at POINT in a list. This will include a list of type/field names when applicable. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-ctxt-current-assignment &optional point
     Return the current assignment near the cursor at POINT. Return a list as per semantic-ctxt-current-symbol. Return nil if there is nothing relevant. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-ctxt-current-function &optional point
     Return the current function call the cursor is in at POINT. The function returned is the one accepting the arguments that the cursor is currently in.  It will not return function symbol if the cursor is on the text representing that function. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-ctxt-current-argument &optional point
     Return the index of the argument position the cursor is on at POINT. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-ctxt-current-thing
     Calculate a thing identified by the current cursor position. Calls previously defined semantic-ctxt-current-... calls until something gets a match.  See semantic-ctxt-current-symbol, semantic-ctxt-current-function, and semantic-ctxt-current-assignment for details on the return value.

 -- Function: semantic-ctxt-current-class-list &optional point
     Return a list of tag classes that are allowed at POINT. If POINT is nil, the current buffer location is used. For example, in Emacs Lisp, the symbol after a ( is most likely a function.  In a makefile, symbols after a : are rules, and symbols after a $( are variables. This function can be overloaded (see define-mode-local-override for details).

 -- Function: semantic-ctxt-scoped-types &optional point
     Return a list of type names currently in scope at POINT. The return value can be a mixed list of either strings (names of types that are in scope) or actual tags (type declared locally that may or may not have a name.) This function can be overloaded (see define-mode-local-override for details).

Analysis of the current context
===============================

The context parsing API is used in a context analysis library.  This library provides high level routines for scanning through token databases to create lists of token associates.  At it's core is a set of EIEIO classes defining a context.  The context contains information about what was parsed at a given position, such as the strings there, and they type of assignment.  The analysis library then searches the databases to determine the types and names available.

Two high level functions which can be run interactively are:

 -- Command: semantic-analyze-current-context &optional position
     Analyze the current context at optional POSITION. If called interactively, display interesting information about POSITION in a separate buffer. Returns an object based on symbol semantic-analyze-context.

     This function can be overriden with the symbol analyze-context. When overriding this function, your override will be called while cursor is at POSITION.  In addition, your function will not be called if a cached copy of the return object is found. This function can be overloaded (see define-mode-local-override for details).

 -- Command: semantic-analyze-possible-completions context
     Return a list of semantic tags which are possible completions. CONTEXT is either a position (such as point), or a precalculated context.  Passing in a context is useful if the caller also needs to access parts of the analysis. Completions run through the following filters:

       * Elements currently in scope

       * Constants currently in scope

       * Elements match the :prefix in the CONTEXT.

       * Type of the completion matches the type of the context.

     Context type matching can identify the following:

       * No specific type

       * Assignment into a variable of some type.

       * Argument to a function with type constraints.

     When called interactively, displays the list of possible completions in a buffer. This function can be overloaded (see define-mode-local-override for details).

Analysis Overview
-----------------

The semantic analysis function semantic-analye-current-context creates an Analysis Object.  See Analysis Objects.  This object contains many useful pieces of information needed to do any other kind of intelligent action on the local context.

If you call this function interactively, it will popup a buffer with a summary of the return value.  This is useful when debugging.

     +--------+   +----------------+   +----------------------------+
     | Buffer |---| Context Parser |---| Local Context Synax Result |
     +--------+   +----------------+   +----------------------------+
         |                                        |
     +--------+   +-----------+                   |
     | Parser |---| Tag Table |------------+      |
     +--------+   +-----------+            |      |
                                           V      V
     +-------------+                     +-------------------+
     | Semantic DB |-------------------->| Semantic Analyzer |
     +-------------+                     +-------------------+
                                                  |
                                                  V
                                          +-----------------+
                                          | Analysis Object |
                                          +-----------------+

Analysis Objects
----------------

Completion Overview
-------------------

Application level Data structure debugger
*****************************************

The data structures that Semantic provides can be complex, and figuring out why some level of application API performs incorrectly can be difficult.

The semantic Application debugger provides a way to look inside the various data structures of Semantic in a structures and complete way to help identify what a problem may be.

App Debugger Entry Points
=========================

There are a few basic functions that enter into the debugger:

 -- Command: semantic-adebug-bovinate
     The same as bovinate. Display the results in a debug buffer.

 -- Command: semantic-adebug-searchdb regex
     Search the semanticdb for REGEX for the current buffer. Display the results as a debug list.

 -- Command: semantic-adebug-analyze
     Perform semantic-analyze-current-context. Display the results as a debug list.

adebug-mode
===========

The semantic debugger mode provides a simple user facing UI for looking into the data structures.

 -- Command: semantic-adebug-mode
     Major-mode for the Analyzer debugger.

     Keybindings:

     spc
          semantic-adebug-expand-or-contract

     n
          semantic-adebug-next-expando

     p
          semantic-adebug-prev-expando

     n
          semantic-adebug-next

     p
          semantic-adebug-prev

     <mouse-2>
          semantic-adebug-expand-or-contract-mouse

Create new debugger entry commands
==================================

Creating a new Application debugger entry point is easy. First, get a datastructure you need to analyze.

The first function to call is:

 -- Function: semantic-adebug-new-buffer name
     Create a new adebug buffer with NAME.

Next, you need to find the correct function for inserting your datastructure.  All adebug insertion functions are of the form semantic-adebug-insert-THING, where THING is whatever you object is.  Use Emacs help to pick something out.

GNU Free Documentation License
******************************

     [file fdl.texi not included]