Adding
Parsers
Each parser in Source-Navigator
is capable of understanding source files written in a specific programming
language. In the Source-Navigator suite, parsers are stand-alone executables
which adhere to a consistent command line interface. This interface allows
Source-Navigator to control the parser's behavior through command line
switches that the parser is expected to observe.
The Source-Navigator Software
Development Kit (SDK) provides a C-based application programming interface
(API) which enables parsers to insert information
into a project database.
All files listed in SDK-Related
Files can be found in the .../share/sdk directory.
|
|
|
|
|
|
|
libdbutils.a
libpafdb.a
libsnptools.a
libtcl8.1.a
|
Source-Navigator library
files
|
/parsers/examples/assembly
|
Makefile
README
a.c
abrowser.l.in
b.c
build-macros
linux-i486-elf.m4
solaris-sparc.m4
toolbox.m4
|
|
|
Makefile
README
blobsql.elf
ebrowser.l
|
Makefile for ebrowser.l
The latest release information
Example source
Parser ( lex)
|
Parsers may be implemented
in any programming language. Naturally, the task of writing a parser is
significantly simpler when using compiler generation tools such as GNU
flex. All of the examples provided with Source-Navigator are written using
GNU flex.
A sample parser for the Elf
language (an embedded SQL-like language) is provided with the SDK. To experiment
with this parser, change to the .../share/sdk/parsers directory
and type:
make test
This will compile the parser
and then parse a number of Elf source files, placing items of interest
into a project database.
The
Parser Toolbox Library
A library, implemented above
the API, simplifies the task of writing new parsers. This library, known
as the
parser toolbox, allows programmers
to focus on the issues of parsing source files in their chosen language
and then storing the relevant information in the database.
The toolbox provides a number
of C functions that can help you write your parser with less effort. These
functions can be grouped as follows:
-
functions to maintain line and column counts
in your source files. Whenever the parser encounters an interesting symbol
in the source program, it must know where in the source file the symbol
occurred.
That is, the function
sn_advance_line()
will increment a line counter maintained internally by the toolbox library.
This function would be called along with other actions that the parser
might perform when it encounters a newline character in the source text.
-
a function to determine the name of the source
file currently being parsed.
-
miscellaneous text processing functions for
counting the number of lines and columns consumed by a given block of text
and so on.
-
an entry point, called sn_main , which
manages all of the details of interfacing with Source-Navigator and parsing
each source file. In most circumstances, it is sufficient for the parser
to call sn_main and allow this function to call back to your actual
parsing function when required.
-
This introduces
a number of important concepts best explained with an example:
int sn_main(int argc, char *argv[], char *lang_string,
FILE **infile_ref, int(*parse)(), void(*reset)());
char group[] = "java";
int main(int argc, char *argv[])
{
return sn_main(argc, argv, group, &yyin, yylex, reset);
}
When calling
sn_main,
it is necessary to pass:
-
the argc and argv variables
as passed into your main() function. This allows the library to
access the command line options given by Source-Navigator.
-
a pointer to a string identifying the language.
In the example above, the string is called group .
-
a pointer to a FILE * stream variable.
You must pass the address of this variable as the library will manipulate
the stream when opening new source files. If you are using GNU flex/bison,
then pass a pointer to the global variable yyin .
-
A pointer to a function which takes no arguments
and returns an int. This is a pointer to your actual parsing function.
If you are using GNU flex/bison, then pass yylex or
yyparse.
-
A pointer to a function which takes no arguments
and returns void . This function is expected to perform any actions
prior to processing the next source file. Typically this function might
look like:
void reset()
{
sn_reset_line(); /* reset line count */
sn_reset_column(); /* reset column count */
}
A detailed description of
the functions available in the parser toolbox library can be found in
.../share/sdk/include/snptools.h.
Unless the parser has to do
something out of the ordinary, it should be possible to create a new parser
by following these steps:
-
#include snptools.h in your program
(or within the verbatim section of your lex specification).
-
Define a main function which calls
sn_main
with the appropriate arguments and returns the result of sn_main
to the host environment.
-
Utilize the parser toolbox routines to simplify
your work within the parser. For example, sn_message may be called
to display messages in a dialog box that is shown to the user during the
parsing process.
-
When your parser recognizes important language
constructs such as comments, function declarations, or function invocations,
call the appropriate sn_insert function to insert this information
into the project database.
-
Link your final program with the sntools
library. A typical command line for linking a parser can be found in the
Makefile given in the .../share/sdk/parsers/examples/elf directory.
Project database calls
The toolbox library provides
a number of functions for inserting information into the database that
the parser will encounter in the source text. Each of these functions return
an
int with two possible return values:
sn_insert_symbol
int sn_insert_symbol(int id_type, char *classname,
char *identifier, char *filename, int start_lineno,
int startCol, int endLine, int endCol, unsigned long attrib,
char *returnType, char *argTypes, char *argNames,
char *comment, int highStartLine, int highStartCol,
int highEndLine, int highEndCol);
sn_insert_symbol
inserts a symbol into the project database.
type determines the type of
the symbol. Possible values are:
|
type definitions (e.g.
a C typedef).
|
|
class definition (particularly
for object-oriented languages).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
member variables within
a common block.
|
|
|
|
member function declarations.
|
|
|
|
|
|
definitions of unions
or variant records.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
reference to undefined
symbols.
|
|
the name of the class,
structure or common block of the symbol if the symbol's type is one of
SN_MBR_FUNC_DEF,
SN_MBR_VAR_DEF,
SN_COMMON_MBR_VAR_DEF,
or SN_CLASS_INHERIT. Otherwise,
classname must be a NULL
pointer. If the symbol's type is SN_CLASS_INHERIT,
classname
contains the name of the base class.
|
|
the name of the symbol
to be inserted into the project database.
|
|
the name of the source
file in which this symbol was encountered.
|
|
the line number of the
position where the symbol starts.
|
|
the column number of
the position where the symbol starts.
|
|
the line number of the
position where the symbol ends.
|
|
the column number of
the position where the symbol ends.
|
|
contains attributes of
the symbol definitions (see sn.h).
|
|
a string describing the
return type of the function, subroutine or method. If the symbol is not
one of these types, pass a NULL pointer.
|
|
a string containing a
comma-separated list of argument types for the argument list of functions,
subroutines or methods. If the symbol is not one of these types, pass a
NULL pointer.
|
|
a string containing a
comma-separated list of argument names for the argument list of functions,
subroutines or methods. If the symbol is not one of these types, pass a
NULL pointer.
|
|
a string containing the
comment that often occurs after a definition in the source text. Note that
8-bit characters including '\n' may be used in the string. If
there is no comment, pass a NULL pointer.
|
|
the line number of the
position where the highlighting of the symbol starts.
|
|
the column number of
the position where the highlighting of the symbol starts.
|
|
the line number of the
position where the highlighting of the symbol ends.
|
|
the column number of
the position where the highlighting of the symbol ends.
|
Examples
The following example inserts
a class definition:
sn_insert_symbol(SN_CLASS_DEF, NULL, classname,
sn_current_file(), sn_line(), sn_column(), sn_line(),
sn_column() + strlen(classname), 0L, NULL, NULL,
NULL, NULL, sn_line(), sn_column(), sn_line(),
sn_column() + strlen(classname));
The following example inserts
a method definition:
sn_insert_symbol(SN_MBR_FUNC_DEF, classname, methodname,
sn_current_file(), sn_line(), sn_column(), sn_line(),
sn_column() + strlen(methodname), 0L, NULL, NULL,
NULL, NULL, sn_line(), sn_column(), sn_line(),
sn_column() + strlen(methodname));
For a C function with the
following prototype:
int transform(struct coord * p, int x, int y, unsigned att);
the following example inserts
the definition into the database:
sn_insert_symbol(SN_FUNC_DEF, NULL, "transform",
sn_current_file(), sn_line(), sn_column(), sn_line(),
sn_column + len, 0L, "int", "struct coord *, int, int,
unsigned", "p,x,y,att", NULL, NULL, sn_line(), sn_column(),
sn_line(), sn_column() + len);
sn_insert_xref
sn_insert_xref
inserts cross-referencing information.
One of the most useful aspects
of Source-Navigator is its cross-referencing capabilities. For instance,
it is possible to see which functions are used by other functions or which
functions modify a particular global variable.
Where possible, a parser should
attempt to collect information of this nature and insert it into the project
database. In a language with a flat namespace such as C, this can be achieved
by noting the name of the current function within the parser. If a function
invocation is encountered in the source text, then the cross-referencing
can be inferred based on the current function's name and the function being
called.
Cross-referencing information
is added using:
int sn_insert_xref(int type,
int scope_type, int scope_level,
char *classname, char *funcname, char *argtypes,
char *refclass, char *refsymbol, char *ref_arg_types,
char *filename, int lineno, int acc);
|
describes the type of
the referenced symbol. It must be one of the following:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
describes the type of
the location where the cross-reference information is reported. It must
be one of the following:
|
|
|
|
|
|
|
|
describes the scope level
of the referenced symbol. It must be one of:
|
|
|
|
|
|
a string containing the
class name of the method if scope_type is SN_MBR_FUNC_DEF
; otherwise, it must be a NULL pointer.
|
|
a string containing the
function, method or subroutine name in which the reference information
is reported.
|
|
a string containing a
comma-separated list of argument types for the argument list of functions,
subroutines or methods. Pass NULL if there are no arguments.
|
|
a string containing the
class, structure or common block name of the referred symbol. If the symbol
is not within a namespace, pass a NULL pointer.
|
|
the name of the referred
symbol.
|
|
a string containing a
comma-separated list of argument types if the referred symbol's type is
a method, subroutine or function. Pass NULL if there are no arguments or
the referred symbol is not one of these types.
|
|
the name of the source
file in which the reference information is reported.
|
|
the line number of the
source file in which the reference information is reported.
|
|
the level of access to
the referenced symbol and is one of:
|
|
symbol is read (e.g.
if (x) { ... }).
|
|
symbol is modified (e.g.
x
= 10).
|
|
variable is passed to
a subroutine/function.
|
|
|
Examples
The following example inserts
cross-referencing information for a function that is called from another
function.
sn_insert_xref(SN_REF_TO_FUNCTION, SN_FUNC_DEF,
SN_REF_SCOPE_GLOBAL, NULL, currentFunction, NULL, NULL,
calledFunction, NULL, sn_current_file(), sn_line(),
SN_REF_PASS);
The following example inserts
cross-referencing information for a function that is called from a member
function called
insert which belongs to a C++ class called
Stack
.
sn_insert_xref(SN_REF_TO_MBR_VAR, SN_MBR_FUNC_DEF,
SN_REF_SCOPE_GLOBAL, "Stack", "insert", NULL, "Stack",
"i", NULL, sn_current_file(), sn_line(), SN_REF_READ);
sn_insert_comment
sn_insert_comment
inserts comments into the project database.
When comments are encountered
in the source text, the parser should call
sn_insert_comment to
add these comments to the project database. In some Source-Navigator projects,
the user will choose to not include comments, but the parser should call
this function regardless. The library function will decide whether or not
to actually store the information in the database.
Comments are added to the
database using:
int sn_insert_comment(char *classname, char *funcname,
char *filename, char *comment, int beg_line, int beg_col);
|
a string containing the
name of the class or method where the comment was found or NULL.
|
|
a string containing the
name of the function or method in which the comment was found, or NULL
if the comment is outside any function or method scope.
|
|
a string containing the
name of the current file being parsed.
|
|
a string containing the
comment without the comment separators. For example, in Tcl this would
exclude the leading " #" character.
|
|
the line number of the
source file where the comment begins.
|
|
the column number of
the source file where the comment begins.
|
Integration with
Source-Navigator
A completed parser can be
integrated into Source-Navigator by following these steps:
-
Copy the parser executable into the .../bin
directory.
-
Edit sn_prop.cfg in the directory .../share/etc.
The Tcl procedure sn_add_parser is used to add parsers to the
configuration. The following example shows how to include support for Java:
sn_add_parser java -suffix {*.java} \
-brow_cmd $odd_path(bindir)/jbrowser \
-high_cmd $odd_path(bindir)/jbrowser \
-high_switch "-h"
When Source-Navigator is
restarted, support for the new language becomes available. When creating
a project, the
Parsers tab in the
Project Preferences
dialog shows the new language and its associated filename extensions.
This is all that is required
to add new language support to Source-Navigator. If a project is created
which contains files with any of the specified filename extensions, the
new parser is invoked to process those files as a part of the overall parsing
process.