1 Semantic 2.0 is a major new version.
5 Semantic now uses `inversion' to track version numbers of different
6 releases. Semanticdb table files save the version number they are
7 created at, and can identify old tables to regenerate them when an
8 incompatible upgrade occurs.
14 There is a new naming convention for the things Semantic refers too.
15 The list of renamed functions is far to long to list here. All
16 functions that used the old conventions have been changed to use the
17 new conventions, and are not explicitly listed here unless there is
18 an incompatible change.
22 token - Refers to a lexical analyzer production.
23 stream - A list of tokens
24 tag - Refers to a datastructure created by a grammar to represent
25 something in a language file
26 table - A hierarchical list of tags.
27 tag-class - A tag may represent a function, data type, or variable.
28 parse - Run a source file through a parser.
32 token - Could be a lexical token, or a tag
33 nonterminal - Could be a grammar nonterminal, or a tag
34 stream - Could be a list of lexical tokens, or a tag table.
35 token-type - New represented as tag-class
36 bovinate - Run a source file through a parser.
38 *** Obsoletion stratety
40 The new framework function `semantic-alias-obsolete' has been used to
41 obsolete old functions and allow old code to continue working. Byte
42 compiler warnings will be produced if old functions are used. The
43 old function will be completely removed in a future release.
45 For variables, the function `semantic-varalias-obsolete' has been
46 used. When available it uses a native alias routine for compatibility
47 or displays a warning.
51 *** The old `semantic-flex' API is deprecated.
53 The lexical analysis toolkit has been completely rewritten as a new
56 *** Lexical token management.
59 Create a new lexical token.
61 `semantic-lex-token-class'
62 Fetch the class of a lexical token.
64 `semantic-lex-token-bounds'
65 Fetch the start and end locations of a lexical token.
67 `semantic-lex-token-start' and `semantic-lex-token-end'
68 Respectively fetch the start and end position of a lexical token.
70 `semantic-lex-token-text'
71 Fetch the text associated with a lexical token.
73 *** Macros to easily build custom lexers.
76 Define a new lexer as a set of lexical rules.
79 Base macro to create a lexical rule.
81 A lexical rule associates a PATTERN to an ACTION.
83 A PATTERN describes how to match data in the input stream, with a
84 combination of regular expressions and strings.
86 An ACTION is a set of arbitrary Emacs Lisp statements executed when
87 the input stream matches the corresponding PATTERN, typically to push
88 a new token on the lexical stream.
90 `define-lex-regex-analyzer' and `define-lex-simple-regex-analyzer'
91 Create lexical rules that match a regexp.
93 `define-lex-block-analyzer'
94 Create a lexical rule for paired delimiters blocks.
96 *** A set of useful lexical rules is predefined.
98 `semantic-lex-beginning-of-line'
99 `semantic-lex-newline'
100 `semantic-lex-newline-as-whitespace'
101 `semantic-lex-ignore-newline'
102 `semantic-lex-whitespace'
103 `semantic-lex-ignore-whitespace'
104 `semantic-lex-number'
105 `semantic-lex-symbol-or-keyword'
106 `semantic-lex-charquote'
107 `semantic-lex-punctuation'
108 `semantic-lex-punctuation-type'
109 `semantic-lex-paren-or-list'
110 `semantic-lex-open-paren'
111 `semantic-lex-close-paren'
112 `semantic-lex-string'
113 `semantic-lex-comments'
114 `semantic-lex-comments-as-whitespace'
115 `semantic-lex-ignore-comments'
116 `semantic-lex-default-action'
118 *** Some lexers are predefined too.
120 A comment lexer: `semantic-comment-lexer' that handles comments.
121 A `semantic-simple-lexer' that ignores comments and whitespace, and
122 returns tokens corresponding to syntax as specified by the syntax
125 *** `semantic-lex' core "overload" function.
127 Called to lexically analyze text in an area of the current buffer.
129 *** debugging bad syntax
131 Using `semantic-lex-debug-analyzers' combined with `debug-on-error'
132 allows you to debug analyzers inside usually protected parsing
137 *** API to manage a parse tree state.
139 `semantic-parse-tree-set-unparseable'
140 Indicate that there are lexical issues that prevent parsing.
142 `semantic-parse-tree-set-unparseable-p'
143 Return non-nil if there are lexical issues that prevent parsing.
145 `semantic-parse-tree-set-needs-update'
146 Indicate that the current parse tree needs to be updated.
148 `semantic-parse-tree-needs-update-p'
149 Return non-nil if the current parse tree needs to be updated.
151 `semantic-parse-tree-set-needs-rebuild'
152 Indicate that the current parse tree needs to be rebuilt.
154 `semantic-parse-tree-needs-rebuild-p'
155 Return non-nil if the current parse tree needs to be rebuilt.
157 `semantic-parse-tree-set-up-to-date'
158 Indicate that the current parse tree is up to date.
160 `semantic-parse-tree-up-to-date-p'
161 Return non-nil if the current parse tree is up to date.
163 *** Core "overload" functions to abstract call to parsers.
165 New parsers can be plugged-in easily by overriding the following
168 `semantic-parse-stream' called to parse a given stream, starting at a
169 given nonterminal rule.
171 `semantic-parse-changes' called to reparse changes in the current
174 `semantic-parse-region' called to parse a buffer's area.
176 *** Four core parsers are available.
178 - Two general parsers
180 - The Semantic "bovinator".
181 - A new LALR(1) parser: Wisent, port of Bison 1.3 in Elisp.
183 - Two specific parsers
185 - The Elisp parser based on the Emacs built-in function `read'.
186 - A regexp parser used in `texinfo-mode'.
188 *** Iterative parser.
190 Iterative parsers are better than rule-based iterative functions
191 in that they can handle obscure errors more cleanly.
193 The new `semantic-repeat-parse-whole-stream' helper function abstracts
194 this action for other parser centric routines.
196 *** Incremental parser.
198 In Semantic 1.x, changes were handled in a simplistic manner, where
199 tags that changed were reparsed one at a time. Any other form of
200 edit were managed through a full re-parse.
202 The new `semantic-edits-incremental-parser' attempts to minimize the
203 number of times a full re-parse needs to occur. While overlays and
204 tags will continue to be recycled in the simple case, new cases
205 where tags are inserted or old tags removed from the original list
208 *** Lexical Safety feature.
210 You can protect code from reparsing the buffer if there are lexical
211 errors with the `semantic-lex-catch-errors' macro.
213 *** Deprecated and removed API.
215 The `semantic-flex' API is deprecated, and replaced by the new
216 `semantic-lex' API. The old API still exists for compatibility with
219 `semantic-parse-region' should be used instead of
220 `semantic-bovinate-region-until-error' and
221 `semantic-bovinate-from-nonterminal-full' which still exist for
222 compatibility with the Semantic 1.x API.
224 The old incremental parser `semantic-rebovinate-token' and associated
225 `semantic-show-dirty-mode' have been removed and replaced respectively
226 by new `semantic-edits-incremental-parser' and
227 `semantic-highlight-edits-mode'.
229 `semantic-bovinate-toplevel-override' is replaced by the new core
232 `semantic-bovine-toplevel-full-reparse-needed-p' and
233 `semantic-bovine-toplevel-partial-reparse-needed-p' are replaced by
234 the new parse tree state management API.
236 `semanticdb-project-predicates' hook is renamed
237 `semanticdb-project-predicate-functions'.
241 Semantic 2.0 introduced a new common grammar framework to simplify
242 development of grammars suitable to parser needs.
244 ** New "abstract" major mode: `semantic-grammar-mode'.
245 That defines a useful environment to develop grammars (indentation,
246 syntax highlighting, parsing, etc.).
248 By deriving new "concrete" major modes one can provide customized
249 generators that convert a grammar parse tree into Elisp forms needed
250 by a particular parser.
252 ** New "overloads" are provided to customize Elisp generation.
254 `grammar-setupcode-builder'
255 That returns the setup code form.
257 `grammar-parsetable-builder'
258 That returns the parser table value.
260 `grammar-keywordtable-builder'
261 That returns the keyword table table value.
263 `grammar-tokentable-builder'
264 That returns the token table value.
266 ** New grammar modes derived from `semantic-grammar-mode'.
268 *** `bovine-grammar-mode'.
269 That converts grammar input form into Elisp code to be used by the
270 "bovinator". Such grammars are associated to the .by file extension.
271 Old grammars in .bnf files are no longer supported.
273 *** `wisent-grammar-mode'.
274 That converts grammar input form into Elisp code to be used by the
275 Wisent LALR parser. Such grammars are associated to the .wy
278 ** Grammar build process
280 A new grammar construction process separates generated code from hand
281 written code. An semantic specific EDE extension will generate
282 Makefile rules to build these files.
284 Language specific human written code must call the automatically
285 generated setup function.
287 *** Auto-generation of lexical rules
289 The new %type statement combined with the use of %token and %keyword
290 statements permits the declaration of a lexical type and associates it
291 with patterns that define how to match lexical tokens of that type.
293 The grammar construction process can benefit from the %type, %keyword
294 and %token declarations to automatically generate the definition of a
295 lexical rule for each explicitly declared lexical type.
297 Default values are provided for well known types like <keyword>,
298 <symbol>, <string>, <number>, <punctuation>, and <block>. Those types
299 assume that the correct patterns are provided by %keyword and %token
300 statements, a simple "%type <type>" declaration should generally
301 suffice to auto-generate a suitable lexical rule.
303 It is then easy to put predefined and auto-generated lexical rules
304 together to build ad-hoc lexical analyzers. Examples are available
305 among the grammars included in the distribution.
309 A file FOO.by will create the file FOO-by.el, and FOO-by.elc
314 A file FOO.wy will create the file FOO-wy.el, and FOO-wy.elc
319 ** secondary overlays
320 An API for adding secondary overlays onto a tag has been added.
321 This allows you to add/remove arbitrary visual effects onto a tag.
322 Uses tag link hooks to make sure the overlays are added/removed
326 New functions `semantic-set-tag-folded' and `semantic-tag-folded-p'
327 allow you to put a tag into a 'folded' state where only the 1st line
334 *** `semanticdb-abstract-table'
335 New base class for all tables of tokens lists.
337 *** `semantidb-table'
338 Inherits from the abstract table.
340 *** `semanticdb-project-database'
341 No longer saves itself to a file.
343 ** semanticdb-file.el
344 New routines for databases saved to disk in plain EIEIO save files.
345 Depends on `inversion' for save file compatibility tests.
347 *** `semanticdb-project-database-file'
348 File based project database type.
350 ** semanticdb-search.el
351 All old search routines have been moved into this file.
352 All old search routines are now methods on database projects.
353 All old search routines have been obsoleted. Use semanticdb-find.el
354 based routines instead. These functions are still maintained because
355 the semanticdb-find based routines do not yet handle all the same
356 search paramters as the old routines.
358 *** `semanticdb-search-results-table'
359 Class for any search in a database with no tables of its own.
361 *** Different search classes
362 Searches in databases have been broken into three classes.
363 - Basic search - on values as stored directly in a semantic token.
364 - Advanced search - complex searches of relationships. Needed for
365 system databases which cannot support a generic search.
366 - Generic search - Takes a predicate. System databases cannot usually
367 support this style of search.
369 ** semanticdb-find.el
370 New prefered search routines which start with `semanticdb-find...'.
371 These routines take fewer parameters. There are more types of search
372 routines than before to make up for the missing parameters.
375 Scans the current buffer, and all files included into this buffer.
376 Only scans top level tags.
379 A Deep search will "flatten" the tags in a file so they are all
380 visible, including parts of structures or classes.
383 As a basic search, but scans all files in the current project.
385 *** semanticdb-find-translate-path
386 This new routines determines the "path" to scan. There are two basic
387 types of paths. The default is to examine the specified buffers
388 include list, and only scan those files. Optional "brute" path will
389 scan all files in the current project, including system level databases.
391 *** database Find Results
392 Find Results are somewhat more formalized, and have routines to
395 semanticdb-find-results-p
396 semanticdb-strip-find-results
397 semanticdb-find-result-with-nil-p
398 semanticdb-find-result-length
399 semanticdb-find-result-nth
401 Find results can be passed into any semanticdb-find routine as the
402 path and only those results will be scanned.
404 *** New types of searches not originally in semanticdb-search.el
406 **** semanticdb-find-*-by-name-for-completion
407 Searches "for completion" use different matching constraints
408 equivalent to the regular expression "^chars", which provide a speed
411 ** semanticdb-system.el
412 Representation of a database belonging to system libraries.
413 These databases may not come from source code, and can represent
414 object files, header file libraries and the like.
416 *** `semanticdb-project-database-system'
417 Baseclass for any system database.
419 *** `semanticdb-project-database-system-c'
420 Class for C libraries of header files.
423 Special system database representing Emacs' internal state.
424 Implements all search routines optimized as much as possible.
426 *** `semanticdb-table-emacs-lisp'
427 Table representing search results for Emacs Lisp symbols.
429 *** `semanticdb-project-database-emacs-lisp'
430 Database representing Emacs' internal symbol obarray.
433 Routines for building semantic.cache files from a command line script.
437 ** `semantic-find-tag-parent-by-overlay'
438 Find the parent of a tag by overlays.
442 ** `semantic-show-parser-state-mode'
443 Display the parser state in the modeline.
445 ** `semantic-highlight-edits-mode'
446 Highlight areas that a user edits.
448 ** `semantic-stickyfunc-mode'
449 Make the current functions header `sticky' to the top of the current
450 window. (Emacs 21 required)
452 ** `semantic-auto-parse-mode' obsolete
453 ** `semantic-summary-mode' obsolete
455 ** `semantic-idle-scheduler-mode'
456 Replaces seman`tic-auto-parse-mode'. Reparses all buffers.
457 Also schedules additional functionality.
459 ** `semantic-idle-summary-mode'
460 Replaces seman`tic-summary-mode'. Uses the new idle scheduler
463 ** `semantic-idle-completions-mode'
464 New minor mode for automatically going into inline completion
467 ** `semantic-decoration-mode'
468 New minor mode that manages any form of decoration to be added to
469 tags. It absorbs the duties of:
471 ** `semantic-show-tag-boundaries-mode': obsolete
472 ** `semantic-highlight-by-attribute-mode': obsolete
476 ** `semantic-before-auto-parse-hooks'
477 Run before option `semantic-auto-parse-mode' begins parsing.
479 ** `semantic-after-auto-parse-hooks'
480 Run after option `semantic-auto-parse-mode' has parsed.
484 ** `semantic-analyze-current-context'
485 This is now an overridable function.
486 The objects contain more details on scoping for more accurate
489 ** `semantic-analyze-possible-completions'
490 This is now an overridable function.
494 A lot of things are now auto-loaded on demand. Setup should now be
495 done through the cedet.el initialization file.
499 ** Python parser added.
500 The LALR grammar is based on the official grammar with slight
501 modifications. The tokens generated are formatted similar to
502 those produced by the Java parser.
504 ** Erlang parser added.
505 The grammar was written for semantic 1.4 in BNF format, and converted
508 ** HTML external parser added.
509 The regexp based parser was copied from the texinfo example
510 and modified for semantic 2.0.
512 ** Makefile parser improved.
513 There is now excellent smart completion support.
515 ** Emacs lisp parser improved.
516 The Emacs Lisp parser is now more flexible. Using the new macros
517 `semantic-elisp-setup-form-parser' and
518 `semantic-elisp-reuse-form-parser', it is easy to parse all kind of
519 Lisp forms to produce semantic tags.
521 ** Texinfo parser improved
522 The texinfo parser was updated, and supports the new
528 paragraph-separate: "[
\f]*$"