/usr/share/doc/flex-doc/html/Cxx.html

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- 
The flex manual is placed under the same licensing conditions as the
rest of flex:

Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2012
The Flex Project.

Copyright (C) 1990, 1997 The Regents of the University of California.
All rights reserved.

This code is derived from software contributed to Berkeley by
Vern Paxson.

The United States Government has rights in this work pursuant
to contract no. DE-AC03-76SF00098 between the United States
Department of Energy and the University of California.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

Neither the name of the University nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.

THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. -->
<!-- Created by GNU Texinfo 6.5, http://www.gnu.org/software/texinfo/ -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Cxx (Lexical Analysis With Flex, for Flex 2.6.4)</title>

<meta name="description" content="Cxx (Lexical Analysis With Flex, for Flex 2.6.4)">
<meta name="keywords" content="Cxx (Lexical Analysis With Flex, for Flex 2.6.4)">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<link href="index.html#Top" rel="start" title="Top">
<link href="Indices.html#Indices" rel="index" title="Indices">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="index.html#Top" rel="up" title="Top">
<link href="Reentrant.html#Reentrant" rel="next" title="Reentrant">
<link href="Performance.html#Performance" rel="prev" title="Performance">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.indentedblock {margin-right: 0em}
blockquote.smallindentedblock {margin-right: 0em; font-size: smaller}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smalllisp {margin-left: 3.2em}
kbd {font-style: oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nolinebreak {white-space: nowrap}
span.roman {font-family: initial; font-weight: normal}
span.sansserif {font-family: sans-serif; font-weight: normal}
ul.no-bullet {list-style: none}
-->
</style>


</head>

<body lang="en">
<a name="Cxx"></a>
<div class="header">
<p>
Next: <a href="Reentrant.html#Reentrant" accesskey="n" rel="next">Reentrant</a>, Previous: <a href="Performance.html#Performance" accesskey="p" rel="prev">Performance</a>, Up: <a href="index.html#Top" accesskey="u" rel="up">Top</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Indices.html#Indices" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="Generating-C_002b_002b-Scanners"></a>
<h2 class="chapter">18 Generating C++ Scanners</h2>

<a name="index-c_002b_002b_002c-experimental-form-of-scanner-class"></a>
<a name="index-experimental-form-of-c_002b_002b-scanner-class"></a>
<p><strong>IMPORTANT</strong>: the present form of the scanning class is <em>experimental</em>
and may change considerably between major releases.
</p>
<a name="index-C_002b_002b"></a>
<a name="index-member-functions_002c-C_002b_002b"></a>
<a name="index-methods_002c-c_002b_002b"></a>
<p><code>flex</code> provides two different ways to generate scanners for use
with C++.  The first way is to simply compile a scanner generated by
<code>flex</code> using a C++ compiler instead of a C compiler.  You should
not encounter any compilation errors (see <a href="Reporting-Bugs.html#Reporting-Bugs">Reporting Bugs</a>).  You can
then use C++ code in your rule actions instead of C code.  Note that the
default input source for your scanner remains <samp>yyin</samp>, and default
echoing is still done to <samp>yyout</samp>.  Both of these remain <code>FILE
*</code> variables and not C++ <em>streams</em>.
</p>
<p>You can also use <code>flex</code> to generate a C++ scanner class, using the
&lsquo;<samp>-+</samp>&rsquo; option (or, equivalently, <code>%option c++)</code>, which is
automatically specified if the name of the <code>flex</code> executable ends
in a &rsquo;+&rsquo;, such as <code>flex++</code>.  When using this option, <code>flex</code>
defaults to generating the scanner to the file <samp>lex.yy.cc</samp> instead
of <samp>lex.yy.c</samp>.  The generated scanner includes the header file
<samp>FlexLexer.h</samp>, which defines the interface to two C++ classes.
</p>
<p>The first class in <samp>FlexLexer.h</samp>, <code>FlexLexer</code>,
provides an abstract base class defining the general scanner class
interface.  It provides the following member functions:
</p>
<dl compact="compact">
<dd><a name="index-YYText-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>const char* YYText()</code></dt>
<dd><p>returns the text of the most recently matched token, the equivalent of
<code>yytext</code>.
</p>
<a name="index-YYLeng-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>int YYLeng()</code></dt>
<dd><p>returns the length of the most recently matched token, the equivalent of
<code>yyleng</code>.
</p>
<a name="index-lineno-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>int lineno() const</code></dt>
<dd><p>returns the current input line number (see <code>%option yylineno)</code>, or
<code>1</code> if <code>%option yylineno</code> was not used.
</p>
<a name="index-set_005fdebug-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>void set_debug( int flag )</code></dt>
<dd><p>sets the debugging flag for the scanner, equivalent to assigning to
<code>yy_flex_debug</code> (see <a href="Scanner-Options.html#Scanner-Options">Scanner Options</a>).  Note that you must build
the scanner using <code>%option debug</code> to include debugging information
in it.
</p>
<a name="index-debug-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>int debug() const</code></dt>
<dd><p>returns the current setting of the debugging flag.
</p></dd>
</dl>

<p>Also provided are member functions equivalent to
<code>yy_switch_to_buffer()</code>, <code>yy_create_buffer()</code> (though the
first argument is an <code>istream&amp;</code> object reference and not a
<code>FILE*)</code>, <code>yy_flush_buffer()</code>, <code>yy_delete_buffer()</code>, and
<code>yyrestart()</code> (again, the first argument is a <code>istream&amp;</code>
object reference).
</p>
<a name="index-yyFlexLexer-_0028C_002b_002b-only_0029"></a>
<a name="index-FlexLexer-_0028C_002b_002b-only_0029"></a>
<p>The second class defined in <samp>FlexLexer.h</samp> is <code>yyFlexLexer</code>,
which is derived from <code>FlexLexer</code>.  It defines the following
additional member functions:
</p>
<dl compact="compact">
<dd><a name="index-yyFlexLexer-constructor-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>yyFlexLexer( istream* arg_yyin = 0, ostream* arg_yyout = 0 )</code></dt>
<dt><code>yyFlexLexer( istream&amp; arg_yyin, ostream&amp; arg_yyout )</code></dt>
<dd><p>constructs a <code>yyFlexLexer</code> object using the given streams for input
and output.  If not specified, the streams default to <code>cin</code> and
<code>cout</code>, respectively.  <code>yyFlexLexer</code> does not take ownership of
its stream arguments.  It&rsquo;s up to the user to ensure the streams pointed
to remain alive at least as long as the <code>yyFlexLexer</code> instance.
</p>
<a name="index-yylex-_0028C_002b_002b-version_0029"></a>
</dd>
<dt><code>virtual int yylex()</code></dt>
<dd><p>performs the same role is <code>yylex()</code> does for ordinary <code>flex</code>
scanners: it scans the input stream, consuming tokens, until a rule&rsquo;s
action returns a value.  If you derive a subclass <code>S</code> from
<code>yyFlexLexer</code> and want to access the member functions and variables
of <code>S</code> inside <code>yylex()</code>, then you need to use <code>%option
yyclass=&quot;S&quot;</code> to inform <code>flex</code> that you will be using that subclass
instead of <code>yyFlexLexer</code>.  In this case, rather than generating
<code>yyFlexLexer::yylex()</code>, <code>flex</code> generates <code>S::yylex()</code>
(and also generates a dummy <code>yyFlexLexer::yylex()</code> that calls
<code>yyFlexLexer::LexerError()</code> if called).
</p>
<a name="index-switch_005fstreams-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>virtual void switch_streams(istream* new_in = 0, ostream* new_out = 0)</code></dt>
<dt><code>virtual void switch_streams(istream&amp; new_in, ostream&amp; new_out)</code></dt>
<dd><p>reassigns <code>yyin</code> to <code>new_in</code> (if non-null) and <code>yyout</code> to
<code>new_out</code> (if non-null), deleting the previous input buffer if
<code>yyin</code> is reassigned.
</p>
</dd>
<dt><code>int yylex( istream* new_in, ostream* new_out = 0 )</code></dt>
<dt><code>int yylex( istream&amp; new_in, ostream&amp; new_out )</code></dt>
<dd><p>first switches the input streams via <code>switch_streams( new_in,
new_out )</code> and then returns the value of <code>yylex()</code>.
</p></dd>
</dl>

<p>In addition, <code>yyFlexLexer</code> defines the following protected virtual
functions which you can redefine in derived classes to tailor the
scanner:
</p>
<dl compact="compact">
<dd><a name="index-LexerInput-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>virtual int LexerInput( char* buf, int max_size )</code></dt>
<dd><p>reads up to <code>max_size</code> characters into <code>buf</code> and returns the
number of characters read.  To indicate end-of-input, return 0
characters.  Note that <code>interactive</code> scanners (see the &lsquo;<samp>-B</samp>&rsquo;
and &lsquo;<samp>-I</samp>&rsquo; flags in <a href="Scanner-Options.html#Scanner-Options">Scanner Options</a>) define the macro
<code>YY_INTERACTIVE</code>.  If you redefine <code>LexerInput()</code> and need to
take different actions depending on whether or not the scanner might be
scanning an interactive input source, you can test for the presence of
this name via <code>#ifdef</code> statements.
</p>
<a name="index-LexerOutput-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>virtual void LexerOutput( const char* buf, int size )</code></dt>
<dd><p>writes out <code>size</code> characters from the buffer <code>buf</code>, which, while
<code>NUL</code>-terminated, may also contain internal <code>NUL</code>s if the
scanner&rsquo;s rules can match text with <code>NUL</code>s in them.
</p>
<a name="index-error-reporting_002c-in-C_002b_002b"></a>
<a name="index-LexerError-_0028C_002b_002b-only_0029"></a>
</dd>
<dt><code>virtual void LexerError( const char* msg )</code></dt>
<dd><p>reports a fatal error message.  The default version of this function
writes the message to the stream <code>cerr</code> and exits.
</p></dd>
</dl>

<p>Note that a <code>yyFlexLexer</code> object contains its <em>entire</em>
scanning state.  Thus you can use such objects to create reentrant
scanners, but see also <a href="Reentrant.html#Reentrant">Reentrant</a>.  You can instantiate multiple
instances of the same <code>yyFlexLexer</code> class, and you can also combine
multiple C++ scanner classes together in the same program using the
&lsquo;<samp>-P</samp>&rsquo; option discussed above.
</p>
<p>Finally, note that the <code>%array</code> feature is not available to C++
scanner classes; you must use <code>%pointer</code> (the default).
</p>
<p>Here is an example of a simple C++ scanner:
</p>
<a name="index-C_002b_002b-scanners_002c-use-of"></a>
<div class="example">
<pre class="verbatim">     // An example of using the flex C++ scanner class.

    %{
    #include &lt;iostream&gt;
    using namespace std;
    int mylineno = 0;
    %}

    %option noyywrap c++

    string  \&quot;[^\n&quot;]+\&quot;

    ws      [ \t]+

    alpha   [A-Za-z]
    dig     [0-9]
    name    ({alpha}|{dig}|\$)({alpha}|{dig}|[_.\-/$])*
    num1    [-+]?{dig}+\.?([eE][-+]?{dig}+)?
    num2    [-+]?{dig}*\.{dig}+([eE][-+]?{dig}+)?
    number  {num1}|{num2}

    %%

    {ws}    /* skip blanks and tabs */

    &quot;/*&quot;    {
            int c;

            while((c = yyinput()) != 0)
                {
                if(c == '\n')
                    ++mylineno;

                else if(c == '*')
                    {
                    if((c = yyinput()) == '/')
                        break;
                    else
                        unput(c);
                    }
                }
            }

    {number}  cout &lt;&lt; &quot;number &quot; &lt;&lt; YYText() &lt;&lt; '\n';

    \n        mylineno++;

    {name}    cout &lt;&lt; &quot;name &quot; &lt;&lt; YYText() &lt;&lt; '\n';

    {string}  cout &lt;&lt; &quot;string &quot; &lt;&lt; YYText() &lt;&lt; '\n';

    %%

	// This include is required if main() is an another source file.
	//#include &lt;FlexLexer.h&gt;

    int main( int /* argc */, char** /* argv */ )
    {
        FlexLexer* lexer = new yyFlexLexer;
        while(lexer-&gt;yylex() != 0)
            ;
        return 0;
    }
</pre></div>

<a name="index-C_002b_002b_002c-multiple-different-scanners"></a>
<p>If you want to create multiple (different) lexer classes, you use the
&lsquo;<samp>-P</samp>&rsquo; flag (or the <code>prefix=</code> option) to rename each
<code>yyFlexLexer</code> to some other &lsquo;<samp>xxFlexLexer</samp>&rsquo;.  You then can
include <samp>&lt;FlexLexer.h&gt;</samp> in your other sources once per lexer class,
first renaming <code>yyFlexLexer</code> as follows:
</p>
<a name="index-include-files_002c-with-C_002b_002b"></a>
<a name="index-header-files_002c-with-C_002b_002b"></a>
<a name="index-C_002b_002b-scanners_002c-including-multiple-scanners"></a>
<div class="example">
<pre class="verbatim">    #undef yyFlexLexer
    #define yyFlexLexer xxFlexLexer
    #include &lt;FlexLexer.h&gt;

    #undef yyFlexLexer
    #define yyFlexLexer zzFlexLexer
    #include &lt;FlexLexer.h&gt;
</pre></div>

<p>if, for example, you used <code>%option prefix=&quot;xx&quot;</code> for one of your
scanners and <code>%option prefix=&quot;zz&quot;</code> for the other.
</p>
<hr>
<div class="header">
<p>
Next: <a href="Reentrant.html#Reentrant" accesskey="n" rel="next">Reentrant</a>, Previous: <a href="Performance.html#Performance" accesskey="p" rel="prev">Performance</a>, Up: <a href="index.html#Top" accesskey="u" rel="up">Top</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Indices.html#Indices" title="Index" rel="index">Index</a>]</p>
</div>



</body>
</html>
flex-doc 2.6.4-6 / usr / share / doc / flex-doc / html / Cxx.html
