/usr/share/doc/flex-doc/html/Can-I-fake-multi_002dbyte-character-support

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- 
The flex manual is placed under the same licensing conditions as the
rest of flex:

Copyright (C) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2012
The Flex Project.

Copyright (C) 1990, 1997 The Regents of the University of California.
All rights reserved.

This code is derived from software contributed to Berkeley by
Vern Paxson.

The United States Government has rights in this work pursuant
to contract no. DE-AC03-76SF00098 between the United States
Department of Energy and the University of California.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

Neither the name of the University nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.

THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. -->
<!-- Created by GNU Texinfo 6.5, http://www.gnu.org/software/texinfo/ -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Can I fake multi-byte character support? (Lexical Analysis With Flex, for Flex 2.6.4)</title>

<meta name="description" content="Can I fake multi-byte character support? (Lexical Analysis With Flex, for Flex 2.6.4)">
<meta name="keywords" content="Can I fake multi-byte character support? (Lexical Analysis With Flex, for Flex 2.6.4)">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<link href="index.html#Top" rel="start" title="Top">
<link href="Indices.html#Indices" rel="index" title="Indices">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="FAQ.html#FAQ" rel="up" title="FAQ">
<link href="deleteme01.html#deleteme01" rel="next" title="deleteme01">
<link href="Is-backing-up-a-big-deal_003f.html#Is-backing-up-a-big-deal_003f" rel="prev" title="Is backing up a big deal?">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.indentedblock {margin-right: 0em}
blockquote.smallindentedblock {margin-right: 0em; font-size: smaller}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smalllisp {margin-left: 3.2em}
kbd {font-style: oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nolinebreak {white-space: nowrap}
span.roman {font-family: initial; font-weight: normal}
span.sansserif {font-family: sans-serif; font-weight: normal}
ul.no-bullet {list-style: none}
-->
</style>


</head>

<body lang="en">
<a name="Can-I-fake-multi_002dbyte-character-support_003f"></a>
<div class="header">
<p>
Next: <a href="deleteme01.html#deleteme01" accesskey="n" rel="next">deleteme01</a>, Previous: <a href="Is-backing-up-a-big-deal_003f.html#Is-backing-up-a-big-deal_003f" accesskey="p" rel="prev">Is backing up a big deal?</a>, Up: <a href="FAQ.html#FAQ" accesskey="u" rel="up">FAQ</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Indices.html#Indices" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="Can-I-fake-multi_002dbyte-character-support_003f-1"></a>
<h3 class="unnumberedsec">Can I fake multi-byte character support?</h3>
<div class="example">
<pre class="verbatim">To: Heeman_Lee@hp.com
Subject: Re: flex - multi-byte support?
In-reply-to: Your message of Thu, 03 Oct 1996 17:24:04 PDT.
Date: Fri, 04 Oct 1996 11:42:18 PDT
From: Vern Paxson &lt;vern&gt;

&gt;      I assume as long as my *.l file defines the
&gt;      range of expected character code values (in octal format), flex will
&gt;      scan the file and read multi-byte characters correctly. But I have no
&gt;      confidence in this assumption.

Your lack of confidence is justified - this won't work.

Flex has in it a widespread assumption that the input is processed
one byte at a time.  Fixing this is on the to-do list, but is involved,
so it won't happen any time soon.  In the interim, the best I can suggest
(unless you want to try fixing it yourself) is to write your rules in
terms of pairs of bytes, using definitions in the first section:

	X	\xfe\xc2
	...
	%%
	foo{X}bar	found_foo_fe_c2_bar();

etc.  Definitely a pain - sorry about that.

By the way, the email address you used for me is ancient, indicating you
have a very old version of flex.  You can get the most recent, 2.5.4, from
ftp.ee.lbl.gov.

		Vern
</pre></div>




</body>
</html>
flex-doc 2.6.4-6 / usr / share / doc / flex-doc / html / Can-I-fake-multi_002dbyte-character-support_003f.html
