157 lines
5.1 KiB
HTML
157 lines
5.1 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Language" content="en-us">
|
|
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
|
|
<meta name="GENERATOR" content="Microsoft FrontPage 6.0">
|
|
<meta name="ProgId" content="FrontPage.Editor.Document">
|
|
|
|
<title>Boost Char Delimiters Separator</title>
|
|
</head>
|
|
|
|
<body bgcolor="#FFFFFF" text="#000000" link="#0000EE" vlink="#551A8B" alink=
|
|
"#FF0000">
|
|
<p><img src="../../../boost.png" alt="C++ Boost" width="277" height=
|
|
"86"><br></p><font color="red">Note: This class is deprecated. Please use
|
|
<a href="char_separator.htm"><tt>char_separator</tt></a> instead.</font>
|
|
|
|
<h1 align="center">Char Delimiters Separator</h1>
|
|
<pre>
|
|
template <class Char, class Traits = std::char_traits<Char> >
|
|
class char_delimiters_separator{
|
|
</pre>
|
|
|
|
<p>The char_delimiters_separator class is an implementation of the <a href=
|
|
"tokenizerfunction.htm">TokenizerFunction</a> concept that can be used to
|
|
break text up into tokens. It is the default TokenizerFunction for
|
|
tokenizer and token_iterator_generator. An example is below.</p>
|
|
|
|
<h2>Example</h2>
|
|
<pre>
|
|
// simple_example_4.cpp
|
|
#include<iostream>
|
|
#include<boost/tokenizer.hpp>
|
|
#include<string>
|
|
|
|
int main(){
|
|
using namespace std;
|
|
using namespace boost;
|
|
string s = "This is, a test";
|
|
tokenizer<char_delimiters_separator<char> > tok(s);
|
|
for(tokenizer<char_delimiters_separator<char> >::iterator beg=tok.begin(); beg!=tok.end();++beg){
|
|
cout << *beg << "\n";
|
|
}
|
|
}
|
|
</pre>
|
|
|
|
<h2>Construction and Usage</h2>
|
|
|
|
<p>There is one constructor of interest. It is as follows</p>
|
|
<pre>
|
|
explicit char_delimiters_separator(bool return_delims = false,
|
|
const Char* returnable = "",const Char* nonreturnable = "" )
|
|
</pre>
|
|
|
|
<table border="1" summary="">
|
|
<tr>
|
|
<td>
|
|
<p align="center"><strong>Parameter</strong></p>
|
|
</td>
|
|
|
|
<td>
|
|
<p align="center"><strong>Description</strong></p>
|
|
</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>return_delims</td>
|
|
|
|
<td>Whether or not to return the delimiters that have been found. Note
|
|
that not all delimiters can be returned. See the other two parameters
|
|
for explanation.</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>returnable</td>
|
|
|
|
<td>This specifies the returnable delimiters. These are the delimiters
|
|
that can be returned as tokens when return_delims is true. Since these
|
|
are typically punctuation, if a 0 is provided as the argument, then the
|
|
returnable delmiters will be all characters Cfor which std::ispunct(C)
|
|
yields a true value. If an argument of "" is provided, then this is
|
|
taken to mean that there are noreturnable delimiters.</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>nonreturnable</td>
|
|
|
|
<td>This specifies the nonreturnable delimiters. These are delimiters
|
|
that cannot be returned as tokens. Since these are typically
|
|
whitespace, if 0 is specified as an argument, then the nonreturnable
|
|
delimiters will be all characters C for which std::isspace(C) yields a
|
|
true value. If an argument of "" is provided, then this is taken to
|
|
mean that there are no non-returnable delimiters.</td>
|
|
</tr>
|
|
</table>
|
|
|
|
<p>The reason there is a distinction between nonreturnable and returnable
|
|
delimiters is that some delimiters are just used to split up tokens and are
|
|
nothing more. Take for example the following string "b c +". Assume you are
|
|
writing a simple calculator to parse expression in post fix notation. While
|
|
both the space and the + separate tokens, you only only interested in the +
|
|
and not in the space. Indeed having the space returned as a token would
|
|
only complicate your code. In this case you would specify + as a
|
|
returnable, and space as a nonreturnable delimiter.</p>
|
|
|
|
<p>To use this class, pass an object of it anywhere a TokenizerFunction
|
|
object is required.</p>
|
|
|
|
<h3>Template Parameters</h3>
|
|
|
|
<table border="1" summary="">
|
|
<tr>
|
|
<th>Parameter</th>
|
|
|
|
<th>Description</th>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td><tt>Char</tt></td>
|
|
|
|
<td>The type of the elements within a token, typically
|
|
<tt>char</tt>.</td>
|
|
</tr>
|
|
|
|
<tr>
|
|
<td>Traits</td>
|
|
|
|
<td>The traits class for Char, typically
|
|
std::char_traits<Char></td>
|
|
</tr>
|
|
</table>
|
|
|
|
<h2>Model of</h2>
|
|
|
|
<p><a href="tokenizerfunction.htm">TokenizerFunction</a></p>
|
|
|
|
<p> </p>
|
|
<hr>
|
|
|
|
<p><a href="http://validator.w3.org/check?uri=referer"><img border="0" src=
|
|
"../../doc/images/valid-html401.png" alt="Valid HTML 4.01 Transitional"
|
|
height="31" width="88"></a></p>
|
|
|
|
<p>Revised
|
|
<!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->25
|
|
December, 2006<!--webbot bot="Timestamp" endspan i-checksum="38518" --></p>
|
|
|
|
<p><i>Copyright © 2001 John R. Bandela</i></p>
|
|
|
|
<p><i>Distributed under the Boost Software License, Version 1.0. (See
|
|
accompanying file <a href="../../LICENSE_1_0.txt">LICENSE_1_0.txt</a> or
|
|
copy at <a href=
|
|
"http://www.boost.org/LICENSE_1_0.txt">http://www.boost.org/LICENSE_1_0.txt</a>)</i></p>
|
|
</body>
|
|
</html>
|