The websocket stream is optimized to contain a small
circular static buffer, reducing the number of I/O calls when
reading data. The size of the buffer is tuned for maximum
performance with TCP/IP and no long needs configuration:
* read_some replaces read_frame
* write_some replaces write_Frame
* async_read_some replaces async_read_frame
* async_write_some replaces async_write_frame
* websocket::stream::read_buffer_size is removed
Actions Required:
* Remove calls websocket::stream::read_buffer_size
* Use read_some and write_some instead of read_frame and write_frame
* static_buffer is added
Note this is the same name from two versions ago, when
static_buffer was renamed to flat_static_buffer for
consistency and to clear the name for a circular static
buffer.
Actions Required:
* Callers who depend on static_buffer returning sequences of
exactly length one should switch to flat_static_buffer.
* parser now has a callback feature for intercepting chunk headers
and chunk bodies
* The names for basic_parser derived class callbacks have been
refactored
* basic_parser offers an additional callback for distinguishing
chunk body data.
Actions Required:
* Adjust signatures for required members of user-defined
subclasses of basic_parser
* Use the new basic_parser chunk callbacks for accessing
chunk extensions and chunk bodies.
New buffer sequence classes are provided to allow full
control over the serialization of chunk-encoded message
payloads:
* chunk_header
A ConstBufferSequence representing the chunk header.
It includes a hexadecimal-encoded size, an optional
set of chunk extensions, and the trailing CRLF
required to denote the end of the chunk header.
This allows the caller to manually serialize the chunk
body in one or more calls to a stream output function.
The caller must also output an object of type `chunk_crlf`
to the stream to indicate the end of the chunk body.
* chunk_crlf
A small ConstBufferSequence that simply represents
the two character sequence "\r\n" (CRLF). This is needed
for the case where the caller wants to output a chunk
body as a series of buffers (i.e. "chunking a chunk").
* chunk_body
A ConstBufferSequence representing a complete chunk.
This includes the size, an optional set of chunk extensions,
a caller provided buffer containing the body, and the
required CRLF that follows.
* chunk_final
A ConstBufferSequence representing a final chunk. It
includes an optional set of caller-provided field trailers
* chunk_extensions
A container for building a set of chunk extensions to use
during serialization. The use of the container is optional,
callers may provide their own buffer containing a correctly
formatted set of chunk extensions, or they may use their
own convenience container which meets the requirements.
The basic_fields container is modified to allow construction
outside the context of a message. The container can be used
to provide trailers to `chunk_final`.
Actions Required:
* Remove references to ChunkDecorators. Use the new chunk-encoding
buffer sequences to manually produce a chunked payload body in
the case where control over the chunk-extensions and/or trailers
is required.
These types now perform error-based initialization in
a separate init() functions instead of in the constructor.
Actions Required:
* Modify instances of user-defined BodyReader and BodyWriter
types to perfrom two-phase initialization, as per the
updated documented type requirements.
fix#581
* request and response headers now have convenience
constructors so important fields like method, target,
result, and version may be set upon construction.
Actions Required:
* Evaluate each message constructor call site and
adjust the constructor argument list as needed.
fix#185, fix#489
* SSE4.2 is detected
* basic_parser uses SSE4.2 if available
* basic_parser tries to parse on the initial buffer and if
it does not find the end of header it shifts to a new
strategy of waiting for the end of header to defeat
slow loris attacks. This coincidentally is also faster
than the previous algorithm.
`put` returns the number of bytes actually transferred
from the input buffers.
Actions Required:
* Return the number of bytes actually transferred from the
input buffers in user defined `Body::writer::put` functions.
fix#550
* default body limits are now 1MB/8MB
* default header limit is 8KB
Actions Required:
* Call body_limit and/or header_limit as needed to adjust the
limits to suitable values if the defaults are insufficient.
The is_deferred nested type is removed from the BodyReader
requirements.
Performance for sending messages with `file_body` was cut
almost in half with is_deferred as `std::true_type` since
it caused double the number of socket writes. If the
deferred behavior is absolutely necessary, callers can get
the same effect by manually sending the headers first.
Actions Required:
* Callers who need the behavior of is_deferred as `std::true_type`
should manually construct a serializer and serialize the header
first, followed by the body.
fix#322
The new callback is informed of all frame types including close.
Actions Required:
* Change calls to websocket::stream::ping_callback to use
websocket::stream::control_callback
* Change user defined ping callbacks to have the new
signature and adjust the callback definition appropriately.
Actions Required:
* Change calls to msg.prepare to msg.prepare_payload. For messages
with a user-defined Fields, provide the function prepare_payload_impl
in the fields type according to the Fields requirements.
* FieldsReader now requires chunked() and keep_alive()
* serializer logic calls into the reader to calculate
the message metadata
* Removed some of the previous requirements of FieldsReader
Actions Required:
* Implement chunked() and keep_alive() for user defined FieldsReader types.
* `parser` is now templated on Allocator
* `parser` only produces messages using `basic_fields<Allocator>`
as the Fields type.
* message-oriented read and async_read only work on messages
using basic_fields as the Fields type.
Actions Required:
* Callers using `parser` with Fields types other than basic_fields
will need to create their own subclass of basic_parser to work
with their custom fields type.
* detail::read_size_helper is removed
* public function read_size helps determine the optimum
size for calls to a DynamicBuffer's prepare function.
* special case for Asio types which don't conform to
the concept but have their own helper function.
* Container interface more closely matches std::vector
* While preserving the invariant that duplicate fields
with the same case-insensitive name have their order
preserved.
read_size_helper can return zero if the buffer reaches
its maximum size, causing infinite loops in HTTP. The
function maybe_read_size_helper is provided to throw
an exception instead of returning a value for this case.
* frame_info struct is removed
* read_frame and async_read_frame return a bool indicating
if the frame is the last frame of the current message.
Actions Required:
* Remove the frame_info parameter from all read frame call sites
* Check the return value 'fin' for calls to read_frame
* Change ReadHandlers passed to async_read_frame to have
the signature void(error_code, bool fin), use the bool
to indicate if the frame is the last frame.
fix#446
* Read signatures no longer include `opcode`
* stream::got_binary and stream::got_text inform the caller if
the current incoming message is binary or text.
Actions Required:
* Remove the `opcode` reference parameter from calls to synchronous
and asynchronous read functions, replace the logic with calls to
stream::got_binary and stream::got_text instead.
fix#446
* The ping_callback option struct is removed.
Actions Required:
* Change call sites which use ping_callback with set_option to
call stream::ping_callback instead.
fix#446
* The write_buffer_size option struct is removed.
Actions Required:
* Change call sites which use write_buffer_size with set_option to
call stream::write_buffer_size instead.
fix#446
* The read_message_max option struct is removed.
Actions Required:
* Change call sites which use read_message_max with set_option to
call stream::read_message_max instead.
fix#446
* The read_buffer_size option struct is removed
Actions Required:
* Change call sites which use read_buffer_size with set_option to
call stream::read_buffer_size instead.
fix#446
* message_type is removed
Actions Required:
* Change call sites which use message_type with set_option
to call stream::binary or stream::text instead.
fix#374, fix#446
* auto_fragment option struct is removed
Actions Required:
* Change call sites which use auto_fragment with set_option
to call stream::auto_fragment instead.
fix#440
WARNING
multi_buffer constructor now takes a maximum size instead of the
"allocation size". This is a breaking change. The allocation size
feature is removed. To update calling code, remove the allocation
size parameter from call sites.
* multi_buffer now uses a geometric growth algorithm for better performance.
The static buffer is updated:
* reset() is no longer a member. Use b.consume(b.size()) instead.
* Simplified implementaton, uses asio instead of custom types
* Better stream performance: consuming the input makes room
available in the output. This class is now suitable for
HTTP reads.
These changes permit the static_buffer wrapper to adapt a user
memory buffer if desired, including a stack based array.
The static_buffer_n class may also be used for this purpose,
it comes with its own storage.
* header::result is a family of functions to replace header::status
* header interface now uses status enum and also ints
* reason-phrase is no longer stored unless the user explicitly
requests it.
* When serializing, the standard reason is used for the
corresponding status code unless the user has changed it.
The verb interfaces now use verb::unknown instead of
boost::optional<verb> == boost::none to indicate that
the request-method is an unrecognized string.
The http::header interface is modified to focus more on the
verb enum rather than the string. For recognized verbs, the
implementation stores an integer instead of the string.
Unknown verbs are still stored as strings.
* header::method now returns a verb
* header::method_string returns the method text
fix#404
Parsing and serialization interfaces have been fine tuned and unified.
For parsing these stream operations are defined:
* read
* read_header
* read_some
* async_read
* async_read_header
* async_read_some
For serialization these stream operations are defined:
* write
* write_header
* write_some
* async_write
* async_write_header
* async_write_some
fix#398
A new enum status is added for the status code.
The function obsolete_reason returns default reason phrasing.
If a response has an empty reason, the serializer will
automatically insert the default reason phrase for the
status code.
fix#397
method enum class is added to represent all known request methods.
Functions are provided to convert from strings to and from the method
enumeration.
The fields container is modified to also work with the enum.
serializer interface is changed to be buffer-only, no streams,
and placed in its own header file.
Operations on serializers are moved to free functions as part
of the HTTP write family of synchronous and asynchronous algorithms.
A new class `serializer` is introduced to permit incremental
serialization of HTTP messages. Existing free functions are
re-implemented in terms of this new class.
* The BodyReader concept is refined to support a greater variety
of strategies for providing buffers representing the body to
the serialization algorithms.
* Added buffer_body, a new model of Body which allows the caller
to provide a series of owned buffers using their own serialization
loop.
* Added empty_body, a model of Body which is for serialization only,
to represent HTTP messages with no content body.
* Removed overloads of write and async_write which send only
the HTTP header.
* Removed public interfaces for performing low-level chunk encoding.
The concept type traits are renamed for consistency,
and consolidated into a single header file <beast/core/type_traits.hpp>
A new section, Core Concepts, is added to the documentation describing all
of the core utility classes and functions. This also includes a complete
explanation and sample program describing how to write asynchronous initiation
functions and their associated composed operations.
This function converts integers to their decimal
representation as a static string.
In addition, static_string::resize no longer initializes
characters if the new size is larger.
A new function, buffers(), returns an implementation defined object
which wraps a ConstBufferSequence and supports formatting to a
std::ostream.
The function to_string is removed, as the new implementation allows
conversion to string using boost::lexical_cast on the return value
of the call to buffers(). Streaming to an output stream is more
efficient: no dynamic allocations are performed.
Example:
streambuf sb;
std::cout << buffers(sb.data()) << std::endl;
This eliminates beast::write output for dynamic buffers and replaces
it with the function ostream() that wraps a caller provided dynamic
buffer and returns the result as a std::ostream derived object.
Callers may now produce formatted output into any object meeting the
requirements of DynamicBuffer using operator<< and the standard stream
modifiers such as std::endl.
This new technique is more efficient, as implementations of operator<<
can now write directly into the output using std::ostream::write and
std::ostream::put.
Example of use:
beast::streambuf sb;
beast::ostream(sb) << "Hello, world!" << std::endl;
fix#124
The http::header data members "method", "url", and "reason"
are changed from data members, to pairs of get and set
functions which forward the call to the Fields type used
to instantiate the template.
Previously, these data members were implemented using
std::string. With this change, the implementation of the
Fields type used to instantiate the template is now in
control of the representation of those values. This permits
custom memory allocation strategies including uniform use of
the Allocator type already provided to beast::http::basic_fields.
fix#332
This removes the keep_alive option from the WebSocket stream.
Callers who wish to control the behavior of the Connection
header may do so in the decorator and completion handlers
for the handshake and accept functions.
fix#325
This removes unused and misleading handshake error codes.
All semantic handshake failures are now reported using
the same code, error::handshake_failed. Errors originating
from stream operations still use the underlying stream
error codes (for example: boost::asio::error::connection_reset).
fix#195
This function returns `true` when the passed HTTP Request
indicates a WebSocket Upgrade. It does not validate the
contents of the fields: it just trivially accepts requests
which can only be a WebSocket Upgrade message.
Callers who wish to manually read HTTP requests in their
server implementation can use this function to determine if
the request should be routed to an instance of websocket::stream.
fix#80, #212, fix#303, fix#314, fix#317
websocket::stream now provides the following families of
functions for performing handshakes:
When operating in the server role:
* stream::accept
* stream::accept_ex
* stream::async_accept
* stream::async_accept_ex
When operating in the client role:
* stream::handshake
* stream::handshake_ex
* stream::async_handshake
* stream::async_handshake_ex
Member functions ending with "_ex" allow an additional
RequestDecorator parameter (for the accept family of
functions) or ResponseDecorator parameter (for the
handshake family of functions).
The decorator is called to optionally modify the contents
of the HTTP request or HTTP response object generated by
the implementation, before the message is sent. This
permits callers to set the User-Agent or Server fields,
add or modify HTTP fields related to subprotocols, or
perform any required transformation of the HTTP message
for application-specific needs.
The handshake() family of functions now have an additional
set of overloads accepting a parameter of type response_type&,
allowing the caller to receive the HTTP Response to the
Upgrade handshake. This permits inspection of the response
to handle things like subprotocols, authentication, or
other application-specific needs.
The new implementation does not require any state to be
stored in the stream object. Therefore, websocket::stream
objects are now smaller in size.
The overload of set_option for setting a decorator on the
stream is removed. The only way to set decorators now is
with a suitable overload of accept or handshake.
This adds enough functions to the accept and async_accept
overload set to cover all combinations of request, buffers,
and error_code parameters. This also fixes a defect where
providing a complete Upgrade request when there are additional
unprocessed octets remaining in the callers stream buffer
could not be furnished to the WebSocket implementation.
fix#123fix#154fix#265
This completely replaces the HTTP parser used to read and
parse incoming messages. The new version is faster than
the old one, and written in fewer lines. A summary of
changes:
* parse and async_parse renamed to read and async_read
* basic_parser is optimized for linear buffers,
use flat_streambuf for the best performance with these
functions:
- http::read
- http::read_some
- http::async_read
- http::async_read_some
* The overloads of read() and async_read() which take
just a header have been removed, since they would
throw away important parse metadata.
* The derived class callbacks for basic_parser have
been streamlined. All strings passed to callbacks
are presented in their entirety, instead of being
provided in pieces.
These changes allow use-cases that were previously
difficult or impossible, such as:
- Efficient relaying
- Late body type commitment
- Expect: 100-continue handling
Objects of this type meet the requirements of DynamicBuffer
and offer an additional invariant: buffer sequences returned
by data() and prepare() are always of length one.
fix#257
This fixes a problem when doing split parsing (header then body).
The problem arises when the first buffer passed to the header
parser contains exactly the full header and nothing more. The
symptom is that when parsing the body, the parse will erroneously
be considered complete.
* Add handler_ptr test and increase coverage
* Add test for prepare_buffer
* Move is_call_possible tests to a .cpp file
* Tidy up docs and declarations
fix#248
This additionally invokes the pong callback for received pings, allowing
callers to more efficiently detect when a connection is still lively:
* pong_callback renamed to ping_callback
* Callback signature has an extra `bool` to indicate if the received
control frame is a ping or pong.
fix#215
This change guarantees that temporary memory allocated
through the asio hooks by the Beast implementation is
deallocated before invoking the final handler when performing
composed operations.
The change is accomplished by replacing std::shared_ptr with
a thread-safe custom container handler_ptr to manage composed
operation state. The container tracks other instances which
manage the same object and resets them in a safe way before
invoking the final handler.
handler_ptr is provided as a public interface so that users of
this library can utilize the same idiom to write their own
composed operations.
fix#171
Several names and HTTP identifiers are renamed to be
more consistent, self-explanatory, and concise:
* "Fields" is a collection of HTTP header fields (rfc7230 section 3.2)
* "Header" is the Start Line plus Fields. Another way to look at it is,
the HTTP message minus the body.
* `basic_fields` replaces `basic_headers`
* `fields` replaces `headers`
* `Fields` replaces `Headers` in template parameter lists
* `header` replaces `message_headers`
* `header::fields` replaces `message_headers::fields`
The changes are cosmetic and do not affect run-time behavior.
buffer_cat now determines if all of the buffer sequences in
the list of passed buffer sequences each have value types which
are convertible to mutable_buffer. The returned concatenated
sequence will be a MutableBufferSequence if all the passed
buffer sequences meet the requirements of MutableBufferSequence,
else the returned concatenated sequence will be a ConstBufferSequence.
This fixes a bug where instantiations of consuming_buffers with
buffer sequence types whose value_type is not const_buffer or
mutable_buffer can cause compilation errors.
The function consumed_buffers is removed.
fix#154, fix#156
This adds public interfaces for transforming buffer
sequences into their chunk-encoded equivalents. The
transformations are O(1) in space and time.
message_headers is now a set of partial class template
specializations instead of a template class alias. This solves
a problem where template functions taking message_headers as a
parameter could not deduce argument types, since std::conditional
obscured the deduced context.
Both classes are refactored to share declarations using an #ifdef,
to eliminate an ugly set of extra declarations needed when building
the documentation.
Copy and move class special members are added.
A new function message::base() is provided which returns the
message_headers portion of a message.
fix#127
* Added beast::detail::ignore_unused based on boost::ignore_unused
* Added -Wextra compilation flag when building with gcc
* Fixed all unused parameter warnings with ignore_unused
* Fixed all missing includes when building each .hpp separately
These changes support parsing the headers separately from the body.
* on_headers now returns void
* on_body_what is a new required callback which returns body_what
fix#114, fix#117, fix#136
* Added init() to Reader requirements
* Reader must be nothrow constructible
* Reader is now constructed right before reading the body
- The message passed on construction is filled in
This patch rectifies flush() of beast::http::parser_v1
to properly handle the case when an HTTP header has
empty value.
Without the fix an empty-valued HTTP header is being
concatenated with the header directly following it.
This situation can be replicated using eg. curl:
curl <url> -H "X-First;" -H "X-Second: bla"
What Beast's client would see is a single header named
as "X-FirstX-Second".
New overloads of suite::expect take the file and line number
as individual parameters, cleaning up the file name output
by showing only the filename part (to not leak the full path,
which might contain sensitive information).
A new macro BEAST_EXPECTS allows an additional reason
string as well as reporting the file and line. Typical usage:
```
error_code ec;
...
if(! BEAST_EXPECTS(! ec, ec.message()))
return;
```
basic_headers no longer combines fields with the same name by appending
a comma and concatenating the two values together. This was breaking
certain header fields which expect each value to be distinct, such as
the "Set-Cookie" header.
Now the container behaves more like a multi set with respect to insertion
of multiple values with the same field name. Additional member functions
are provided to provide extra functionality.
in addition to passing ${CMAKE_THREAD_LIBS_INIT} to the linker, this
interface library will also add -pthread to the compile options when
supported
Signed-off-by: Casey Bodley <cbodley@redhat.com>
New parser set_option function for controlling independent size limits
on headers and body. By default request and response parsers are set up
with reasonable limits to prevent resource exhaustion attacks.
* Parser adheres strictly to rfc7230
* Increased test coverage
* Headers and body maximum size limit options
Conform to the Networking TS by renaming the Streambuf concept
to DynamicBuffer in all places. Values of types meeting the
requirements of DynamicBuffer are renamed to dynabuf.
See:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4478.html#requirements.dynamic_buffers
* Headers renamed
* Formal parameter names renamed
* Template argument types renamed
* Documentation updated
ext_list:
Iterable container of comma separated extensions, where each extension
is a token followed an optional list of semicolon delimited parameters,
with each parameter consisting of a name / value pair. The value can
be a token or quoted-string.
param_list:
Iterable container of semicolon delimited parameters, where each parameter
is a name / value pair. The value can be a token or quoted-string.
token_list
Iterable container of comma delimited tokens.
* Remove obsolete rfc2616 functions
* Refactor and consolidate case-insensitive string helpers
* Improve test coverage
* tests for invokable in composed ops
* Update documentation
* Add License badge to README
* Target Windows 7 SDK and later
* Make role_type private
* Remove extra unused masking functions
* Allow stream reuse / reconnect after failure
* Restructure logic of composed operations
* Allow 0 for read_message_max meaning no limit
* Respect keep alive when building HTTP responses
* Check version in upgrade request
* Response with 426 status on unsupported WebSocket version
* Remove unnecessary Sec-WebSocket-Key in HTTP responses
* Rename to mask_buffer_size
* Remove maybe_throw
* Add ping, async_ping, async_on_pong
* Add ping_op
* Add pong_op
* Fix crash in accept_op
* Fix suspend in close_op
* Fix read_frame_op logic
* Fix crash in read_op
* Fix races in echo sync and async echo servers
This fixed a bug where in some cases the parser could dereference
past the end of the caller provided buffer. The unit test is
improved to allocate memory in separate pieces for the split-buffer
matrix test, to give address-sanitizer more to work with.
The log member is changed to derive from std::ostream. A new
class dstream is derived from std::ostream to support redirection
to the Visual Studio Output Window if a debugger is attached.
Obsolete classes abstract_ostream and its derived variants are
removed.
A new concept Parser is introduced with routines to read from a stream
into the parser. This solves a problem with the old read interface where
messages must be default constructible and move assignable.
Parser fixes:
* Fix detect invalid reason-phrase octets
* Fix write_eof to set the 'complete' state on success
* Fix consider parse complete if eof received on empty body
WebSocket:
* Increase coverage
Core:
* Test buffer_cat iterator move members
HTTP:
* Fixed yield / resume in writer
* Fixed message serialization with chunked encoding
* Test yield / resume in writer
* Test all conditional branches during message serialization
* Test chunked encoding
* Increase coverage on parse_error
* Add parse_error::general
WebSocket:
* Add error::general
* Increase coverage in error
The message class now behaves like a pair with respect to the construction
of the body and headers. Additional constructors allow construction of
just the body portion from a tuple, leaving the headers default
constructed.
Previous constructors are removed as they were a notational convenience
for assembling HTTP/1 requests and responses. They are not necessary
as this library aims at library writers and not end users.
* Don't include the test code in coverage reports
* Add test code for missing coverage
Other:
* Improve the README.md
* Fix warning in sha1_context
* Tidy up the examples use of namespaces
* Various fixes to documentation and javadocs
The version field is moved into message_v1, all public interfaces
are reworked to identify HTTP/1 wire format operations (suffix "_v1")
versus general HTTP.
websocket:
* Move echo server to test/
* Fix warnings
* Fix maskgen being uncopyable
* Simplify utf8_checker special member declarations
* Fix stream move assignable when owning the next layer
* Add javadocs for stream special members
* Add stream unit tests
* Move throwing member definitions to the .ipp file
* Use get_lowest_layer in stream declaration
* Perform type checks at each call site instead of constructor
* Demote close_code to a non-class enum:
Otherwise, application specific close codes
cannot be assigned without using static_cast.
core:
* Add streambuf_readstream special members tests
* Add move assignment operator to streambuf_readstream
* Add detail/get_lowest_layer trait
* Add static_string tests
* Move static_string from websocket to core
* Fix warnings
* Port cmake scripts to linux
* Add command line options for running test suites
* Add examples to CMakeLists
* Return std::uint64_t from writer::content_length
* basic_parser::write takes asio::const_buffer instead of pointer and size
* Turn message test back on now that it passes
* Rename to http::headers, use std::allocator, remove http_headers
* http::message::method is now a string
* Refactor to_string for ConstBufferSequence
* Remove chunk_encode from the public interface
* Initialize members for default constructed iterators
* Disallow default construction for dependent buffer sequences
Refactor http::message serialization:
* Serialization no longer creates a copy of the
headers and modifies them
* New function prepare(), sets Connection, Transfer-Encoding,
Content-Length based on the body attributes and caller options.
Callers can use prepare() to have the fields set automatically,
or they can set the fields manually.
* Use write for operator<<
* Tests for serialization
