C++23 std::string_view added a range constructor that is constrained with
a concept check that in particular checks if the range is contiguous. In
that check, the range iterator type is checked. Since fs::path members now
test whether the source argument is convertible to std::string_view, that
concept check is performed whenever the overload resolution or SFINAE check
is performed. This caused a problem if the check was performed before
fs::path::iterator is defined, since the result of the check formally
changes when the iterator gets defined.
To work around this, move any fs::path non-template member functions that
call to other members (including constructors) which may involve overload
resolution or SFINAE checks that might require testing whether
std::string_view is constructible from fs::path out of fs::path definition
and past the fs::path::iterator definition.
This was also reported to gcc team:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106808
Path traits and relevant path members were reworked to better support
wider range of types that are compatible with path constructors, assignment
and appending members. Added support for C++17 std::string_view,
boost::string_view and boost::container::string as the possible string
types accepted by path members.
Also extended support for types convertible to one of the string types.
Previously, user's type had to be convertible to a string with a character
type that matches the native path character type. Now all supported character
types are acceptable.
Additionally, restricted members accepting a pair of iterators to only accept
iterators whose value types are one of the supported path character types.
Lastly, path::compare and comparison operators now only accept path arguments,
relying on path conversion constructors to do the job of supporting various
source types. Also removed noexcept from compare as it is using lex_compare
and iterators internally and those can throw.
Closes https://github.com/boostorg/filesystem/issues/208.
Users are advised to use string types and iterators instead of containers
to construct/assign/append to paths.
In v4, the support for containers is removed.
This follows up the previous update for POSIX.
The new implementation of remove_all on Windows Vista and later uses
NtCreateFile internal function in order to open files relative to
a previously opened directory handle, similar to POSIX openat.
Furthermore, querying file status and removing the file is now also
done through file handles to avoid performing path resolutions.
Closes https://github.com/boostorg/filesystem/issues/224.
The previous implementation could still allow for following symlinks
while remove_all is running if a directory was replaced with a symlink
higher in the tree than remove_all is currently processing. This was
reported here:
https://github.com/boostorg/filesystem/issues/224#issuecomment-1183738097
The solution is to use POSIX.1-2008 *at APIs to prevent symlink resolution
higher in the directory tree while iterating over the subtree in remove_all.
This required updating the directory iterator construction interface so that
it is possible to pass the base directory fd and return fd of the directory
used by the iterator. This is done via platform-specific params that are
currently defined only for POSIX. Additionally, status, symlink_status and
remove were extended to accept the base directory fd as well.
Other systems, including Windows, remain vulnerable.
Related to https://github.com/boostorg/filesystem/issues/224.
Clang triggers -Wunused-function for get_dir_itr_imp_extra_data, which
may or may not be used depending on the target platform. Better to disable
the warning rather than add macro checks.
We have changed v4 path::extension() to not return the filename if
it starts with a dot, but we have not made path::replace_extension
version-dependent, which made v3 behave like v4. Fixing this,
and optimizing v4 path::replace_extension in the process.
directory_iterator implementation now explicitly opens a directory
handle and relies on GetFileInformationByHandleEx or NtQueryDirectoryFile
to query information of the files in the directory. With
GetFileInformationByHandleEx, there are at least three ways to request
the information, each supported on different Windows versions and
different filesystems and providing different sets of information.
We support all three (FILE_ID_BOTH_DIR_INFO, FILE_FULL_DIR_INFO and
FILE_ID_EXTD_DIR_INFO), and we fall back to older ones if the newer
ones are not supported. GetFileInformationByHandleEx is available
since Windows Vista.
NtQueryDirectoryFile is an NT API that provides similar information to
GetFileInformationByHandleEx, but exists since at least Windows XP,
but some sources state it has existed since NT 4.0. This API is now
used when GetFileInformationByHandleEx is not available and it replaces
FindFirstFileW/FindNextFileW/FindClose based implementation (which
supposedly uses NtQueryDirectoryFile internally). Being an NT API,
we have to handle NTSTATUS error codes returned from it, so we had to
refactor unique path code that also used NTSTATUS previously.
Opening a directory handle explicitly allows to support O_NOFOLLOW
semantics on Windows, as we are able to prevent CreateFile from
following to the reparse point target. This is needed for remove_all
implementation to fix CVE-2022-21658. Similar to POSIX, remove_all_impl
on Windows now requests the directory iterator to not follow symlinks
when it is used to dive into a directory.
FindFirstFileW/FindNextFileW/FindClose based implementation is still
preserved for now for Windows CE, as it is unlikely to support neither
GetFileInformationByHandleEx nor NtQueryDirectoryFile. Given that
Windows CE has been untested for many years and is probably broken
anyway, its support is now declared deprecated. The related code,
including Find*-based directory iterator, will be removed in a
future release.
Closes https://github.com/boostorg/filesystem/issues/224.
And a couple cleanup changes:
- Since we now may use GetFileInformationByHandleEx, we are now using
it to query FILE_ATTRIBUTE_TAG_INFO for testing whether a reparse
point is a symlink.
- In resize_file_impl, we now specify all sharing flags to allow the
operation to succeed if the file is already opened by someone.
Turns out string_file.hpp was tested in operations_unit_test, though not
included directly. Since string_file.hpp contents are disabled when the
test is compiled (as any other deprecated functionality), the test failed
to compile.
Moved the string_file.hpp test to deprecated_test, which explicitly enables
deprecated functionality. Also, include string_file.hpp in filesystem.hpp
if explicitly requested by defining BOOST_FILESYSTEM_DEPRECATED.
Marked codecvt error category constructor constexpr and defined
it unconditionally to work around clang bug in XCode 8: the compiler
requires a user-defined constructor to initialize the static const object
of the category.
When constexpr is not available (more precisely, when it is not enabled
by Boost.System), try to dynamic-initialize the category instance early.
For MSVC, invoke codecvt_error_category() from the early global initialization
routine in path.cpp. For other compilers, use a global initializer, possibly
augmented with init_priority attribute.
Closes https://github.com/boostorg/filesystem/issues/227.
Another process could replace the directory being processed by remove_all
with a symlink after remove_all called symlink_status but before
it creates a directory iterator. As a result, remove_all would remove
the linked directory contents instead of removing the symlink.
On POSIX systems that support fdopendir and O_NOFOLLOW flag for open(2),
this can be prevented by opening the directory with O_NOFOLLOW before
iterating. This will fail if the directory was replaced with a symlink.
No protection on other systems.
Reported in https://github.com/boostorg/filesystem/issues/224.
Reworked remove() operation to separate POSIX and Windows implementations.
On Windows, if the file to be removed is read-only, try to reset the read-only
attribute before deleting the file. If deleting fails (other than because the
file is already deleted), try to restore the read-only attribute.
As a side effect, we were able to remove an implementation detail value from
the file_type enum that was used by the old remove() implementation.
Added a test for remove() on a read-only file on Windows. Also added tests
for remove_all(), including for cases with symlinks, hardlinks and read-only
files.
Also, corrected mklink /J argument in tests. The command accepts /j (lowercase)
to the same effect, but the formal help lists /J (uppercase) to create junctions.
Reported in https://github.com/boostorg/filesystem/issues/216.
In v4 path::lexically_normal, don't generate a trailing dot element if the
original path ends with a directory separator or dot. Also omit the trailing
directory separator the normalized path ends with a dot-dot element.
Additionally, convert directory separators to preferred separators in
root name on Windows (in v3 and v4). This may be significant for UNC paths.
If the source path ends with a non-empty filename, and the appended path
is empty, C++17 std::filesystem requires to add a trailing directory
separator.
In Boost.Filesystem v3 path appending mostly worked as a slight upgrade
of concatenation, where appending would only add directory separators
when necessary, but not consider semantics of the root name and root
directory of the appended paths. This would work well for relative paths,
but produce unexpected results for paths with root names.
In v4, we now implement appending that is aware of root name and directory
of the appendedn paths. This means that appending a path with a root name
and/or directory no longer concatenates the paths, but rather rebases the
appended path on top of the source path. In particular, if the appended path
has a root name different from the source path, the append operation will
act as assignment.
This is closer to C++20 std::filesystem but not exactly the same. The
difference is for the case when the appended path is absolute. The C++20
spec requires assignment in this case, Boost.Filesystem v4 deliberately
omits this check. This is to ensure the correct result for UNC paths on
POSIX systems, where "//net/foo" / "/bar" is expected to produce "//net/bar",
not "/bar".
As part of this work, refactored path constructors and operators for more
optimal implementation and reducing the number of overloads.
Closes https://github.com/boostorg/filesystem/issues/214.
Apparently, gcc wants the friend declaration of lex_compare_v3 to be annotated
with BOOST_FILESYSTEM_DECL, same as the previous declaration in the namespace
scope, and emits a -Wattributes warning otherwise.
Closes https://github.com/boostorg/filesystem/issues/213.
When the path ends with a non-root directory separator, no longer
produce a trailing dot element (filename). Instead, return an empty
path.
This affects not only path iterators and path::filename, but also any
other APIs that rely on them.
Closes https://github.com/boostorg/filesystem/issues/193.
Boost.Filesystem v4 will contain breaking changes from v3 that are required
for better compatibility with C++17 std::filesystem. It will also remove
the deprecated features of v3.
Updated docs to reflect the differences between v3 and v4. Updated tests
to verify both v3 and v4 where the differences are present.
This doesn't compile with gcc 8 on MinGW-w64, and fails in runtime with
gcc 10.2 and clang 8.0.1 on Cygwin64 because character code conversion errors,
so basically std::filesystem never works with wide paths on Windows.
We still use wide paths as `const wchar_t*` with libc++ though.
Also, changed BOOST_FILESYSTEM_C_STR definition to accept the path as
an argument and use that definition in the tests rather than duplicating it.
Related to https://github.com/boostorg/filesystem/issues/181.
libstdc++ starting 8.1.0 in C++17 mode support std::filesystem and is able to
open the file streams with std::filesystem::path constructible from wide
strings.
libc++ starting 7.0 also supports std::filesystem::path in C++17 mode and may
also support passing `const wchar_t*` strings to open files.
Closes https://github.com/boostorg/filesystem/issues/181.
- Unified root name and root directory parsing that was scattered and
duplicated across different algorithms. The new implementation is
consolidated in a single function for parsing root name and root
directory, which is used from various algorithms.
- The new root name parsing now supports Windows local device ("\\.\")
and NT path ("\??\") prefixes. It also adds support for filesystem
("\\?\") prefix to some of the higher level algorithms that were
using custom parsing previously. Tests updated to verify these prefixes.
- Some of the path decomposition methods were unified with presence checking
methods (e.g. root_name with has_root_name). This makes these methods
work consistently and also makes the has_* methods less expensive as
they no longer have to construct a path only to check if it is empty.
- The filename accessor no longer returns root name if the whole path
only consists of a root name. This also affects stem and extension as
those accessors are based on filename. This is a breaking change.
- Cleaned up code:
- Removed redundant checks for std::wstring support.
- Added header/footer headers to globally disable compiler warnings.
- Removed commented out super-deprecated code.
- Added missing includes and removed includes that are not needed.
- Nonessential code formatting.
Avoid creating root directory path only to test if it's empty. Avoid
calling root_directory multiple times in root_path.
Also, nonessential code reformatting.
These options allow to synchronize the copied data and attributes with
the permanent storage. Note that by default on POSIX systems copy_file
used to synchronize data in previous releases, and this commit changes
this. The caller now has to explicitly request syncing, as it has
significant performance implications.
Closes https://github.com/boostorg/filesystem/issues/186.
This makes a difference in case if Boost.Filesystem is linked statically into
user's application. In this case filesystem_error used to be not marked as
publicly visible, which could make it impossible to catch the exception if
it crossed shared library boundary. By marking the class as publicly visible
we ensure that RTTI is always visible, even in static builds.
The exported members are also marked with dllexport/dllimport attributes
for compatibility with Windows.
