For some system files and folders like "System Volume Information"
CreateFileW fails with ERROR_ACCESS_DENIED while GetFileAttributesW
succeeds. GetFileAttributesW doesn't allow to discover whether the
file is a symlink or some other kind of a reparse point, so this
fallback will only work for files that are not reparse points,
symlinks or not. For reparse points continue to report error.
Closes https://github.com/boostorg/filesystem/issues/234.
Even though the syscall number is defined in kernel headers, the syscall
is blacklisted by seccomp in runtime.
Note that Android 11 also introduces the libc wrapper for statx, so
effectively the library will always use the libc wrapper on Android.
Reported in https://github.com/boostorg/filesystem/issues/229.
Windows 10 1709 and later support POSIX semantics for removing files,
which means the file name is removed from the filesystem namespace as
soon as the file is marked for deletion. This makes opening the file
afterwards impossible, and allows creating a new file with the same
name, even if the deleted file is still open and in use.
The implementation uses runtime detection of the feature in the OS.
We are also using two more implementations for file removal: one that
employs the more recent FILE_DISPOSITION_FLAG_IGNORE_READONLY_ATTRIBUTE
flag (available since Windows 10 1809), and FILE_DISPOSITION_INFO
structure (supported since Windows Vista). The former allows to optimize
removal of read-only files, and the latter allows to make file deletion
atomic (i.e. not prone to failure if the file is replaced on the filesystem
while the operation is executing). The implementation is chosen in
runtime, depending on which one succeeds removing a file.
Also, added support for deleting read-only directories, in addition
to non-directory files, and simplified code a little.
Closes https://github.com/boostorg/filesystem/issues/216.
Using handles allows to reduce the number of system calls and avoids requesting
reparse point info, if the file is one. This should improve performance.
We have changed v4 path::extension() to not return the filename if
it starts with a dot, but we have not made path::replace_extension
version-dependent, which made v3 behave like v4. Fixing this,
and optimizing v4 path::replace_extension in the process.
Also, make a more robust check whether the handle refers to a symlink
in case if GetFileInformationByHandleEx is not available. If it is
not a symlink, but some other type of a reparse point, continue
processing it as if it is a regular directory.
Also, in remove_all_impl, check whether creating a directory iterator
fails due to the file not being a directory. Interpret this the same
way as with ELOOP - the error indicates that the directory was replaced
with some other kind of file between querying its type and creating
the directory iterator. Retry the operation.
directory_iterator implementation now explicitly opens a directory
handle and relies on GetFileInformationByHandleEx or NtQueryDirectoryFile
to query information of the files in the directory. With
GetFileInformationByHandleEx, there are at least three ways to request
the information, each supported on different Windows versions and
different filesystems and providing different sets of information.
We support all three (FILE_ID_BOTH_DIR_INFO, FILE_FULL_DIR_INFO and
FILE_ID_EXTD_DIR_INFO), and we fall back to older ones if the newer
ones are not supported. GetFileInformationByHandleEx is available
since Windows Vista.
NtQueryDirectoryFile is an NT API that provides similar information to
GetFileInformationByHandleEx, but exists since at least Windows XP,
but some sources state it has existed since NT 4.0. This API is now
used when GetFileInformationByHandleEx is not available and it replaces
FindFirstFileW/FindNextFileW/FindClose based implementation (which
supposedly uses NtQueryDirectoryFile internally). Being an NT API,
we have to handle NTSTATUS error codes returned from it, so we had to
refactor unique path code that also used NTSTATUS previously.
Opening a directory handle explicitly allows to support O_NOFOLLOW
semantics on Windows, as we are able to prevent CreateFile from
following to the reparse point target. This is needed for remove_all
implementation to fix CVE-2022-21658. Similar to POSIX, remove_all_impl
on Windows now requests the directory iterator to not follow symlinks
when it is used to dive into a directory.
FindFirstFileW/FindNextFileW/FindClose based implementation is still
preserved for now for Windows CE, as it is unlikely to support neither
GetFileInformationByHandleEx nor NtQueryDirectoryFile. Given that
Windows CE has been untested for many years and is probably broken
anyway, its support is now declared deprecated. The related code,
including Find*-based directory iterator, will be removed in a
future release.
Closes https://github.com/boostorg/filesystem/issues/224.
And a couple cleanup changes:
- Since we now may use GetFileInformationByHandleEx, we are now using
it to query FILE_ATTRIBUTE_TAG_INFO for testing whether a reparse
point is a symlink.
- In resize_file_impl, we now specify all sharing flags to allow the
operation to succeed if the file is already opened by someone.
Turns out string_file.hpp was tested in operations_unit_test, though not
included directly. Since string_file.hpp contents are disabled when the
test is compiled (as any other deprecated functionality), the test failed
to compile.
Moved the string_file.hpp test to deprecated_test, which explicitly enables
deprecated functionality. Also, include string_file.hpp in filesystem.hpp
if explicitly requested by defining BOOST_FILESYSTEM_DEPRECATED.
This was supposed to be a workaround for clang bug that required an
initializer for global constant objects without a user-defined constructor,
but the initializer classes do have user-defined constructors, so this
workaround should not be needed. Removing it simplifies code a bit.
Marked codecvt error category constructor constexpr and defined
it unconditionally to work around clang bug in XCode 8: the compiler
requires a user-defined constructor to initialize the static const object
of the category.
When constexpr is not available (more precisely, when it is not enabled
by Boost.System), try to dynamic-initialize the category instance early.
For MSVC, invoke codecvt_error_category() from the early global initialization
routine in path.cpp. For other compilers, use a global initializer, possibly
augmented with init_priority attribute.
Closes https://github.com/boostorg/filesystem/issues/227.
Another process could replace the directory being processed by remove_all
with a symlink after remove_all called symlink_status but before
it creates a directory iterator. As a result, remove_all would remove
the linked directory contents instead of removing the symlink.
On POSIX systems that support fdopendir and O_NOFOLLOW flag for open(2),
this can be prevented by opening the directory with O_NOFOLLOW before
iterating. This will fail if the directory was replaced with a symlink.
No protection on other systems.
Reported in https://github.com/boostorg/filesystem/issues/224.
By using volatile qualifier for the internal member of the globals_retainer
class we ensure the constructor formally has observable behavior, which means
it cannot be optimized away.
Related to https://github.com/boostorg/filesystem/issues/217.
MSVC and possibly some other compilers that don't support __attribute__((used))
may remove the global p_init_path_globals pointer in a special data section
because it is not referenced anywhere. Add a dummy global object that
references the pointer in its constructor as a workaround.
Fixes https://github.com/boostorg/filesystem/issues/217.
Reworked remove() operation to separate POSIX and Windows implementations.
On Windows, if the file to be removed is read-only, try to reset the read-only
attribute before deleting the file. If deleting fails (other than because the
file is already deleted), try to restore the read-only attribute.
As a side effect, we were able to remove an implementation detail value from
the file_type enum that was used by the old remove() implementation.
Added a test for remove() on a read-only file on Windows. Also added tests
for remove_all(), including for cases with symlinks, hardlinks and read-only
files.
Also, corrected mklink /J argument in tests. The command accepts /j (lowercase)
to the same effect, but the formal help lists /J (uppercase) to create junctions.
Reported in https://github.com/boostorg/filesystem/issues/216.
In Ubuntu 20.04 there appeared an updated version of the
software-properties-common package in focal-updates, which ships a newer
apt-add-repository version that doesn't support -P/-S/-U command line arguments.
Since we cannot rely on package version checks to determine apt-add-repository
capabilities, we have to parse its --help output instead.
Also, made source list processing more protected against spaces.
In v4 path::lexically_normal, don't generate a trailing dot element if the
original path ends with a directory separator or dot. Also omit the trailing
directory separator the normalized path ends with a dot-dot element.
Additionally, convert directory separators to preferred separators in
root name on Windows (in v3 and v4). This may be significant for UNC paths.
If the source path ends with a non-empty filename, and the appended path
is empty, C++17 std::filesystem requires to add a trailing directory
separator.
Similar to absolute(), canonical/weakly_canonical needed to be updated
to avoid discarding the root name of the path when appending a root
directory during path composition.
Because of the changed semantics of appending operations in v4, path
composition in absolute() would produce incorrect results because at some
point it would append root directory and therefore discard root name
that was potentially added before. The updated implementation fixes that,
and also fixes the case when the input path is already absolute and
starts with a root directory, and the base path has a root name.
Previously, the returned path would contain the root name from the
base path, while the correct thing to do is to return the input path
as is.
Appending an absolute path now results in assigning the path, as
specified in C++17. This change is made for consistency with C++
and other languages that implement path manipulation (e.g. Python).
In Boost.Filesystem v3 path appending mostly worked as a slight upgrade
of concatenation, where appending would only add directory separators
when necessary, but not consider semantics of the root name and root
directory of the appended paths. This would work well for relative paths,
but produce unexpected results for paths with root names.
In v4, we now implement appending that is aware of root name and directory
of the appendedn paths. This means that appending a path with a root name
and/or directory no longer concatenates the paths, but rather rebases the
appended path on top of the source path. In particular, if the appended path
has a root name different from the source path, the append operation will
act as assignment.
This is closer to C++20 std::filesystem but not exactly the same. The
difference is for the case when the appended path is absolute. The C++20
spec requires assignment in this case, Boost.Filesystem v4 deliberately
omits this check. This is to ensure the correct result for UNC paths on
POSIX systems, where "//net/foo" / "/bar" is expected to produce "//net/bar",
not "/bar".
As part of this work, refactored path constructors and operators for more
optimal implementation and reducing the number of overloads.
Closes https://github.com/boostorg/filesystem/issues/214.
The "unused" attribute suppresses warnings emitted by clang for global
constants that are used to hook into early initialization of globals. The
"used" attribute is a precaution to ensure these hooks are not eliminated
by compiler or linker.
