MSVC and possibly some other compilers that don't support __attribute__((used))
may remove the global p_init_path_globals pointer in a special data section
because it is not referenced anywhere. Add a dummy global object that
references the pointer in its constructor as a workaround.
Fixes https://github.com/boostorg/filesystem/issues/217.
Reworked remove() operation to separate POSIX and Windows implementations.
On Windows, if the file to be removed is read-only, try to reset the read-only
attribute before deleting the file. If deleting fails (other than because the
file is already deleted), try to restore the read-only attribute.
As a side effect, we were able to remove an implementation detail value from
the file_type enum that was used by the old remove() implementation.
Added a test for remove() on a read-only file on Windows. Also added tests
for remove_all(), including for cases with symlinks, hardlinks and read-only
files.
Also, corrected mklink /J argument in tests. The command accepts /j (lowercase)
to the same effect, but the formal help lists /J (uppercase) to create junctions.
Reported in https://github.com/boostorg/filesystem/issues/216.
In v4 path::lexically_normal, don't generate a trailing dot element if the
original path ends with a directory separator or dot. Also omit the trailing
directory separator the normalized path ends with a dot-dot element.
Additionally, convert directory separators to preferred separators in
root name on Windows (in v3 and v4). This may be significant for UNC paths.
Because of the changed semantics of appending operations in v4, path
composition in absolute() would produce incorrect results because at some
point it would append root directory and therefore discard root name
that was potentially added before. The updated implementation fixes that,
and also fixes the case when the input path is already absolute and
starts with a root directory, and the base path has a root name.
Previously, the returned path would contain the root name from the
base path, while the correct thing to do is to return the input path
as is.
Appending an absolute path now results in assigning the path, as
specified in C++17. This change is made for consistency with C++
and other languages that implement path manipulation (e.g. Python).
In Boost.Filesystem v3 path appending mostly worked as a slight upgrade
of concatenation, where appending would only add directory separators
when necessary, but not consider semantics of the root name and root
directory of the appended paths. This would work well for relative paths,
but produce unexpected results for paths with root names.
In v4, we now implement appending that is aware of root name and directory
of the appendedn paths. This means that appending a path with a root name
and/or directory no longer concatenates the paths, but rather rebases the
appended path on top of the source path. In particular, if the appended path
has a root name different from the source path, the append operation will
act as assignment.
This is closer to C++20 std::filesystem but not exactly the same. The
difference is for the case when the appended path is absolute. The C++20
spec requires assignment in this case, Boost.Filesystem v4 deliberately
omits this check. This is to ensure the correct result for UNC paths on
POSIX systems, where "//net/foo" / "/bar" is expected to produce "//net/bar",
not "/bar".
As part of this work, refactored path constructors and operators for more
optimal implementation and reducing the number of overloads.
Closes https://github.com/boostorg/filesystem/issues/214.
When the path ends with a non-root directory separator, no longer
produce a trailing dot element (filename). Instead, return an empty
path.
This affects not only path iterators and path::filename, but also any
other APIs that rely on them.
Closes https://github.com/boostorg/filesystem/issues/193.
The new implementation is prepared for the removal of the implicit
trailing dots in v4 path. It also no longer uses recursion
internally and therefore is better protected against stack overflows.
As a side effect of this rewrite, create_directories no longer reports
error if the input path consists entirely of dot and dot-dot elements.
This is in line with C++20 std::filesystem behavior.
Windows APIs such as GetFileAttributesW perform lexical path normalization
internally, which means e.g. "C:\a\.." resolves to an existing path
even if "C:\a" doesn't. This breaks depection of the longest sequence
of existing path elements in weakly_canonical and results in an error
in canonical that is called on that sequence.
As a workaround, perform forward iteration on Windows, so that we
stop on the first path element that doesn't exist.
Also, while at it, corrected error code reported from weakly_canonical
when status fails with an error.
Closes https://github.com/boostorg/filesystem/issues/201.
This works around recurringissues when Boost.Filesystem is used during
program termination (for example, in Boost.Log, when it performs the final
log file rotation). At that point, the path locale as well as dot and dot-dot
paths may no longer be available.
Also, MSVC 14.2 has a bug[1] that results in a deadlock whet dot or
dot-dot path is being created during program termination, while atexit
callbacks are being run in the main thread. This change works around it
as the new code does not call atexit on initialization of these paths.
This is only supported on MSVC, GCC, Clang and compatible compilers that
support MSVC-specific or GCC-specific means to customize global initialization
order.
[1]: https://github.com/boostorg/log/issues/153
Fall back to reading /dev/(u)random if getrandom fails with ENOSYS.
Also, extracted the portability macros for atomics to a separate header
to be able to use them in unique_path.cpp. Rearranged function pointers
initialization to decouple the initializer object from the particular
system calls.
For getrandom, the ENOSYS failure is only cached if the compiler supports
specifying global object initialization priority, which is needed to ensure
that the function pointer is initialized before the syscall initializer
in a different TU. If the compiler does not support this feature, just
always attempt getrandom first.
This can be useful if the syscall is present at compile time but fails with
ENOSYS at run time (for example, in Docker containers that restrict the syscall,
even if available on the host).
Additionally, marked statx syscall wrappers with attributes to disable MSAN
for them. It was reported that MSAN on clang 10 is showing errors
accessing uninitialized data in stx_mask, which must be initialized by the
syscall.
Related to https://github.com/boostorg/filesystem/issues/172
Related to https://github.com/boostorg/filesystem/issues/185
Boost.Filesystem v4 will contain breaking changes from v3 that are required
for better compatibility with C++17 std::filesystem. It will also remove
the deprecated features of v3.
Updated docs to reflect the differences between v3 and v4. Updated tests
to verify both v3 and v4 where the differences are present.
This is a breaking change.
path::filename accessor now only returns the actual filename or the implied
trailing dot element of the path, if it ends with a separator other than
root directory. This makes boost::filesystem::path behavior closer to that
of std::filesystem::path.
Updated tests and docs accordingly.
Closes https://github.com/boostorg/filesystem/issues/194.
If Windows is running in Developer mode, it is possible to specify
SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag to CreateSymbolicLinkW
so that the call doesn't require elevated privileges.
While at it, explicitly separated implementation of create_symlink and
create_directory_symlink for POSIX and Windows.
This doesn't compile with gcc 8 on MinGW-w64, and fails in runtime with
gcc 10.2 and clang 8.0.1 on Cygwin64 because character code conversion errors,
so basically std::filesystem never works with wide paths on Windows.
We still use wide paths as `const wchar_t*` with libc++ though.
Also, changed BOOST_FILESYSTEM_C_STR definition to accept the path as
an argument and use that definition in the tests rather than duplicating it.
Related to https://github.com/boostorg/filesystem/issues/181.
libstdc++ starting 8.1.0 in C++17 mode support std::filesystem and is able to
open the file streams with std::filesystem::path constructible from wide
strings.
libc++ starting 7.0 also supports std::filesystem::path in C++17 mode and may
also support passing `const wchar_t*` strings to open files.
Closes https://github.com/boostorg/filesystem/issues/181.
Filenames starting with a dot (and no other dots) are commonly treated
as filenames with no extension rather than an extension. This is also
the behavior mandated in C++17 filesystem.
Reported in https://github.com/boostorg/filesystem/issues/88.
The print name can be empty for some reparse points (e.g. mount points
created by Box cloud storage driver and directory junctions created by
junction.exe). It is supposed to be mostly used for presenting a "simple"
path to the user and not to actually locate the file.
The substitute name is the actionable replacement path, but it is in
NT path format and can potentially point to unmounted volumes and
UNC resources. The implementation attempts to convert the NT path
to Win32 path by matching commonly known patterns against the NT path.
If no pattern matches, we create a Win32 path by converting the NT path
prefix to "\\?\".
Related to https://github.com/boostorg/filesystem/issues/187.
The new implementation is also not relying on the root name format and
is more pertormant as it avoids unnecessarily copying path elements during
operation.
Note that this commit does not remove the trailing dot elements in the
normalized paths.
- Unified root name and root directory parsing that was scattered and
duplicated across different algorithms. The new implementation is
consolidated in a single function for parsing root name and root
directory, which is used from various algorithms.
- The new root name parsing now supports Windows local device ("\\.\")
and NT path ("\??\") prefixes. It also adds support for filesystem
("\\?\") prefix to some of the higher level algorithms that were
using custom parsing previously. Tests updated to verify these prefixes.
- Some of the path decomposition methods were unified with presence checking
methods (e.g. root_name with has_root_name). This makes these methods
work consistently and also makes the has_* methods less expensive as
they no longer have to construct a path only to check if it is empty.
- The filename accessor no longer returns root name if the whole path
only consists of a root name. This also affects stem and extension as
those accessors are based on filename. This is a breaking change.
- Cleaned up code:
- Removed redundant checks for std::wstring support.
- Added header/footer headers to globally disable compiler warnings.
- Removed commented out super-deprecated code.
- Added missing includes and removed includes that are not needed.
- Nonessential code formatting.
When canonical() resolves symlink, it is possible that a symlink resolves
to an absolute path with a different root. We need to update the root
path so that when we restart symlink resolution the check for the
root path still works.
Also, slightly refactored the canonical() implementation to reduce code
size and possibly optimize the generated code.
The buffer size is now selected based on the file size and filesystem block
size and is limited with min and max. This allows to reduce memory consumption
and possibly increase performance when copying smaller files.
Some filesystems have regular files with generated content. Such files have
arbitrary size, including zero, but have actual content. Linux system calls
sendfile or copy_file_range will not copy contents of such files, so we must
use a read/write loop to handle them.
Check the type of the source filesystem before using sendfile or
copy_file_range and fallback to the read/write loop if it matches one of
the blacklisted filesystems: procfs, sysfs, tracefs or debugfs.
Also, added a test to verify that copy_file works on procfs.
Since sendfile and copy_file_range can fail for some filesystems
(e.g. eCryptFS), we have to fallback to the read/write loop in copy_file
implementation. Additionally, since we implement the fallback now,
fallback to sendfile if copy_file_range fails with EXDEV and use
copy_file_range on older kernels that don't implement it for
cross-filesystem copying. This may be beneficial if copy_file_range
is used within a filesystem, and is performed on a remote server NFS or CIFS).
Also, it was discovered that copy_file_range can also fail with EOPNOTSUPP
when it is performed on an NFSv4 filesystem and the remote server does
not support COPY operation. This happens on some patched kernels in RHEL/CentOS.
Lastly, to make sure the copy_file_data pointer is accessed atomically,
it is now declared as an atomic value. If std::atomic is unavailable,
Boost.Atomic is used.
Fixes https://github.com/boostorg/filesystem/issues/184.
These options allow to synchronize the copied data and attributes with
the permanent storage. Note that by default on POSIX systems copy_file
used to synchronize data in previous releases, and this commit changes
this. The caller now has to explicitly request syncing, as it has
significant performance implications.
Closes https://github.com/boostorg/filesystem/issues/186.
At least HP-UX is known to leave the file descriptor open if close() returns
EINTR. On other systems (Linux, BSD, Solaris, AIX) the file descriptor
is closed in the same situation, and closing it again may potentially close
the wrong descriptor if it is reused by another thread. We introduce
close_fd internal helper to abstract away these platform differences.
By defining these new config macros the user can configure the library
to avoid using some system APIs even if they are detected as available
by the library build scripts. This can be useful in case if the API
is known to consistently fail at runtime on the target system.
Related to https://github.com/boostorg/filesystem/issues/172.
create_directories used to ignore errors returned by status()
calls issued internally. The operation would likely fail anyway,
but the error codes returned by create_directories would be incorrect.
Also, it is better to terminate the operation as early as possible
when an error is encountered.
Reported in https://github.com/boostorg/filesystem/issues/182.
The Linux statx system call allows to specify the data the caller is
interested in. This has the potential of improving performance
if some information is expensive to provide.
Also, changed hard_link_count to return static_cast<uintmax_t>(-1)
in case of errors. Changed file_size to report ENOSYS instead of
EPERM when the operation is invoked on a non-regular file.
This follows C++20 definition, which returns a minimum representable time
point value in case of error.
Also, slight refactoring of last_write_time setter function and a fix
to ensure the error code is cleared incase of success.
