Commit Graph

309 Commits

Author SHA1 Message Date
Yonatan Komornik
70df5de1b2
AsyncIO compression part 1 - refactor of existing asyncio code (#3021)
* Refactored fileio.c:
- Extracted asyncio code to fileio_asyncio.c/.h
- Moved type definitions to fileio_types.h
- Moved common macro definitions needed by both fileio.c and fileio_asyncio.c to fileio_common.h

* Bugfix - rename fileio_asycio to fileio_asyncio

* Added copyrights & license to new files

* CR fixes
2022-01-24 14:43:02 -08:00
Nick Terrell
8ea3d57de4 [build][asm] Pass ASFLAGS to the assembler instead of CFLAGS
* Add `-Wa,--noexecstack` to both `ASFLAGS` and `CFLAGS`
* Pass `ASFLAGS` to `.S` compilation instead of `CFLAGS`

Fixes #3006.
2022-01-18 15:11:29 -08:00
W. Felix Handte
4620ce6a9a Makefiles: Add noexecstack Options to Compilation and Linking
Hopefully this marks the binary artifacts `noexecstack` even on platforms
where binaries default to true.
2022-01-05 15:12:31 -05:00
Yann Collet
a134737ad4
Merge pull request #2687 from sapiippo/dev
Makefile: fix build for mingw
2021-12-02 10:53:02 -08:00
Alexander Kanavin
1e514feec6 Makefile: sort all wildcard file list expansions
Otherwise the order is non-deterministic and breaks
reproducible builds.
2021-12-02 12:04:11 +01:00
Yann Collet
7868f38019
Merge pull request #2747 from Helflym/dev
Add AIX support in Makefile
2021-10-01 08:13:39 -07:00
Nick Terrell
8bf699aa59 [build] Add support for ASM files in Make + CMake
* Extract out common portion of `lib/Makefile` into `lib/libzstd.mk`.
  Most relevantly, the way we find library files.
* Use `lib/libzstd.mk` in the other Makefiles instead of repeating the
  same code.
* Add a test `tests/test-variants.sh` that checks that the builds of
  `make -C programs allVariants` are correct, and run it in Actions.
* Adds support for ASM files in the CMake build.

The Meson build is not updated because it lists every file in zstd,
and supports ASM off the bat, so the Huffman ASM commit will just add
the ASM file to the list.

The Visual Studios build is not updated because I'm not adding ASM
support to Visual Studios yet.
2021-09-17 14:13:53 -07:00
Yann Collet
f21977c5e6 fix playTests.sh when EXE_PREFIX not null 2021-08-29 17:20:12 -07:00
Clément Chigot
399849e236 Makefile: add AIX support
For lib, AIX linker doesn't allow --soname.
2021-08-13 10:25:14 +02:00
Samuli Piippo
fa2a6d4746 Makefile: fix build for mingw
Add ${EXT} to required places to make install succeed for mingw build.
2021-05-27 16:09:37 +03:00
sen
13449d7ce1
Add PHONY targets to makefiles (#2629) 2021-05-07 14:03:19 -04:00
W. Felix Handte
ee122baacf Detect Presence of md5 on Darwin
This fixes #2568.
2021-05-04 12:33:19 -04:00
Niclas Rosenvik
e7647180cd Stop complaining about hash tool not found
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
2021-04-02 13:00:19 -07:00
Nick Terrell
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
nia
74f85818a6 Use standard md5 tool on NetBSD.
This avoids a GNU coreutils dependency.

-n is used to match the output format of coreutils:
http://man.netbsd.org/md5.1
2021-02-11 10:50:11 +01:00
Nick Terrell
54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Yann Collet
0bad3e5c0f parallel make build on linux
fix #2474
2021-01-18 11:33:03 -08:00
Yann Collet
3324e87cff Added library version check 2021-01-07 10:37:27 -08:00
Yann Collet
cefdc023f7 The CLI can be linked to libzstd dynamic library
invoking target zstd-dll
2021-01-06 18:00:24 -08:00
Nick Terrell
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann Collet
f2ac2b7bcf try to fix cross-compiler tests 2020-12-21 15:43:14 -08:00
Yann Collet
9a9d3f76c4 fixed zstd+sanitizer build 2020-12-20 17:53:04 -08:00
Yann Collet
f9884036c2 fixed zstd recipe 2020-12-20 17:19:23 -08:00
Yann Collet
9648bf027b try to keep libzstd.a "as is" once created
to be compatible with scenarios such as
`make -j allmost`
2020-12-20 17:10:57 -08:00
Yann Collet
c2939b96be refinement : only copy if binary is different 2020-12-10 14:31:35 -08:00
Yann Collet
c56723ab03 replace final links by direct copy
link can behave slightly differently from real binaries,
breaking a few scripts relying on "real binary" assumption.
2020-12-10 13:25:08 -08:00
Yann Collet
0adce4631d Merge branch 'libzstd_autoconf_full' of github.com:facebook/zstd into libzstd_autoconf_full 2020-10-28 10:25:55 -07:00
Yann Collet
f6ecf1568f minor Makefile refactor
hopefully improving readability
2020-10-28 09:39:15 -07:00
Yann Collet
ceccd7ae2d Merge branch 'dev' into libzstd_autoconf_full 2020-10-27 15:45:30 -07:00
Björn Ketelaars
1f661b5f6b 'head -c BYTES' is non-portable 2020-10-27 16:55:23 +01:00
Yann Collet
456db0c377 make install only rebuild binaries if they don't exist
Now `make` followed by `make install` doesn't rebuild binaries

also : only generated target directories if they don't already exist
2020-10-23 16:46:49 -07:00
Yann Collet
a6ee614a44 make zstd is now differentiated from zstd-nomt
avoids mixing object files using different flags
2020-10-23 16:08:21 -07:00
Yann Collet
89b961ea46 simplified silent mode maintenance 2020-10-23 10:41:17 -07:00
Yann Collet
e3867fb735 fixed libzstd.dll compilation on mingw
and zstd linking
2020-10-22 11:52:19 -07:00
Yann Collet
dd24496951 programs/zstd also automatically generate object dir per conf
same rules as lib/libzstd
can also be controlled via HASH and BUILD_DIR
2020-10-21 23:38:33 -07:00
Yann Collet
01ecad2326 zstd in programs/ can also receive a DEBUGLEVEL value
simplify tests : only set DEBUGLEVEL, not the flags directly
2020-10-21 23:13:46 -07:00
Yann Collet
684f8ea970 decouple zstd object files from lib/
That was a subtle one :
VPATH is affecting search for both %.c source and %.o object files.
This meant that, when an object file already exists in lib/,
it's used in programs/,
even though programs/ is supposed to generate its own %.o object files.

With the new vpath directive, this is no longer the case :
the search is only activated for %.c source files.
Now, local programs/%.o are always generated
even if equivalent ones are already created in lib/.

It more clearly guarantees that lib/ and programs/ can use different compilation directives
without mixing resulting %.o object files.
2020-10-20 18:16:11 -07:00
Yann Collet
eb1959d926 fix .deps list
and minor simplications
2020-10-16 11:08:52 -07:00
Yann Collet
c355d149de support verbose build output with V=1 2020-10-16 00:21:49 -07:00
Yann Collet
b6b87c2b77 track header dependencies during build
modifying a header file will now trigger recompilation of related *.c units
2020-10-16 00:18:16 -07:00
Yann Collet
80cee8d3fe faster rebuild of zstd
Building the zstd CLI costs time.
Some part of it is incompressible, leading to substantial iteration delay when testing code modifications.
That's mainly because all source files from the library must be rebuilt from source every time.

The main reason we don't build the CLI from library object files
is that we can't just build the object directly in the lib/ directory
(which they would by default)
since they use different compilation flags.
Specifically, the CLI enables multithreading, while the library doesn't (by default).

This is solved in this commit, by generating the object files locally.
Now, the CLI and the library can employ different sets of flags, without tripping over each other.
All library object files are generated directly into programs/ dir.
This works because no 2 source files have the same name.

Now, modifying a file doesn't require to recompile the entire lib, just the modified files.
The recipe is also compatible with `-j` parallel build, leading to large build time reductions on multi-core systems.
2020-10-16 00:01:41 -07:00
Martin Liska
5b9d6b992a
Update comment about PGO build. 2020-07-09 09:31:51 +02:00
Yann Collet
f5f0bb2591
Merge pull request #2125 from Artoria2e5/patch-2
programs/Makefile: make it actually work with clang
2020-05-12 13:48:23 -07:00
Mingye Wang
c3398d2dda
programs/Makefile: make it actually work with clang
Recent versions of clang seem to require a step of conversion from the profraw file to a "compiler-readable" profdata file to let PGO work.
2020-05-10 16:36:08 +08:00
Yann Collet
e5366bee16 updated zstd -h
to help readability.
Group arguments into categories,
try to give better visibility to more common arguments.
2020-05-08 14:14:46 -07:00
W. Felix Handte
7dcca6bc64 Also Move programs/ Directory to Relative Includes 2020-05-04 15:20:26 -04:00
Yann Collet
6f62a9caaa fixed zstd-nolegacy target
when compiled as part of allVariants

(though I'm unsure why it was working before ...)
2020-04-29 11:56:21 -07:00
Yann Collet
7ea2ae6649 added test linking user program to multi-threaded libzstd 2020-04-28 21:18:29 -07:00
Nick Terrell
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Björn Ketelaars
cdd59c50e7 Fix forgotten portable header prefix
PR #1987 forgot to prefix another include causing lz4 detection to fail.
2020-02-15 12:42:00 +01:00