townforge/zstd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,629 Commits 1 Branch 0 Tags 29 MiB
cc
Commit Graph

9629 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dima Krasner
c1f25eaec0 Added a Meson project 2017-02-16 19:37:08 +02:00
Przemyslaw Skibinski
40dadd65cc join tests into pairs 2017-02-16 18:19:36 +01:00
Przemyslaw Skibinski
6babbff58d move MOREFLAGS to circle.yml 2017-02-16 17:52:49 +01:00
Yann Collet
3ebec8383b Merge pull request #549 from iburinoc/matchoverlenwrite 
Fix wildcopy overwriting data still in window
 2017-02-16 08:39:52 -08:00
Przemyslaw Skibinski
84452ca29f more balanced tests 2017-02-16 16:27:40 +01:00
Przemyslaw Skibinski
21d9022b88 two groups of tests 2017-02-16 15:59:00 +01:00
Przemyslaw Skibinski
6b64abb287 improved clangtest 2017-02-16 15:28:08 +01:00
Przemyslaw Skibinski
9a0161d376 imporved test-zstd 2017-02-16 15:13:33 +01:00
Przemyslaw Skibinski
fa492a3eca Tests for thread 1 (when CIRCLE_NODE_TOTAL=1) or thread 2 2017-02-16 14:39:21 +01:00
Przemyslaw Skibinski
cb76944861 final colon 2017-02-16 13:51:21 +01:00
Przemyslaw Skibinski
d3ff834562 check CIRCLE_NODE_TOTAL 2017-02-16 13:45:40 +01:00
Przemyslaw Skibinski
9e97a8a45a check $CIRCLE_NODE_INDEX 2017-02-16 13:36:12 +01:00
Przemyslaw Skibinski
f8a5749c22 circle.yml: run only short tests 2017-02-16 13:08:30 +01:00
Przemyslaw Skibinski
b0511aeca5 fix travis.yml 2017-02-16 12:33:25 +01:00
Przemyslaw Skibinski
e0d2a146d1 .travis.yml: detect "$TRAVIS_EVENT_TYPE" = "cron" 2017-02-16 12:29:08 +01:00
Sean Purcell
887eaa9e21 Fix wildcopy overwriting data still in window 2017-02-15 16:43:45 -08:00
Yann Collet
2252d29a5a Merge branch 'dev' of github.com:facebook/zstd into dev 2017-02-15 12:00:50 -08:00
Yann Collet
4596037042 updated fse version 
feature minor refactoring (removing FSE_abs())
also : fix a few minor issues recently introduced in examples
 2017-02-15 12:00:03 -08:00
Yann Collet
44f82d781f Merge pull request #545 from terrelln/force-window 
[zstdmt] Fix MSAN failure with ZSTD_p_forceWindow
 2017-02-15 10:20:15 -08:00
Przemyslaw Skibinski
acb6e57ad2 use FindFirstFileA instead of FindFirstFile 2017-02-15 17:13:35 +01:00
Przemyslaw Skibinski
6e59b3ce01 added UTIL_fseek 2017-02-15 17:03:16 +01:00
Przemyslaw Skibinski
3aaa1dae4e simplified zlib detection 2017-02-15 09:17:39 +01:00
Przemyslaw Skibinski
d444f27f35 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-02-15 09:10:08 +01:00
Przemyslaw Skibinski
90e5412a4e added -I/usr/include/x86_64-linux-gnu for asan32 2017-02-14 23:30:23 +01:00
Yann Collet
f0b9a8dddb Merge pull request #547 from inikep/dev11 
Avoid fseek()'s 2GiB barrier with MacOS and *BSD
 2017-02-14 12:29:00 -08:00
Przemyslaw Skibinski
9b5a1e9d97 added circle.yml 2017-02-14 20:06:41 +01:00
Przemyslaw Skibinski
226a6387a9 Merge remote-tracking branch 'refs/remotes/facebook/dev' into gz_compress 2017-02-14 20:04:56 +01:00
Yann Collet
c09d16ba8c preset behavior for gzip, gunzip and gzcat 
when zstd is called through a link named gzip, gunzip or gzcat,
provides the same behavior as the related program.
gzip compresses using --format=gz
both gzip and gunzip enable --rm by default
 2017-02-14 10:45:19 -08:00
Yann Collet
aca067f19d Merge pull request #543 from inikep/gz_compress 
Gz compress
 2017-02-14 10:30:22 -08:00
Nick Terrell
74b81ada25 Don't run test-pool with QEMU 
> make test -n
    ...
    ./pool
    > make test -n QEMU_SYS=valgrind
    ...
    ./legacy
    # ./pool not run
 2017-02-14 10:08:18 -08:00
Yann Collet
9696bfc2ad Merge pull request #544 from ds77/avoid-empty 
Portable way to avoid empty unit warning in threading.c
 2017-02-14 00:54:55 -08:00
Przemyslaw Skibinski
ce13d087d9 fix LONG_SEEK 2017-02-14 09:52:52 +01:00
Yann Collet
1decff24b8 Merge pull request #546 from zefanxu/dev 
fix memory leak
 2017-02-14 00:49:06 -08:00
Przemyslaw Skibinski
970419535f fixed function name (2) 2017-02-14 09:47:29 +01:00
Przemyslaw Skibinski
bf336572bf Avoid fseek()'s 2GiB barrier with MacOS and *BSD 2017-02-14 09:45:33 +01:00
Przemyslaw Skibinski
abd6302423 Windows resources updated to v1.1.4 2017-02-14 09:39:09 +01:00
Przemyslaw Skibinski
442c75f132 removed UTIL_doesFileExists (replaced with UTIL_isRegFile) 2017-02-14 09:38:51 +01:00
Przemyslaw Skibinski
b876b96ce1 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2017-02-14 09:26:03 +01:00
Przemyslaw Skibinski
98509a70ac fixed function name 2017-02-14 09:23:32 +01:00
zefanxu2
2bb6fc2a94 fix memory leak 2017-02-13 21:12:59 -06:00
Nick Terrell
ecf90ca24b [zstdmt] Fix MSAN failure with ZSTD_p_forceWindow 
Reproduction steps:

```
make zstreamtest CC=clang CFLAGS="-O3 -g -fsanitize=memory -fsanitize-memory-track-origins"
./zstreamtest -vv -t4178 -i4178 -s4531
```

How to get to the error in gdb (may be a more efficient way):

* 2 breaks at zstd_compress.c:2418  -- in ZSTD_compressContinue_internal()
* 2 breaks at zstd_compress.c:2276  -- in ZSTD_compressBlock_internal()
* 1 break at zstd_compress.c:1547

Why the error occurred:

When `zc->forceWindow == 1`, after calling `ZSTD_loadDictionaryContent()` we
have `zc->loadedDictEnd == zc->nextToUpdate == 0`. But, we've really loaded up
to `iend` into the dictionary. Then in `ZSTD_compressBlock_internal()` we see
that `current > zc->nextToUpdate + 384`, so we load the last 192 bytes a second
time. In this case the bytes we are loading are a block of all 0s, starting in
the previous block. So when we are loading the last 192 bytes, we find a `match`
in the future, 183 bytes beyond `ip`. Since the block is all 0s, the match
extends to the end of the block. But in `ZSTD_count()` we only check that
`pIn < pInLoopLimit`, but since `pMatch > pIn`, `pMatch` eventually points past
the end of the buffer, causing the MSAN failure.

The fix:

The line changed sets sets `zc->nextToUpdate` to the end of the dictionary.
This is the behavior that existed before `ZSTD_p_forceWindow` was introduced.
This fixes the exposing test case. Since the code doesn't fail without
`zc->forceWindow`, it makes sense that this works. I've run the command
`./zstreamtest -T2mn` 64 times without failures. CI should also verify nothing
obvious broke.
 2017-02-13 19:11:22 -08:00
Yann Collet
58af614ef2 push version and NEWS to v1.1.4 2017-02-13 18:32:44 -08:00
ds77
08e6a88a97 avoid empty translation unit warning without #pragma 2017-02-14 00:46:47 +01:00
Przemyslaw Skibinski
1a195b3b7a fixed unitialized variable warning 2017-02-13 22:56:31 +01:00
Przemyslaw Skibinski
48466b36e8 Resolve conflict with branch 'refs/remotes/facebook/dev' 
# Conflicts:
#	programs/fileio.c
 2017-02-13 21:35:39 +01:00
Yann Collet
db2666c10c Merge pull request #536 from iburinoc/multiframe 
Simple API multiframe decoding
 2017-02-13 12:18:16 -08:00
Przemyslaw Skibinski
64f7221958 limit zlib compression level to Z_BEST_COMPRESSION 2017-02-13 21:00:41 +01:00
Yann Collet
73d7a15707 Merge pull request #542 from ds77/large-sparse-fix 
zstdcli: fix writing 2GB+ sparse files under Windows
 2017-02-13 11:57:35 -08:00
Przemyslaw Skibinski
5a3bb05bb2 Merge remote-tracking branch 'refs/remotes/facebook/dev' into gz_compress 
# Conflicts:
#	programs/Makefile
 2017-02-13 20:47:01 +01:00
Przemyslaw Skibinski
35bf23c086 MinGW-w64 requires _FILE_OFFSET_BITS 64 2017-02-13 13:57:29 +01:00