townforge/zstd
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9,629 Commits 1 Branch 0 Tags 29 MiB
cc
Commit Graph

4226 Commits

Author SHA1 Message Date
Yann Collet
ee5b725823 ZSTD_initCStream() optimization : do not allocate a CDict when no dictionary used 2016-10-27 14:20:55 -07:00
Nick Terrell
eb7873a048 ZSTD_execSequence() accepts match in last 7 bytes 
The zstd reference compressor will not emit a match in the last 7
bytes of a block.  The decompressor will also not accept a match
in the last 7 bytes.  This patch makes the decompressor accept a
match in the last 7 bytes.
 2016-10-25 21:24:15 -07:00
Yann Collet
335ad5d4d4 added ZSTD_initDStream_usingDDict() . 
slightly optimized ZSTD_initDStream() when no dictionary .
fixed ZSTD_sizeof_CStream() .
 2016-10-25 17:47:02 -07:00
Yann Collet
9516234e67 first sketch for ZSTD_initCStream_usingCDict() 2016-10-25 16:19:52 -07:00
Yann Collet
62d9a7ddfd Merge pull request #429 from inikep/btopt2 
Btopt2
 2016-10-25 14:48:43 -07:00
Przemyslaw Skibinski
5c5f01f3da added ZSTD_btopt2 strategy 2016-10-25 12:25:07 +02:00
Yann Collet
7b5948cca7 Merge pull request #426 from terrelln/fixes 
Fix various {A, M}SAN bugs
 2016-10-24 23:42:26 -07:00
Yann Collet
37d130031d updated comments on context re-use 2016-10-24 17:22:12 -07:00
Nick Terrell
b2c39a22b0 Fix compiler narrowing warning 2016-10-24 14:50:13 -07:00
Nick Terrell
f698ad6deb Merge remote-tracking branch 'upstream/dev' into fixes 
* upstream/dev:
  added doc\zstd_manual.html
  added contrib\gen_html
  zstd_compression_format.md moved to doc/
  Fix small bug in ZSTD_execSequence()
  improved ZSTD_compressBlock_opt_extDict_generic
  protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob
  zstd_opt.h: small improvement in compression ratio
  improved dicitonary segment merge
  use implicit rules to compile zstd_decompress.c
  detect early impossible decompression scenario in legacy decoder v0.5
  no repeat mode in legacy v0.5
  fixed invalid invocation of dictionary in legacy decoder v0.5
  fix edge case
  fix command line interpretation
  fixed minor corner case
  zstd.h: added the Introduction section
  fixed clang 3.5 warnings
  zstd.h: updated comments
 2016-10-24 13:10:13 -07:00
Yann Collet
4239a207dd Merge pull request #425 from inikep/dev11 
Doc
 2016-10-24 11:11:40 -07:00
Nick Terrell
f9c9af3c2e Reject dictionaries with incomplete entropy tables 
If a dictionary specifies that a symbol has probability zero in its
`matchLength`, `literalLength`, or `offset` FSE table, but the symbol
appears when compressing input, the compressor fails.

Ensure that dictionaries support all `matchLength`, and `literalLength`
codes.  They must also support all of the `offset` codes required to
represent every possible offset that can appear in the first block.
 2016-10-24 10:42:44 -07:00
Przemyslaw Skibinski
984b66cd72 added contrib\gen_html 2016-10-24 15:59:51 +02:00
Przemyslaw Skibinski
3ee94a7600 zstd_compression_format.md moved to doc/ 2016-10-24 15:58:07 +02:00
Yann Collet
97611611a3 Merge pull request #423 from terrelln/exec-seq-patch 
Fix small bug in ZSTD_execSequence()
 2016-10-21 17:02:06 -07:00
Nick Terrell
ae1cb3b3d0 Fix small bug in ZSTD_execSequence() 
`memmove(op, match, sequence.matchLength)` is not the desired behavior.
Overlap is allowed, and handled as if we did `*op++ = *match++`, which
is not how `memmove()` handles overlap.

Only triggered if both of the following conditions are met:
* The match spans extDict & currentPrefixSegment
* `oLitEnd <= oend_w < oLitEnd + length1 < oMatchEnd <= oend`.

These two conditions imply that the block is less than 15 bytes long.
This bug isn't triggered by the streaming API, because it allocates
enough space for the window size + the block size, so there cannot be
a match that is within 8 bytes of the end and overlaps with itself.
It cannot be triggered by the block decompression API because all of
the decompressed data is in the currentPrefixSegment.

Introduced by commit 7158584399
 2016-10-21 12:13:44 -07:00
Przemyslaw Skibinski
4732074a71 improved ZSTD_compressBlock_opt_extDict_generic 2016-10-21 11:19:00 +02:00
Yann Collet
da3bd8b6de protect ZSTD_decodeFrameHeader() from invalid usage, as suggested by @spaskob 2016-10-20 20:11:00 -07:00
Przemyslaw Skibinski
d365ae3497 zstd_opt.h: small improvement in compression ratio 2016-10-20 11:49:02 +02:00
Przemyslaw Skibinski
575ab00db7 Merge remote-tracking branch 'refs/remotes/facebook/dev' into dev11 2016-10-20 11:01:52 +02:00
Nick Terrell
d760529a05 Fix stack buffer overrun when weightTotal == 0 
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
 2016-10-19 11:39:11 -07:00
Nick Terrell
bb68062c59 Unitialized memory read in ZSTD_decodeSeqHeaders() 
Caused by two things:
1. Not checking that `ip` is in range except for the first byte.
2. `ZSTDv0{5,6}_decodeLiteralsBlock()` could return a value larger than `srcSize`.
 2016-10-18 16:41:33 -07:00
Yann Collet
52c1bf93fe improved dicitonary segment merge 2016-10-18 16:34:58 -07:00
Nick Terrell
7b06ad7a05 Backport fix from commit 125d817 
This fixes a read of unitialized memory.
Full commit hash: 125d81774f.
 2016-10-18 14:52:34 -07:00
Nick Terrell
f45b157d95 Backport fix from commit 9e8b09a 
Fixes uninitialized memory reads.
Full commit hash: 9e8b09a7bd
 2016-10-18 14:22:49 -07:00
Yann Collet
f7906d5955 detect early impossible decompression scenario in legacy decoder v0.5 2016-10-18 13:48:32 -07:00
Yann Collet
9313c8d953 no repeat mode in legacy v0.5 2016-10-18 13:36:15 -07:00
Yann Collet
83d7bdee4b fixed invalid invocation of dictionary in legacy decoder v0.5 2016-10-18 12:25:43 -07:00
Yann Collet
197a55ee7b fix edge case 2016-10-18 11:27:52 -07:00
Nick Terrell
fd98087047 Fix stack buffer overflow in HUF_readCTable() 
If `w ==0` on line 153, then `CTable[n].nbBits == tableLog + 1`.
Then `nbPerRank[CTable[n].nbBits]` and `valPerRank[CTable[n].nbBits]`
are stack buffer overflows.
 2016-10-17 18:16:59 -07:00
Yann Collet
06573e17be fixed minor corner case 2016-10-17 17:28:28 -07:00
Nick Terrell
bfd943ace5 Fix buffer overrun in ZSTD_loadDictEntropyStats() 
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`.  This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.

The benchmarks look good, there is no obvious compression performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 271.6 MB/s , 716.8 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 204.8 MB/s , 671.1 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.8 MB/s , 658.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 136.4 MB/s , 665.3 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.9 MB/s , 647.0 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  65.2 MB/s , 670.4 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.9 MB/s , 688.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  32.4 MB/s , 711.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.1 MB/s , 718.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  22.2 MB/s , 721.8 MB/s

  > ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 273.8 MB/s , 722.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 203.2 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 157.4 MB/s , 666.5 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 132.1 MB/s , 661.9 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.8 MB/s , 641.6 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  63.1 MB/s , 677.0 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.3 MB/s , 678.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  33.1 MB/s , 708.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.5 MB/s , 710.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  21.9 MB/s , 723.9 MB/s
 2016-10-17 16:55:52 -07:00
Nick Terrell
4db751668f Fix buffer overrun in ZSTD_loadEntropy() 
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadEntropy()`.  This caused `FSE_buildDTable(dctx->MLTable, ...)`
to overwrite the beginning of `dctx->hufTable`.

The benchmarks look good, there is no obvious performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 268.2 MB/s , 701.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.5 MB/s , 666.9 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 154.9 MB/s , 655.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 128.9 MB/s , 648.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.4 MB/s , 633.4 MB/s

  > ./zstds/zstd.opt.2 -i10 -b1 -e5 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 266.1 MB/s , 703.7 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 199.0 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.2 MB/s , 656.2 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 133.2 MB/s , 647.4 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.3 MB/s , 633.3 MB/s
 2016-10-17 15:51:15 -07:00
Nick Terrell
ccfcc643da Check if dict is empty before reading first byte 2016-10-17 11:46:03 -07:00
Yann Collet
2b361cf2f1 minor opt 2016-10-14 16:09:07 -07:00
Yann Collet
7933434fdf Merge branch 'dev' of github.com:facebook/zstd into dev 2016-10-14 13:32:35 -07:00
Yann Collet
d4cda27b63 new command -M#, to limit memory usage during decompression (#403) 2016-10-14 13:32:20 -07:00
Nick Terrell
3b9cdf9220 Fix ubsan failures (pass NULL to memcpy) 2016-10-12 20:54:42 -07:00
Yann Collet
5d919e7ac3 added ZSTD_error_frameParameter_windowTooLarge (#403) 2016-10-12 17:29:24 -07:00
Yann Collet
e19111c42f make creates libzstd binaries (#415) 2016-10-12 11:09:36 -07:00
Yann Collet
8b70d012f0 fix cmake 2016-10-12 10:23:53 -07:00
Yann Collet
38fb0dc4cf Merge pull request #416 from terrelln/exec-sequence 
Fix ZSTD_execSequence() edge case
 2016-10-12 10:17:53 -07:00
Nick Terrell
7158584399 Fix ZSTD_execSequence() edge case 2016-10-12 10:05:26 -07:00
Yann Collet
f52cd03e73 bumped version number 2016-10-11 17:29:27 -07:00
Yann Collet
ef2357d0d3 created error_private.c, so that a single list of error strings get included 2016-10-11 17:24:50 -07:00
Yann Collet
14efab827b added zstd_errors.h to include installation 2016-10-11 16:51:29 -07:00
Yann Collet
a17fd7312a changed error_public.h into zstd_errors.h 2016-10-11 16:41:09 -07:00
Yann Collet
18b51b99c0 sync fse 2016-10-11 08:21:09 -07:00
inikep
2d2613399a zstd.h: added the Introduction section 2016-10-06 16:28:21 +02:00
inikep
ba1db376ac fixed clang 3.5 warnings 2016-10-06 14:22:48 +02:00
inikep
82057aa7ec zstd.h: updated comments 2016-10-06 13:23:52 +02:00
Yann Collet
df6797447f update dictionary builder warning comments 2016-09-27 15:14:32 +02:00
Yann Collet
47094ea66b added comment on filePos 2016-09-26 18:03:33 +02:00
Yann Collet
cf409a7e2a fixed : init*_advanced() followed by reset() with different pledgedSrcSiz 2016-09-26 16:41:05 +02:00
Yann Collet
2f2639438a zstreamtest can fuzztest pledgedSrcSize 2016-09-26 14:06:08 +02:00
Christophe Chevalier
dc245e91cb Changed to use ZSTDLIBv06_API and ZSTDLIBv07_API for DLL exports to fix warning 
- changed name to prevent collision with ZSTDLIB_API used by non-legacy dll exports
 2016-09-23 17:09:36 +02:00
Yann Collet
21412bb3f6 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-09-22 15:57:56 +02:00
Yann Collet
51f4d566c2 small decompression speed boost for very small data 2016-09-22 15:57:28 +02:00
Yann Collet
97b378a6f8 Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize. 2016-09-21 17:20:19 +02:00
Yann Collet
993060e0f2 cli : better adaptation to small files 2016-09-21 16:46:08 +02:00
Yann Collet
1eb2fdc74f bumped version number 2016-09-18 12:21:47 +02:00
Yann Collet
a6bdf55759 fixed memory leak 2016-09-15 17:02:06 +02:00
Yann Collet
644a8da88a fixed minor conversion warning 2016-09-15 16:16:21 +02:00
Yann Collet
4cb212938c introduced ZSTD_resetCStream() 2016-09-15 14:54:07 +02:00
Yann Collet
fa0c09760c variable renaming 2016-09-15 14:11:01 +02:00
Yann Collet
d7c6589df8 support ZSTD_sizeof_*() on NULL 
added ZSTD_sizeof_CDict()
 2016-09-15 02:57:27 +02:00
Yann Collet
e91c4b4cef introduced ZSTD_resetDStream() . 
added : ZSTD_sizeof_DDict()
 2016-09-14 16:55:44 +02:00
Yann Collet
d092d77cfc minor variable renaming 2016-09-14 16:14:57 +02:00
Yann Collet
64deef3bee Fixed srcSize=1 2016-09-14 00:16:07 +02:00
Yann Collet
26ec254066 new strategy for faster DDict decompression 2016-09-13 16:52:16 +02:00
Yann Collet
ac175d46d4 updated comments 2016-09-13 00:51:47 +02:00
Yann Collet
a3481d6de0 make uninstall 2016-09-12 05:04:26 +02:00
Yann Collet
b3060f7a9e changed streaming decoder behavior : now, when all compressed frame is consumed, it means decompression is completed, with regenerated data fully flushed. 2016-09-09 16:44:16 +02:00
Yann Collet
01c199226a updated decompression streaming example 2016-09-08 19:29:04 +02:00
Yann Collet
5c6d244973 Merge branch 'dev' of github.com:facebook/zstd into dev 2016-09-07 14:54:54 +02:00
Yann Collet
ac8bace6b1 support large skippable frames 2016-09-07 14:54:23 +02:00
Yann Collet
0e07bf3f60 added comments on searchLength min / max (#337) 2016-09-07 06:33:02 +02:00
Yann Collet
95d07d7447 introduced CHECK_E 2016-09-06 16:38:51 +02:00
Yann Collet
3e21ec5b01 introduced CHECK_F 2016-09-06 15:36:19 +02:00
Yann Collet
5c956d593c FORCE_INLINE common definition 2016-09-06 15:05:19 +02:00
Yann Collet
edbcd9f5b2 fixed zbufftest 2016-09-06 14:30:57 +02:00
Yann Collet
b624922b14 fixed checksum 2016-09-06 11:16:57 +02:00
Yann Collet
a7737f6a60 improved compression on small files when using same parameters 2016-09-06 09:44:59 +02:00
Yann Collet
7ae67bb18a small compression speed gains with using_CDict 2016-09-06 06:28:05 +02:00
Yann Collet
1d4208c029 clarified streaming decompression inlined doc 2016-09-06 05:16:40 +02:00
Yann Collet
7c83dfd5c2 ZSTD_frameHeaderSize_prefix (#340), as result of ZSTD_initStream 2016-09-05 19:47:43 +02:00
Yann Collet
fa72f6bdce clarified inline doc for streaming 2016-09-05 17:39:56 +02:00
Yann Collet
c73a8109bb Merge pull request #344 from inikep/dev10 
unified error codes for legacy decoders
 2016-09-05 07:46:33 -07:00
inikep
45db83f98d ZSTD_decodeLiteralsBlock renamed to ZSTDv01_decodeLiteralsBlock 2016-09-05 14:46:24 +02:00
inikep
476964f6a1 ZSTD_decodeSeqHeaders renamed to ZSTDv01_decodeSeqHeaders 2016-09-05 13:34:57 +02:00
inikep
c13faa1b0f legacy decoders: restored #include <intrin.h> for VC++ 2016-09-05 13:25:07 +02:00
inikep
8161e7321a unified error codes for legacy decoders 2016-09-05 12:29:51 +02:00
Thomas Klausner
b85cdabd50 Enable install targets for NetBSD. 2016-09-04 14:37:57 +02:00
Yann Collet
33a0465a51 fixed a few links 2016-09-02 22:11:49 -07:00
Yann Collet
d56dbc02d3 removed g_displayLevel 2016-09-02 17:28:41 -07:00
Yann Collet
855766d73d clarified dictionary in format description 2016-09-02 17:04:49 -07:00
Yann Collet
d725427a3c g_time => local displayTime 2016-09-02 15:32:39 -07:00
Yann Collet
1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
Yann Collet
7304eb7c09 bumped version number 2016-09-01 15:49:26 -07:00
Yann Collet
901e85fe26 version bump 2016-08-31 07:51:25 -07:00
Yann Collet
1c59c20903 removed redundant files 2016-08-31 07:15:44 -07:00
Yann Collet
599c69d917 minor Makefile updates 2016-08-30 13:33:20 -07:00
David Lam
e10f7f3dcb merge 2016-08-30 12:03:36 -07:00
Yann Collet
4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet
3b15f1f10f minor refactor 2016-08-30 09:58:50 -07:00
Yann Collet
240795bef7 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-30 06:51:55 -07:00
Yann Collet
14200a20f0 Fixed issue #304, reported by @borzunov 2016-08-30 06:51:00 -07:00
David Lam
da9d3b7057 Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c 2016-08-29 17:31:51 -07:00
Yann Collet
09c3c8e885 Merge pull request #307 from inikep/dev08 
updated README.md
 2016-08-29 16:32:33 -07:00
inikep
6416b0d705 updated README.md 2016-08-29 13:04:26 +02:00
Yann Collet
23b6e05d8e ZSTD_malloc() and ZSTD_free(), to simplify customMem 2016-08-28 21:05:43 -07:00
Yann Collet
5f53b0335e fixed continuation context 2016-08-28 10:00:49 -07:00
Yann Collet
767d8f66fa legacy contexts can be re-used 2016-08-28 08:19:47 -07:00
Yann Collet
4bf317dd00 first version supporting legacy streams (transparent decoding) 2016-08-28 07:43:34 -07:00
Yann Collet
e19a9ef05d update compression level table 2016-08-26 20:02:49 +02:00
Yann Collet
9a021c1aae fixed some minor clang warnings 2016-08-26 09:05:06 +02:00
Yann Collet
cb5a320705 made -Wdocumentation a clang only flag 2016-08-26 08:06:36 +02:00
Yann Collet
87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
Yann Collet
0d59a6f73a removed debug strings 2016-08-25 22:42:46 +02:00
Yann Collet
5a02b69215 reinforced fix for huge files 2016-08-25 22:24:59 +02:00
Yann Collet
96bdd87de4 fixed : compression bug on very large files 2016-08-25 14:34:42 +02:00
inikep
a3a47ec4d0 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other 2016-08-24 21:25:49 +02:00
Yann Collet
a2cdffe556 fixed wrong parameter issue 2016-08-24 19:42:15 +02:00
inikep
e416e30019 remove unnecessary comments 2016-08-24 17:32:09 +02:00
inikep
4e90f6c1e0 removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK 2016-08-24 17:24:11 +02:00
inikep
83388e109f removed ZSTD_LOG_PARSER 2016-08-24 17:22:20 +02:00
inikep
8a36f8527c removed stats in debug mode 2016-08-24 17:19:12 +02:00
inikep
57ef4b1a0d zstd_v07.c: removed unused macros 2016-08-24 17:16:56 +02:00
Yann Collet
24b68a5b23 update cLevel table for 256KB 2016-08-24 14:22:26 +02:00
Yann Collet
c54692faeb improved level 3 2016-08-24 01:45:46 +02:00
Yann Collet
17e482efdd added ZSTD_setDStreamParameter() 2016-08-23 16:58:10 +02:00
Yann Collet
3071c3e303 STREAM_WINDOW_MAX : protect streaming from unreasonable memory requirements 2016-08-23 01:34:34 +02:00
Yann Collet
70e3b31306 fixed playtests on os-x 2016-08-23 01:18:06 +02:00
Yann Collet
cb3276329a added sizeof CStream and DStream 2016-08-23 00:31:59 +02:00
Yann Collet
d1733f7417 fixed crc bug in rare timing conditions within bench.c 2016-08-21 01:04:46 +02:00
Yann Collet
8baf78a291 minor coding style 2016-08-20 13:04:20 +02:00
Yann Collet
1bee2d5e08 slight decompression speed improvement 2016-08-20 02:59:04 +02:00
Yann Collet
0cfe2ec2fd sync fse version 2016-08-20 00:26:26 +02:00
Yann Collet
af1960396b sync huff0 2016-08-19 19:38:19 +02:00
Yann Collet
7be46bf8f2 promoted streaming API to stable 
(except _advanced() variants)
 2016-08-19 18:39:36 +02:00
Yann Collet
fdba57d513 update version number 2016-08-19 18:32:30 +02:00
Yann Collet
da3fbcb302 Added ZDICT_getDictID() 2016-08-19 14:23:58 +02:00
Yann Collet
a5dbf9f629 Merge pull request #297 from borzunov/dev 
Export functions related to dictionary compression from DLL
 2016-08-18 15:05:01 +02:00
Yann Collet
49d105cfcf better warning and error messages in case of dictionary training failure (#292) 2016-08-18 15:02:11 +02:00
Alexander Borzunov
0f6f17a14f Rename ZSTDLIB_API to ZDICTLIB_API in zdict.h 2016-08-18 16:47:06 +05:00
Alexander Borzunov
1f48382b1a Export functions related to dictionary compression from DLL 2016-08-18 16:12:49 +05:00
Yann Collet
e80d15304a Merge pull request #296 from inikep/Other 
Other
 2016-08-18 11:48:48 +02:00
inikep
a7bb322a93 removed never referenced functions 2016-08-18 10:30:21 +02:00
Yann Collet
18442c1482 minor refactoring 2016-08-18 01:40:32 +02:00
Yann Collet
c411902230 fixed g++ conversion warning 2016-08-17 01:50:54 +02:00
Yann Collet
53e17fbd5e updated streaming API 2016-08-17 01:39:22 +02:00
Yann Collet
655393cc72 updated doc for streaming API 2016-08-16 15:11:28 +02:00
Yann Collet
104e5b072d added : streaming decompression API 2016-08-16 15:11:28 +02:00
Yann Collet
5a0c8e2439 new streaming API (compression) 2016-08-16 15:11:27 +02:00
Yann Collet
ba92046031 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-11 22:10:04 +02:00
Yann Collet
e9b414d825 fixed msan warning (#281) 2016-08-11 22:09:09 +02:00
inikep
5f49eba512 added usage of rep[0]-1 for the optimal parser 2016-08-10 15:01:53 +02:00
inikep
98e08cbe34 fixed: tree not updated after finding very long rep matches 2016-08-10 15:00:30 +02:00
inikep
038d1497c9 fixed compilation with Visual Studio 2005 2016-08-10 14:30:10 +02:00
inikep
48849f86f0 fixed compilation with Intel Compiler with Windows 2016-08-10 14:26:35 +02:00
Yann Collet
1ea5622a32 updated xxhash 2016-08-10 09:40:08 +02:00
Yann Collet
666398e7ed added : xxhash namespace enforced from xxhash.h. 
added : xxhash namespace test.
removed : -DXXH_NAMESPACE
 2016-08-10 08:16:51 +02:00
Yann Collet
8ded0b84aa update xxhash to v0.6.2 2016-08-10 07:40:40 +02:00
Yann Collet
280f9a8754 minor comment 2016-08-08 00:44:00 +02:00
Yann Collet
e0b4a2d40f fixed dictionary generation, reported by Bartosz Taudul 2016-08-03 03:36:03 +02:00
Yann Collet
ae40b18d55 bumped library number 2016-08-03 01:59:21 +02:00
Yann Collet
0763905f44 ZSTD_compress_usingCDict() correctly provides original size by default in frame header 
Fixed dictionary examples
 2016-08-03 01:57:57 +02:00
Yann Collet
bf2bc112bb bench : controlled display update when loading lot of files 2016-08-02 23:48:13 +02:00
Yann Collet
346efccc35 fixed doc typo 2016-08-02 14:26:00 +02:00
Yann Collet
f116e87f59 fixed analyzer warning 2016-08-01 19:15:18 +02:00
Yann Collet
9ba929f1d4 Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-08-01 02:26:59 +02:00
Yann Collet
3ca750372d updated doc (#269) 2016-08-01 02:26:20 +02:00
Yann Collet
c55eb18c11 Merge pull request #267 from inikep/dev08 
fixed ZSTD_compressBlock_opt_extDict_generic
 2016-07-31 22:00:16 +02:00
inikep
056df510aa fixed ZSTD_compressBlock_opt_extDict_generic 2016-07-31 20:08:53 +02:00
Yann Collet
917fe188f1 Implemented repOffset "minus 1" on ll==0 2016-07-31 04:01:57 +02:00
Yann Collet
2a2ba3691c Merge pull request #266 from jrmarino/master 
Enable build on FreeBSD ports (includes DragonFly BSD) [dev branch]
 2016-07-31 02:21:26 +02:00
jrmarino
0d07ec0c0c Enable build on FreeBSD ports (includes DragonFly BSD) 
Zstd has been introduced to FreeBSD ports
(http://www.freshports.org/archivers/zstd/) which DragonFly BSD also
uses.  FreeBSD and DragonFly use the install targets (albeit modified in
some cases) so they must be added to the associated Makefile filters.
 2016-07-30 19:11:15 -05:00
Yann Collet
8cebfd1d26 fix attempt on test-zstd-speed 2016-07-31 01:59:23 +02:00
Yann Collet
66f69e58d2 restore decompression speed on fizzle 2016-07-30 15:32:47 +02:00
Yann Collet
3b2bd1d11c zstd_opt uses same tables as zstd_compress 2016-07-30 13:21:41 +02:00
Yann Collet
f714f59c16 fixed visual warning 2016-07-30 12:05:28 +02:00
Yann Collet
761f8dbbd2 back to normal table cell copy 2016-07-30 11:43:53 +02:00
Yann Collet
3c6b808870 minor decompression speed gains 2016-07-30 03:20:47 +02:00
Yann Collet
70a9ff4af3 fixed too large selectivity level, reported by Ilona Papava 2016-07-30 01:09:14 +02:00
Yann Collet
c0ce4f1211 slightly improved compression speed 2016-07-30 00:55:13 +02:00
Yann Collet
ed57d8530a new seqStore 2016-07-29 21:22:17 +02:00
Yann Collet
6b615d32cd Updated API comments, following suggestions by Bryan O'Sullivan 2016-07-29 19:40:37 +02:00
Yann Collet
c00d30fbe4 Merge pull request #264 from inikep/dev08 
Dev08
 2016-07-29 17:42:30 +02:00
inikep
6b68ba2079 zstd_opt.h: fixed checking of rep codes (2) 2016-07-29 16:45:39 +02:00
inikep
59b86fc141 zstd_opt.h: fixed checking of rep codes 2016-07-29 11:00:33 +02:00
Yann Collet
6a82f0f8bf minor comments 2016-07-29 00:55:45 +02:00
Yann Collet
ffa7d0ac1e clarified comment 2016-07-28 21:01:17 +02:00
Yann Collet
4c5bbf64f9 fixed : frame concatenation without checksum 2016-07-28 20:30:25 +02:00
Yann Collet
60ba31c570 zbuff uses ZSTD_compressEnd() 2016-07-28 19:55:09 +02:00
Yann Collet
16e73033ad introduced stage zbf_end 2016-07-28 16:32:34 +02:00
Yann Collet
62470b4bab Changed ZSTD_compressEnd() 2016-07-28 15:29:08 +02:00
Yann Collet
e7bf9156d1 Clarified API comments, from suggestions by ‎Bryan O'Sullivan‎ 2016-07-28 05:00:57 +02:00
Yann Collet
d469a98c01 Clarified API comments, from suggestions by ‎Bryan O'Sullivan‎ 2016-07-28 04:55:03 +02:00
Yann Collet
19c1002e46 applied ZSTD_compressContinueThenEnd() 2016-07-28 01:25:46 +02:00
Yann Collet
5b56739b63 created ZSTD_compressContinueThenEnd() 2016-07-28 01:17:22 +02:00
Yann Collet
c991cc1828 new frame end, 32-bits checksums 2016-07-28 00:55:43 +02:00
Yann Collet
d4180cad9c minor code refactoring 2016-07-27 21:21:36 +02:00
Yann Collet
731ef16fc1 minor code style refactoring 2016-07-27 21:05:12 +02:00
Yann Collet
4b9ca0a6b5 minor example variation 2016-07-27 19:53:19 +02:00
Yann Collet
4110534886 ZSTD_maxCLevel() is promoted to "stable" API (#254, by @FrancescAlted) 2016-07-27 15:09:11 +02:00
Yann Collet
55a8bea0b5 fixed dictionary generation 2016-07-27 14:48:47 +02:00
Yann Collet
c154d9d6a2 better support for large dictionaries (> 128 KB) 2016-07-27 14:37:00 +02:00
Yann Collet
07626dfa51 improved dictbuilder notifications on selectivity 2016-07-27 13:28:46 +02:00
Yann Collet
f796f7ab45 removed fastscan mode 2016-07-27 12:53:54 +02:00
Yann Collet
dd25a27702 added tutorial warning messages for dictBuilder 2016-07-27 12:43:09 +02:00
inikep
003c7a8568 optimal parser: removed ZSTD_REP_INIT 2016-07-27 11:07:13 +02:00
Yann Collet
04cdd8660d Merge pull request #262 from ebiggers/misc_updates 
Miscellaneous updates
 2016-07-27 01:25:45 +02:00
Eric Biggers
0a55e7a0bb ZSTD_decompressFrame(): use remainingSize instead of iend - ip 
Same behavior, but no need to have redundant variables.
 2016-07-26 13:22:27 -07:00
Eric Biggers
aa6c70bf60 ZSTD_decompressFrame(): pass up error code from ZSTD_decodeFrameHeader() 2016-07-26 13:22:27 -07:00
Eric Biggers
e4d0265ea9 Replace remaining references to "direct mode" with "single segment mode" 2016-07-26 13:22:27 -07:00
Yann Collet
d50f9db3ea Improved speed on clang and gcc -O2, thanks to @ebiggers ! (#263) 2016-07-26 21:30:35 +02:00
Yann Collet
7adc2328a3 fixed --test on zero-length files, reported by @amnilsson 2016-07-26 15:49:24 +02:00
inikep
4178f5c289 fixed gcc warning: always_inline function might not be inlinable 2016-07-25 21:17:45 +02:00
inikep
fca90f8f60 legacy decoder for v0.7 format 2016-07-25 17:49:08 +02:00
Yann Collet
cbc5e9dc19 fixes oob read 2016-07-24 18:02:04 +02:00
Yann Collet
38b75ddeb2 removed special case all-1 huffman distribution 2016-07-24 15:35:59 +02:00
Yann Collet
7ed5e33b89 minor comment changes 2016-07-24 14:26:11 +02:00
Yann Collet
10b9c13d07 fixed doc on cLevel default, reported by Oliver Lange 2016-07-24 01:21:53 +02:00
Yann Collet
f8e7b5363f unified encoding types 2016-07-23 16:31:49 +02:00
Yann Collet
571a59034a changed enccoding type order : raw, rle, compressed, repeat-stats 2016-07-23 15:52:05 +02:00
Yann Collet
c2e1a68d81 changed streamNb order to 1-4-4-4 2016-07-22 17:30:52 +02:00
Yann Collet
772d912c2f more complete support for literals repeat mode 2016-07-22 15:04:25 +02:00
Yann Collet
9f2d82d4a4 fixed : big-endian decoding 2016-07-22 14:37:10 +02:00
Yann Collet
32faf6c8e7 fixed conversion warnings 2016-07-22 14:37:09 +02:00
Yann Collet
5e45a5fbb3 force loop-align to 32 for zstd_decompress 2016-07-22 14:37:09 +02:00
Yann Collet
5288ac0cb7 changed filed order 2016-07-22 14:37:09 +02:00
Yann Collet
198e6aac44 Literals header fields use little endian convention 2016-07-22 14:37:09 +02:00
Yann Collet
6fa05a2371 cBlockSize uses little-endian convention 2016-07-22 14:37:09 +02:00
Yann Collet
7bf72bbf5e update header to v0.8 2016-07-22 14:37:09 +02:00
Yann Collet
5894ea8d01 updated cLevels 2016-07-22 14:36:46 +02:00
Yann Collet
d5c5a77990 minor comments clarifications 2016-07-20 13:35:14 +02:00
Yann Collet
572b817be3 Merge pull request #253 from gymdis/heapmode_off_legacy_fix 
Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0
 2016-07-19 13:52:03 +02:00
Christopher Bergqvist
780a9fa857 Fix compile issue with ZSTD_LEGACY_SUPPORT=1 and ZSTD_HEAPMODE=0 2016-07-19 13:25:38 +02:00
Yann Collet
cf05b9d477 ZSTD_getBlockSizeMax() 2016-07-18 16:52:10 +02:00
Yann Collet
16aa38b0e0 minor doc clarifications 2016-07-18 03:52:47 +02:00
Yann Collet
85f3919960 moved zstd.h to /lib 2016-07-17 20:42:21 +02:00
Yann Collet
9375590462 update version to v0.7.5 2016-07-17 16:44:18 +02:00
Yann Collet
e557fd5e92 minor compression level corrections 2016-07-17 16:21:37 +02:00
Yann Collet
d54b2d23b4 minor static assert for 32/64 bits system. Suggested by @ebiggers 2016-07-17 15:53:18 +02:00
Yann Collet
972e5806ee fixed : premature frame end on zero-sized raw block - reported by @ebiggers 2016-07-17 15:39:24 +02:00
luben karavelov
10f999f856 Add legacy support for the low-level streaming API 2016-07-17 01:03:26 +02:00
Yann Collet
6cacd34d44 minor formatting changes 2016-07-15 17:58:13 +02:00
Yann Collet
f6ff53cd4e implemented dictID reserved ranges 2016-07-15 17:03:38 +02:00
Yann Collet
98c8884999 added target zstd in root Makefile 2016-07-15 16:12:38 +02:00
Yann Collet
961b6a0e34 ZSTD_compressBlock() limits block size depending on windowLog parameter 2016-07-15 11:58:49 +02:00
Yann Collet
227cc39e15 improved efficiency for large messages with small dictionaries 2016-07-15 11:27:09 +02:00
Yann Collet
ea2ecdc315 fixed issue with small dictionary 2016-07-14 23:27:31 +02:00
Yann Collet
e9ed5cdc94 fixed minor coverity warning 2016-07-14 21:02:57 +02:00
Yann Collet
b23e1ce319 removed debugging traces 2016-07-14 17:46:38 +02:00
Yann Collet
17508f1a16 fixed a few minor coverity warnings 2016-07-14 17:18:20 +02:00
Yann Collet
8847238cac simplified ZSTD_estimateCCtxSize() 2016-07-14 17:05:38 +02:00
Yann Collet
69c2cdb45c fixed conversion warning 2016-07-14 16:52:45 +02:00
Yann Collet
5e80dd3261 fixed minor coverity warnings 2016-07-13 19:21:57 +02:00
Yann Collet
3c174f4da9 fixed minor coverity warning 2016-07-13 17:25:53 +02:00
Yann Collet
2b1a3638e6 changed macro name to ZSTDCLI_CLEVEL_DEFAULT 2016-07-13 15:16:00 +02:00
Yann Collet
3c242e79d3 updated compression levels table 2016-07-13 14:56:24 +02:00
Yann Collet
fbc69f8649 changed for #245 2016-07-13 13:52:58 +02:00
Yann Collet
eed2081e55 fixed conversion warning 2016-07-12 15:11:40 +02:00
Yann Collet
a43a854cdb updated paramgrill 2016-07-12 13:42:10 +02:00
Yann Collet
73d74a05b9 fixed dfast strategy 2016-07-12 13:03:48 +02:00
Yann Collet
45dc35628c first version of doubleFast 2016-07-12 09:47:31 +02:00
Yann Collet
d158c35e9f added ZSTD_estimateDCtxSize() 2016-07-11 13:46:25 +02:00
Yann Collet
8e0ee681b8 added ZSTD_sizeofDCtx() 2016-07-11 13:09:52 +02:00
Yann Collet
3ae543ce75 added ZSTD_estimateCCtxSize() 2016-07-11 03:12:17 +02:00
Yann Collet
25c506601c promote ZSTD_getDecompressedSize() to stable API 2016-07-10 01:46:18 +02:00
Yann Collet
3b6ae77e15 comment clarification 2016-07-08 23:42:22 +02:00
Yann Collet
722e14bb65 fixed compilation error in decompression module 2016-07-08 19:22:16 +02:00
Yann Collet
bd10607063 updated spec 2016-07-08 19:16:57 +02:00
Yann Collet
c5fb5b7fcd support offset > 128 MB 2016-07-08 13:13:37 +02:00
Yann Collet
ed3845d3fa introduced ZSTD_WINDOWLOG_MAX_32 (#239), suggested by @GregSlazinski 2016-07-08 12:57:10 +02:00
Yann Collet
26f681451f updated doc 2016-07-08 11:45:08 +02:00
Yann Collet
19c27d27f1 simplified legacy functions, no longer need magic number 2016-07-07 14:40:13 +02:00
Yann Collet
e72efeb0a1 removed "error_public.h" dependency from "zstd.h" 2016-07-07 14:17:40 +02:00
Yann Collet
974f52fc5d Added "dictionary decompression" example 2016-07-07 14:08:00 +02:00
Yann Collet
e09d38e921 removed mem.h dependency from zbuff.h (experimental section) 2016-07-07 13:17:37 +02:00
Yann Collet
f323bf7d32 added : ZSTD_getDecompressedSize() 2016-07-07 13:14:21 +02:00
Yann Collet
52c04fe58f removed mem.h dependency from zstd.h (experimental section) 2016-07-07 11:53:18 +02:00
Yann Collet
f246cf5423 ZSTD_decompress_usingDDict() compatible with Legacy mode 2016-07-06 20:32:27 +02:00
Yann Collet
29652e2618 sample set limitation closer to 2 GB 2016-07-06 16:25:46 +02:00
Yann Collet
99b045b70a dictBuilder protection vs huge sample sets (>2 GB) 2016-07-06 16:12:38 +02:00
Yann Collet
445d49d898 fixed conversion warning 2016-07-06 13:27:22 +02:00
Yann Collet
a295b3170f fixed conversion warning 2016-07-06 13:13:12 +02:00
Yann Collet
517e1ba623 fixed dictBuilder issue with HC levels. Reported by Bartosz Taudul. 2016-07-06 12:35:09 +02:00
Yann Collet
fe07eaa972 simplified ZSTD_decodeSequence() 2016-07-06 02:25:44 +02:00
Yann Collet
9ca73364e6 updated spec 2016-07-05 10:53:38 +02:00
Yann Collet
f9cac7a734 Added GNU separator --, to specifies that all following arguments are necessary file names (and not commands). Suggested by @chipturner (#230) 2016-07-04 18:18:24 +02:00
Yann Collet
23f05ccc6b updated specifications 2016-07-04 16:13:11 +02:00
Yann Collet
d916c908e0 updated doc 2016-07-04 00:42:58 +02:00
Yann Collet
698cb63305 Updated specifications 2016-07-03 18:49:35 +02:00
Yann Collet
d57dffbe76 ZSTD_storeSeq takes an U32 as offset type 2016-07-03 01:48:26 +02:00
Yann Collet
302ff036f6 simplified repcodes for lazy_extDict 2016-07-03 01:28:16 +02:00
Yann Collet
9634f67107 fix lazy parser 2016-07-03 01:23:58 +02:00
Yann Collet
92d75667e4 fix for fast mode 2016-07-03 01:10:53 +02:00
Yann Collet
5e734ad09b revert fix 2016-07-02 23:55:34 +02:00
Yann Collet
0d5bf8f06f fixed risk of segfault on very large files (multiple GB) 2016-07-02 21:39:47 +02:00
Yann Collet
2fa9904844 update specification and comments 2016-07-01 20:55:28 +02:00
Yann Collet
c093208ab8 fix : potential leak (#229) 2016-06-30 14:07:30 +02:00
Yann Collet
6c6e1751f6 use ZSTD_getParams() to simplify code 2016-06-27 15:28:45 +02:00
Yann Collet
3d2cd7f816 Introduced ZSTD_getParams() 
bench now uses ZSTD_createCDict_advanced()
 2016-06-27 15:12:26 +02:00
Yann Collet
529d9c7dee updated version to v0.7.2 2016-06-27 10:03:10 +02:00
Yann Collet
d4f4e58ee1 fixed ZSTD_decompressBlock() using multiple blocks 2016-06-27 01:31:35 +02:00
Yann Collet
63b5e7a2ea Improved comments 2016-06-26 17:42:15 +02:00
Yann Collet
3755eb8fea fixed strict-aliasing warning on gcc6 2016-06-22 13:15:53 +02:00
Yann Collet
23042929da Fixed : dictBuilder fails if first sample is too small 2016-06-22 11:05:34 +02:00
Yann Collet
391a128794 fix : segfault in command line during automatic overwrite protection mode 2016-06-21 17:06:25 +02:00
Yann Collet
bda68c253b refactored ZBUFF_compressEnd() for better maintainability 2016-06-21 15:18:11 +02:00
Yann Collet
aa29226b7c fix : ZBUFF_compressEnd() gives right amount remaining to flush, including future epilogue 2016-06-21 14:04:57 +02:00
Yann Collet
f15c1cb00c Fixed : ZBUFF_compressEnd() called multiple times with too small dst buffer (#206) 2016-06-21 13:11:48 +02:00
Yann Collet
a49e066b26 clarified comments on ZSTD_compressContinue() 2016-06-21 11:54:03 +02:00
Yann Collet
d4f38d0dcd updated library to v0.7.1 2016-06-21 10:15:43 +02:00
Yann Collet
22d76322ce minor refactor 2016-06-21 08:01:51 +02:00
Yann Collet
a436a529bc minor : fast_extDict does no longer skip first byte 2016-06-20 23:34:04 +02:00
Yann Collet
4623d11571 new correction, less extreme replacement value 2016-06-20 19:15:37 +02:00
Yann Collet
5477cc25f7 fixed corruption error related to inter-blocks rep-offset 2016-06-20 18:31:25 +02:00
Yann Collet
e4811ba761 Modified : ZSTD_createDDict() accepts dictionary < 8 bytes in pure content mode (reported by @chipturner) 2016-06-19 23:06:54 +02:00
Yann Collet
06d9a73b48 minor refactor, using WILDCOPY_OVERLENGTH macro instead of hard-coded 8 2016-06-19 14:27:21 +02:00
Yann Collet
19cab46f2f Joined seqStore initialization at dispatch point 2016-06-17 12:54:52 +02:00
Yann Collet
510cff3570 minor comment change 2016-06-16 16:39:55 +02:00
Yann Collet
4948f270b3 make room for reserved "information bit" in frame header 2016-06-16 15:38:51 +02:00
Yann Collet
23ba41533a Fixed zstd_opt encoding error with repeat-offsets 2016-06-16 13:20:46 +02:00
Yann Collet
80d033fb43 fixed ptr arithmetic warning 2016-06-16 01:41:50 +02:00
Yann Collet
ad39b7a718 zdict stores standard rep-offset. 
It can use custom ones, but the proper formula and impact on statistics is not done yet.
 2016-06-16 01:14:41 +02:00
Yann Collet
736d419289 strengthened dict loading on decompresson side 2016-06-16 01:05:04 +02:00
Yann Collet
8e36a9c169 decoder restores repOffsets from dictionary 2016-06-16 01:05:04 +02:00
Yann Collet
52a0622beb RepsCodes are saved into Dict 
(uncomplete : need decompression to regenerate them)
 2016-06-16 01:05:04 +02:00
Yann Collet
efd0b4993a fixed fuzzer error (inter-block repeated offsets) 2016-06-16 00:53:56 +02:00
Yann Collet
9b998e4d08 Fixed decompression of literals in dictionary mode 2016-06-15 23:11:20 +02:00
Yann Collet
d059092897 fixed conversion warnings 2016-06-14 15:34:24 +02:00
Yann Collet
45c03c564f fixed corruption with inter-blocks repeated offsets 2016-06-14 13:46:11 +02:00
Yann Collet
4266c0a2fd adding inter-blocks rep-offsets 2016-06-14 01:49:25 +02:00
Yann Collet
43dfe01919 Check repIndex for validity 2016-06-13 21:43:06 +02:00
Yann Collet
18c8f79f3e fixed gcc warning on uninitialized structure variable 2016-06-12 22:51:52 +02:00
Yann Collet
cd98f93cff Fixed decompression issue with invalid data 2016-06-11 23:26:22 +02:00
Yann Collet
37fece22e8 enable repeat-entropic-stats mode 2016-06-11 02:52:42 +02:00
Yann Collet
d60a5bf900 Literal decompression builds Huffman tables within shared space (for later re-use) 2016-06-11 02:35:31 +02:00
Yann Collet
237ad4beb3 Added single-stream decompression variant using external DTable 2016-06-11 01:46:03 +02:00
Yann Collet
289bbd52e5 Updated huff0 2016-06-11 01:31:54 +02:00
Yann Collet
1869f7966e Merge pull request #205 from inikep/dev 
legacy decoder for v0.6
 2016-06-10 17:13:07 +02:00
Yann Collet
0974f681a4 completed .gitignore 2016-06-10 14:44:16 +02:00
Yann Collet
9dd12742f3 litBlockType_t is an enum 2016-06-10 00:12:26 +02:00
inikep
4923222412 fixed warnings from Travis 2016-06-09 20:03:30 +02:00
inikep
4000945a1d project updated for legacy decoder zstd_v06.c 2016-06-09 18:12:06 +02:00
inikep
bf853d5510 added legacy decoder for v0.6 format 2016-06-09 17:59:18 +02:00
Yann Collet
662a541431 updated huff0 - now generates a common HUF_DTable type for all decoding tables 2016-06-08 11:11:02 +02:00
Yann Collet
302fb53a76 Removed ZSTD_*_usingPrepared?Ctx() declaration from public space 2016-06-07 12:16:49 +02:00
Yann Collet
81e13ef7cf first implementation of the new dictionary API (untested) 2016-06-07 00:51:51 +02:00
Yann Collet
9d504ae85b Added decoding of RLE blocks 2016-06-06 19:52:35 +02:00
Yann Collet
2cc72f1fd3 fixed initialization issue in bench 2016-06-06 17:50:07 +02:00
Yann Collet
e3d529403d fixed initialization mismatch in ZSTD_copyCCtx() 2016-06-06 11:07:33 +02:00
Yann Collet
142acbdea7 fixed minor visual conversion warning 2016-06-06 00:46:56 +02:00
Yann Collet
673f0d7cdc new frame format, allowing custom window size 2016-06-06 00:26:38 +02:00
Yann Collet
89703d20fb reduced dependencies 2016-06-05 01:50:33 +02:00
Yann Collet
51778b7cca updated README following merging of *_static.h 2016-06-05 01:38:10 +02:00
Yann Collet
a91ca620cf removed HUF_readStats() from public space 2016-06-05 01:33:55 +02:00
Yann Collet
d0e2cd15cb Merged fse_static into fse.h . Now requires FSE_STATIC_LINKING_ONLY macro. 2016-06-05 00:58:01 +02:00
Yann Collet
130fe11394 merged huf_static.h into huf.h . Requires HUF_STATIC_LINKING_ONLY macro. 2016-06-05 00:42:28 +02:00
Yann Collet
dc048d18d3 minor comment (detailing an #include motivation) 2016-06-05 00:32:23 +02:00
Yann Collet
49bb0041af removed ZSTD_highbit() from zstd_internal.h, as it is only used by zstd_compress.c 2016-06-04 20:17:38 +02:00
Yann Collet
d3b7f8d21f Merged zstd_static.h into zstd.h . Now requires ZSTD_STATIC_LINKING_ONLY macro 2016-06-04 19:47:02 +02:00
Yann Collet
ac110a1f21 Removed ZBUFF internal util function from public area 2016-06-04 19:16:49 +02:00
Yann Collet
5347aee8f7 merged zbuff_static.h into zbuff.h . Now requires ZBUFF_STATIC_LINKING_ONLY macro 2016-06-04 19:12:48 +02:00
Yann Collet
e69b8ccceb merged zdict_static.h into zdict.h. Now requires ZDICT_STATIC_LINKING_ONLY macro. 2016-06-04 18:56:23 +02:00
Yann Collet
198d127b35 minor comment change (unfinished description of new header format) 2016-06-04 18:40:55 +02:00
Yann Collet
f4f5affdf7 restore ZBUFF full-block-size, for better performance on small input 2016-06-03 23:09:28 +02:00
Yann Collet
ab7b6f1ece Merge pull request #198 from inikep/dev070 
Dev070
 2016-06-03 21:37:49 +02:00
inikep
3640396b1a fixed: deallocation of structures in case of error in ZBUFF_createCCtx and ZBUFF_createDCtx 2016-06-03 16:36:50 +02:00
Yann Collet
fe48775868 minor decoder code refactoring 2016-06-03 15:41:51 +02:00
inikep
2a74609b90 zlibWrapper: ZWRAP_createCCtx and ZWRAP_freeCCtx use custom memory allocation functions 2016-06-03 14:53:51 +02:00
inikep
3763c77f6b defaultCustomNULL replaced with defaultCustomMem 2016-06-03 13:28:20 +02:00
inikep
36fac00149 removed calloc calls from lib/ 2016-06-03 13:23:04 +02:00
inikep
db2f540414 added defaultCustomNULL 2016-06-03 12:56:56 +02:00
inikep
b74a468fad Merge remote-tracking branch 'refs/remotes/Cyan4973/dev070' into dev070 2016-06-02 22:09:09 +02:00
Yann Collet
923938edde Added -Wdeclaration-after-statement compilation flag 2016-06-02 17:56:00 +02:00
inikep
ff9114aee3 zlibWrapper: added support for custom memory allocation functions 2016-06-02 16:52:36 +02:00
inikep
c4807f4d2f default custom allocation functions moved to zstd_internal.h 2016-06-02 15:11:39 +02:00
inikep
2866951558 opaque parameter for custom memory allocation functions 2016-06-02 13:04:18 +02:00
inikep
9242816b56 fparamsPtr->windowLog==0 means that a frame is skippable 2016-06-01 18:47:04 +02:00
Yann Collet
70d1301d6e Changed ZSTD_adjustCParams() prototype 
`ZSTD_adjustCParams()` is now automatically invoked at the end of `ZSTD_getCParams()`
 2016-06-01 18:45:34 +02:00
Yann Collet
83c3f4427c upgraded zbufftest to also test advanced frame parameters no/checksum no/dictID 2016-06-01 17:44:53 +02:00
inikep
13f42d9085 VS2010 project: reverted zstdlib.rc 2016-06-01 14:44:31 +02:00
inikep
5c2771710d Merge remote-tracking branch 'refs/remotes/Cyan4973/dev070' into dev070 
# Conflicts:
#	.gitignore
#	lib/decompress/zstd_decompress.c
#	programs/zbufftest.c
 2016-06-01 09:16:11 +02:00
Yann Collet
202844ebd0 fixed zbufftest : 2016-06-01 00:44:36 +02:00
Yann Collet
8e3a36a6db decompression validates frame content checksum 2016-06-01 00:18:28 +02:00
inikep
a6b942018d Merge remote-tracking branch 'refs/remotes/origin/dev' into dev070 
# Conflicts:
#	.travis.yml
#	Makefile
#	lib/common/zstd_static.h
#	programs/Makefile
#	projects/VS2008/zstd/zstd.vcproj
#	projects/VS2008/zstdlib/zstdlib.vcproj
#	projects/cmake/lib/CMakeLists.txt
#	projects/cmake/programs/CMakeLists.txt
 2016-06-01 00:07:09 +02:00
Yann Collet
f2a3b6e7b4 added : frame content checksum 2016-05-31 22:23:45 +02:00
inikep
43aa9fe8b3 fixed skippable frame 2016-05-31 19:36:51 +02:00
inikep
f772bf54a5 support for skippable frames 2016-05-31 12:43:46 +02:00
Giuseppe Ottaviano
370b751e24 Expose function to add entropy tables to pre-built dictionary. 
In some cases a custom dictionary building algorithm tailored for a specific
input can be more effective than the one produced by `ZDICT_trainFromBuffer`,
but with the current API it's not possible encode the entropy tables into the
custom-built dictionary.

This commit extracts the logic to add entropy tables to a dictionary from
`ZDICT_trainFromBuffer` and exposes it as a function
`ZDICT_addEntropyTablesFromBuffer`.
 2016-05-30 19:50:09 -07:00
Yann Collet
290aaa7521 Added : ability to manually select the dictionary ID of a newly created dictionary 2016-05-30 21:18:52 +02:00
Yann Collet
30009521d7 fuzzer tests dictBuilder. 
Added : ability to not store dictID during compression; decompression doesn't check dictID then
 2016-05-30 16:17:33 +02:00
Yann Collet
c0a9bf3c2e minor code refactoring 2016-05-30 04:48:32 +02:00
Yann Collet
c46fb924df added dictionary ID (incomplete) 2016-05-29 05:01:04 +02:00
Yann Collet
f51e0660f4 Simplified list of *.c files 2016-05-29 01:39:19 +02:00
Yann Collet
0c5e8b17ad moved xxhash to lib/common 2016-05-29 01:06:30 +02:00
inikep
957823f56f zstdcli: -r (operate recursively on directories) works with dictBuilder and compression 2016-05-25 15:30:55 +02:00
inikep
e75909ef93 ZBUFF_createCCtx_advanced moved to zbuff_static.h 2016-05-25 11:31:16 +02:00
inikep
02c244bf78 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-05-24 17:15:04 +02:00
inikep
fb5df613cf zstd_stats.h included only in debug mode 2016-05-24 15:36:37 +02:00
inikep
71dd67cf57 fullbench: tests for ZBUFF_createCCtx_advanced and ZBUFF_createDCtx_advanced 2016-05-24 10:57:14 +02:00
inikep
13ba880b49 added ZBUFF_createCCtx_advanced and ZBUFF_createDCtx_advanced 2016-05-23 17:04:23 +02:00
inikep
107e243195 added ZSTD_createDCtx_advanced 2016-05-23 16:24:52 +02:00
inikep
50e82c015d added ZSTD_createCCtx_advanced 2016-05-23 15:49:09 +02:00
inikep
35b891c5b0 zstd_stats.h updated to v0.6 2016-05-20 19:42:20 +02:00
Yann Collet
f22a0d653d huff0 dynamic reduction 2016-05-20 14:36:36 +02:00
Yann Collet
45f84ab7cc all programs use ZSTD_VERSION_STRING from zstd.h to display their version 2016-05-20 12:34:40 +02:00
Yann Collet
de4d4da52d changed version to 0.6.2 
removed make-controlled version (now used from zstd.h)
removed std=c99 by default
added : c90/gno90/c99/gnu99/c11 tests
 2016-05-20 12:20:12 +02:00
Yann Collet
7591a7f224 minor refactoring 2016-05-20 11:44:43 +02:00
Yann Collet
236d94fa9a reverted default compression level to 1 2016-05-18 12:06:33 +02:00
Yann Collet
78aa05228a fixed Visual warning in debug mode 2016-05-17 14:30:19 +02:00
Yann Collet
c8e9c4046f Merge branch 'dev' of github.com:Cyan4973/zstd into dev 2016-05-17 12:02:35 +02:00
Yann Collet
ea4daa308d added ZSTDv05_getFrameParams() to "zstd_v05.h" 2016-05-17 12:01:55 +02:00
klemens
268775dae8 spelling-fixes, as of lintian.debian.org 2016-05-16 22:23:24 +02:00
Yann Collet
d9b3cca4d3 restore openVMS support (detected by @thatsafunnyname) 2016-05-13 12:43:36 +02:00
inikep
63ecd747de added common/entropy_common.c 2016-05-13 11:27:56 +02:00
Yann Collet
ccd6e86d45 improved decoding speed by inlining x_isError() functions 2016-05-12 15:55:26 +02:00
Yann Collet
16871680e3 minor refactor : more accurate variable scope 2016-05-12 14:27:00 +02:00
Yann Collet
b19d5d76ec restored inline for better decompression speed 2016-05-12 14:06:31 +02:00
Yann Collet
1ceb5a9295 restored DStream init 
due to strange speed regressions and warnins on gcc 5.3
 2016-05-12 13:50:13 +02:00
Yann Collet
ce27e85209 fixed clang conversion warning 2016-05-11 18:55:27 +02:00
Yann Collet
1032fbe714 update FSE library 2016-05-11 18:30:24 +02:00
Yann Collet
249ae0ca99 better big-endian / little-endian r/w support 2016-05-11 16:38:18 +02:00
Yann Collet
3329d98df9 fixed minor asan warning in legacy decoder 2016-05-11 11:07:54 +02:00
Yann Collet
c75e4c25e8 Added : ZSTD_getErrorString(), to get error string from error enum (#168) 2016-05-10 17:47:11 +02:00
Yann Collet
b3a9938d0c Merge pull request #182 from inikep/dev 
support for Visual Studio 2012, 2013, 2015
 2016-05-10 17:18:12 +02:00
Yann Collet
bd39d54576 zbuff supports byte-by-byte decompression scenarios 2016-05-10 14:14:19 +02:00
inikep
ed9a08538c Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 
# Conflicts:
#	lib/common/util.h
#	programs/paramgrill.c
#	visual/2013/fullbench/fullbench.vcxproj.filters
#	visual/2013/fuzzer/fuzzer.vcxproj.filters
 2016-05-10 13:20:01 +02:00
Yann Collet
918b934a26 minor comment update 2016-05-10 07:00:07 +02:00
inikep
349fcf7ea2 fixed Linux compilation 2016-05-09 18:04:07 +02:00
inikep
aaaf923a17 common/util.h moved to programs/util.h 
decompress/fse_decompress.c moved to common/
 2016-05-09 16:19:25 +02:00
inikep
d44ec48b1c detection of snprintf for Visual <= 2012 2016-05-09 14:37:41 +02:00
Yann Collet
59b6ba7677 Merge pull request #180 from thatsafunnyname/patch-1 
No stdint.h on OpenVMS.
 2016-05-09 11:56:42 +02:00
Yann Collet
4ff73a2eff added : midipix support (#181) 2016-05-09 11:44:00 +02:00
Yann Collet
d275236e6a Fixed Windows API, as detected by @KrzysFR (#171) 2016-05-09 08:19:27 +02:00
Yann Collet
f6ca09b5ff Reduced console display on loading lots of files with zstd --train. Reported by @KrzysFR, see #177 2016-05-09 04:44:45 +02:00
Yann Collet
e1a69a88c0 Expose Advanced Streaming API for v0.5.x legacy 2016-05-08 09:13:40 +02:00
Yann Collet
a5dacdf551 updated fse (minor refactor) 2016-05-06 16:55:27 +02:00
Yann Collet
18dedece91 Support legacy ZSTD_decompress_usingDict() (starting v0.5+) 2016-05-06 16:43:23 +02:00
Yann Collet
8283a2f0aa Fixed decoding corruption error with quad-symbols huffman on legacy decoders 2016-05-06 01:51:31 +02:00
inikep
13c8424ea0 code cleaning 2016-05-05 13:58:56 +02:00
inikep
0bd0faec32 fixed compatibility issues 2016-05-05 13:10:57 +02:00
Yann Collet
6d1d25299a fixed Huff0 quad-symbols decoder (#173) 2016-05-05 12:41:36 +02:00
inikep
9c22e57bfb Compiler Options moved to util.h 2016-05-05 11:53:42 +02:00
Yann Collet
ddb8ebd5b3 Stricter gnu90 tests 2016-05-05 04:59:53 +02:00
Yann Collet
3f8ed50bf1 Added gnu90 test 
work-around : disabled automatic selection of huff0 quad-decoder (see #173)
 2016-05-05 03:01:13 +02:00
inikep
3163403855 UTIL_prepareFileList 2016-05-05 00:25:38 +02:00
thatsafunnyname
b0e5427d1a No stdint.h on OpenVMS. 
No stdint.h on OpenVMS, it is inttypes.h instead, some projects use HAVE_STDINT_H.
 2016-05-04 15:43:42 +01:00
Yann Collet
6f9c056662 Fixed BMI behavior on x86 and x32 targets, reported by @tycho (#178) 2016-05-01 10:26:30 +02:00
inikep
55d047aa92 getTotalFileSize moved to common/util.h 2016-04-28 16:50:13 +02:00
inikep
d5ff2c3d9a ordering of #include 2016-04-28 14:40:45 +02:00
inikep
83c76b4594 bench.c: time functions moved to common/util.h 2016-04-28 13:16:01 +02:00
inikep
69fcd7c0ae getFileSize moved to common/util.h 2016-04-28 12:23:33 +02:00
inikep
1007a1fe3d get rid of some Visual C++ warnings 2016-04-25 15:23:09 +02:00
inikep
6cb083fe3d fixed 64-bit compilation with Visual C++ 2016-04-25 14:42:15 +02:00
inikep
ea4ee3eee0 added compatibility with Visual C++ 2012 2016-04-25 13:09:06 +02:00
inikep
a8138fd767 updated lib/README.md part 2 2016-04-25 11:36:44 +02:00
inikep
49794316c2 updated lib/README.md 2016-04-25 11:31:28 +02:00
inikep
f066816998 fix for g++ compilation 2016-04-22 18:54:05 +02:00
inikep
db3964382c introduced ZSTD_NODECOMPRESS to link only compressor 2016-04-22 18:22:30 +02:00
inikep
a1febea01d Error functions moved to common/zstd_common.c 2016-04-22 17:14:25 +02:00
inikep
3c7c3527d0 introduced ZSTD_NOCOMPRESS to generate decompressor only 2016-04-22 13:59:05 +02:00
inikep
23a0889301 separation of lib/ into common/, compress/, decompress/, dictBuilder/, legacy/ 2016-04-22 12:43:18 +02:00
inikep
4699b3be03 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-21 12:30:58 +02:00
inikep
38654988f3 minor speed improvements 2 
bench.c: block size has to be bigger than 32 bytes
zstdcli.c: support for e.g. -B16k -B16m
 2016-04-21 12:18:47 +02:00
inikep
ef51941822 zst_opt.h: minor compression speed improvement 2016-04-21 11:08:43 +02:00
Yann Collet
e02808ffc5 bumped lib version to 0.6.1 
added ZSTD_VERSION_STRING
 2016-04-20 22:46:16 +02:00
inikep
c5e1d295ee bench.c: force at least one compression and decompression loop 
fix for -i0 with small files
 2016-04-19 09:37:59 +02:00
inikep
2c5eeea635 introduced ZSTD_DEAFULT_CLEVEL for (compressionLevel<=0) 2016-04-15 13:44:46 +02:00
inikep
4b3c5eed99 removed ZSTD_compressBegin_targetSrcSize 2016-04-14 13:43:51 +02:00
inikep
c70c90e8a3 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-13 10:48:26 +02:00
Yann Collet
16748729c9 Updated DICT_MAGIC, since internal dictionary format has changed (length encoding) 2016-04-12 23:26:23 +02:00
Yann Collet
c5b6d9741a Added : zbuff.c to library; reported by Yipei Zhang 2016-04-12 21:46:18 +02:00
Yann Collet
389648cc6d Fixed : contentSize + copyCCtx() 2016-04-12 19:13:08 +02:00
Yann Collet
541dc7c7d4 Added : fuzzer test, checking contentLength value after copyCCtx() usage 2016-04-12 18:00:20 +02:00
Yann Collet
6236eba991 removed headerBuffer 2016-04-12 15:52:33 +02:00
Yann Collet
0085cd3138 created ZSTD_writeFrameHeader() 2016-04-12 14:14:10 +02:00
Yann Collet
abb5c653f1 minor speed improvement for ZSTD_copyCCtx() 2016-04-11 20:42:31 +02:00
Yann Collet
887e7da785 Fixed copyCCtx() on uninitialized cctx (reported by Oliver Lange) 2016-04-11 20:12:27 +02:00
inikep
8844e21dff Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into dev 2016-04-11 16:49:27 +02:00
inikep
f7d210b2e9 cache literal prices for ZSTD_btopt 2016-04-11 16:35:13 +02:00
Yann Collet
582933fde5 fixed round-trip corruption, reported by Oliver Lange 2016-04-11 16:25:56 +02:00
Yann Collet
125d81774f fixed uninitialization error, reported by Maciej Adamczyk 2016-04-11 14:22:56 +02:00
Yann Collet
d673d4cf66 fixed uninitialization error, reported by Maciej Adamczyk 2016-04-11 14:10:23 +02:00
Yann Collet
ccbba296e1 Fix infinite loop with invalid data (fuzzer test) 2016-04-10 14:42:26 +02:00
Yann Collet
793c649ead updated cLevels 2016-04-09 20:32:00 +02:00
Yann Collet
b144bc3e1c Fixed uninitialized warning, reported by Maciej Adamczyk 2016-04-09 17:26:22 +02:00
Yann Collet
9990986bdc fixed minor warnings when compiling with gcc -g (aka -std=gnu90) 2016-04-09 16:17:18 +02:00
Yann Collet
029267ab3f Integrated Legacy v0.5.x decoder (provided by @inikep) 2016-04-09 09:42:27 +02:00
Yann Collet
7eff39f7ea fixed decoding error 2016-04-09 01:51:36 +02:00
Yann Collet
ea63bb7b5e converted fuzzer timer to clock_t 2016-04-08 15:25:32 +02:00
Yann Collet
78267d1abe updated cLevel for block <= 256K 2016-04-08 12:36:19 +02:00
Yann Collet
72cc704514 Merge branch 'dev060' of github.com:Cyan4973/zstd into dev060 2016-04-08 02:02:31 +02:00
Yann Collet
0dbf2874ee faster level 1 at 256 KB 2016-04-08 02:02:12 +02:00
Yann Collet
04a4fd638b Merge pull request #164 from inikep/dev 
only repcode 0 for lazy, lazy2 and btlazy2 strategies
 2016-04-07 20:46:13 +02:00
inikep
afe1f7994b Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 
# Conflicts:
#	lib/zstd_compress.c
 2016-04-07 19:50:03 +02:00
Yann Collet
9e8b09a7bd Fixed memory initialization issue, reported by Maciej Adamczyk 2016-04-07 19:35:23 +02:00
inikep
2bf79f9bff use extDict only when needed 2016-04-07 19:24:33 +02:00
inikep
64d7bcb0dc only repcode 0 for lazy, lazy2 and btlazy2 strategies 2016-04-07 19:14:09 +02:00
inikep
a69640cca1 Revert "removed void ZSTD_compressBlock_greedy" 
This reverts commit 9296400a69.
 2016-04-07 18:22:29 +02:00
Yann Collet
d7bee33caf Fixed make install (reported by Oliver Lange) 2016-04-07 18:16:42 +02:00
Yann Collet
5d39357bc4 minor compression speed optimization 2016-04-07 17:19:00 +02:00
inikep
9296400a69 removed void ZSTD_compressBlock_greedy 2016-04-07 16:58:04 +02:00
Yann Collet
485371600a Implemented default coding table for lengths and offset 2016-04-07 15:24:29 +02:00
Yann Collet
79459da14b Merge pull request #163 from inikep/dev 
3 repcodes and MINMATCH=3
 2016-04-07 14:37:30 +02:00
inikep
81a345b0ce minor speed improvements 2016-04-07 11:35:17 +02:00
inikep
e47f33f980 reordered code 2016-04-06 20:58:00 +02:00
inikep
2db1eb78e9 reordered functions 2016-04-06 17:14:19 +02:00
inikep
e5b6e9e551 fixed support for searchLength=4 2016-04-06 13:15:38 +02:00
inikep
75716851d4 MINMATCH=3 for lazy strategy 2016-04-06 12:34:42 +02:00
inikep
7bc19b6b3e MINMATCH=3 2016-04-06 09:46:01 +02:00
inikep
f8a339d274 working ZSTD_compressBlock_opt_extDict_generic 2016-04-05 23:58:51 +02:00
inikep
83417bbbd2 code formatting 2016-04-05 21:30:03 +02:00
inikep
5ce00aee37 ZSTD_compressBlock_opt_extDict_generic in progress 2016-04-05 21:03:43 +02:00
inikep
faa8d8aae1 introduced ZSTD_compressBlock_greedy 2016-04-05 19:01:10 +02:00
inikep
908fcb384a 3 repcodes with ZSTD_compressBlock_lazy_extDict_generic 2016-04-05 18:16:38 +02:00
inikep
a4385aa23b ZSTD_REP_NUM=3 only 2016-04-05 17:11:56 +02:00
inikep
d5359b2e55 Merge remote-tracking branch 'refs/remotes/origin/repcodes' into dev 2016-04-05 17:07:36 +02:00
inikep
ba73d1bb6b support for ZSTD_REP_NUM=3 2016-04-05 10:30:05 +02:00
inikep
74e3a7ba4c removed kSlotNew 2016-04-05 10:18:37 +02:00
inikep
b3532793a9 experiments with ZSTD_decodeSequence 2016-04-05 10:08:25 +02:00
Yann Collet
46bdb6267e Merge pull request #161 from inikep/dev 
Dev
 2016-04-05 09:59:59 +02:00
inikep
2ec2698ef9 working 4 reps 2016-04-05 09:34:13 +02:00
inikep
19140cd80e zstd_stats.h moved to .debug/ 2016-04-05 08:52:57 +02:00
inikep
d6f208badf improved logging 2016-04-04 21:15:23 +02:00
inikep
78e52044de Merge remote-tracking branch 'refs/remotes/origin/dev' into repcodes 
# Conflicts:
#	lib/zstd_compress.c
#	lib/zstd_decompress.c
#	lib/zstd_internal.h
#	lib/zstd_opt.h
#	programs/bench.c
 2016-04-04 16:28:40 +02:00
inikep
cb70c8ab90 fixed -Werror,-Wconversion warning 2016-04-04 15:43:45 +02:00
inikep
0c7456c5b7 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 
# Conflicts:
#	lib/zstd_compress.c
#	lib/zstd_static.h
 2016-04-04 14:54:53 +02:00
Yann Collet
b59bf963a8 fixed Visual conversion warnings 2016-04-04 14:53:16 +02:00
inikep
227dd4bc74 changed price estimation for optimal parser 2016-04-04 14:34:02 +02:00
Yann Collet
8a57b928ba changed contentLog -> chainLog 2016-04-04 13:49:18 +02:00
Yann Collet
1005fc1276 reduced memory usage of very hich compression levels in 32-bits mode 2016-04-04 13:28:28 +02:00
inikep
19bd48f516 Merge from Cyan4973/dev060 2016-04-04 12:10:00 +02:00
Yann Collet
1535414927 Fixed 32-bits windowLog limitation 
Fixed some macro errors in zbufftest
 2016-04-04 04:22:53 +02:00
Yann Collet
5a88a70f97 fixed buffer sync error in zbuff 2016-04-03 20:24:25 +02:00
Yann Collet
ef36390e4a refined exception for hashlog < min 2016-04-02 00:46:40 +02:00
Yann Collet
40358d01aa Fix : rare bugs when hashLog < min in bt mode 2016-04-02 00:40:09 +02:00
Yann Collet
1d803d8deb fixed zbuff error 775-3 (internal buffer management) 2016-04-01 18:57:41 +02:00
Yann Collet
efb18309ef fixed minor visual 32-64bits conversion warning 2016-04-01 18:54:13 +02:00
Yann Collet
afa60e0f14 fixed decoding error, reported by Maciej Adamczyk 2016-04-01 15:49:41 +02:00
Yann Collet
27caf2afca minor variable isolation 2016-04-01 15:48:48 +02:00
Yann Collet
9bb87e5995 fixed harmless comparison warning 2016-03-30 21:28:15 +02:00
Yann Collet
51d500425f fixed paramgrill 2016-03-30 20:42:19 +02:00
Yann Collet
dd6466a211 fixed clang conversion warning 2016-03-30 20:06:26 +02:00
Yann Collet
3b71925c2d separate params into compressionParams and frameParams 2016-03-30 19:48:05 +02:00
Yann Collet
21588e370b changed validateParams() into checkParams() + adjustParams() 2016-03-30 16:50:44 +02:00
Yann Collet
b44be74244 Fixed zdict 
more dictionary compression tests
 2016-03-26 20:52:14 +01:00
Yann Collet
74bd11954b fixed ARM compatibility 2016-03-26 17:50:26 +01:00
Yann Collet
b9151406de fixed 32-bits compatibility 2016-03-26 17:18:11 +01:00
inikep
97c88e716c fix for -Wshorten-64-to-32 warning in ZSTD_statsUpdatePrices 2016-03-25 13:39:14 +01:00
Yann Collet
433a5cce7e improved decompression speed (680) 2016-03-25 11:43:48 +01:00
inikep
5cc4efdaf8 created zstd_stats.h 2016-03-25 10:52:25 +01:00
Yann Collet
646693e3be support for alternate offset (fusion) 2016-03-24 02:42:15 +01:00
Yann Collet
b21ce15efe minor variables isolation 2016-03-24 02:02:32 +01:00
Yann Collet
7cbe79ab25 fixed write overflow found by fuzzer 2016-03-23 22:31:57 +01:00
Yann Collet
72d706a020 fixed crash at -O3 with customized block size (wrong alignment) 2016-03-23 20:44:12 +01:00
inikep
eaba91a161 added ZSTD_compressBegin_targetSrcSize 2016-03-23 20:30:26 +01:00
Yann Collet
862a85976f Added BMI instructions 2016-03-23 18:45:23 +01:00
inikep
7adceef974 hashLog3 added to ZSTD_CCtx 2016-03-23 15:53:38 +01:00
Yann Collet
6cf45dac81 shared const BIT_mask 2016-03-23 14:41:54 +01:00
Yann Collet
3c017867de minor optimization (DSpeed 665) 2016-03-23 14:09:51 +01:00
Yann Collet
2512597576 minor reordering (DSpeed 665) 2016-03-23 14:00:09 +01:00
Yann Collet
afab02098a improved decoding speed (660) 2016-03-23 13:57:49 +01:00
inikep
472638c861 added mili_sleep and setHighPriority 2016-03-23 12:28:28 +01:00
Yann Collet
9c34df93b6 new decodeSequence, merging parts 2016-03-23 01:54:25 +01:00
Yann Collet
add08d6f61 minor variation - DSpeed at 640 2016-03-23 01:32:41 +01:00
Yann Collet
be391438ff first working version with both encoder and decode alternate LL + ML coding scheme. 
decompression speed highly impacted
 2016-03-22 23:19:28 +01:00
inikep
2872b6f01f Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 
# Conflicts:
#	lib/zstd_compress.c
#	programs/bench.c
 2016-03-22 14:38:34 +01:00
Yann Collet
fadda6c875 first prototype with ML encoding scheme (but not yet decoding scheme) 2016-03-22 12:14:26 +01:00
inikep
2b942881ac improved 4reps in ZSTD_compressBlock_lazy_generic 
kSlotNew = 0
 2016-03-22 11:56:22 +01:00
Yann Collet
1dcf930f72 Merge branch 'dev060' into alternateLLcode 2016-03-22 00:29:57 +01:00
inikep
f2fa0e1198 final repcodes for ZSTD_btopt 2016-03-21 20:14:03 +01:00
inikep
378be62a9e better repcodes 2016-03-21 19:48:54 +01:00
inikep
fc1d7bd93b changed update logic of repcodes 2016-03-21 19:02:16 +01:00
inikep
e7ec8f2b9b improved ratio 2016-03-21 17:23:02 +01:00
Yann Collet
be4605d851 improved decompression speed 2016-03-21 14:29:26 +01:00
Yann Collet
e91477c171 minor variable isolation 2016-03-21 14:22:38 +01:00
Yann Collet
b0aec17a90 code simplification (but reduce decompression speed ...) 2016-03-21 13:24:16 +01:00
Yann Collet
6c62b7dfc8 support default LL distribution 2016-03-21 00:39:19 +01:00
Yann Collet
d64f435f63 handles litLength >= 65535 2016-03-21 00:07:42 +01:00
Yann Collet
597847a2ae first emulation 2016-03-20 19:14:22 +01:00
Yann Collet
ecabfe3777 Improved index overflow protection 2016-03-20 16:20:06 +01:00
Yann Collet
5244738046 minor code refactoring 2016-03-20 16:00:00 +01:00
Yann Collet
de406eebcd minor code refactor 2016-03-20 15:46:10 +01:00
Yann Collet
516ba88022 updated huff0 2016-03-20 05:40:39 +01:00
Yann Collet
a5c2c08c68 code refactoring 2016-03-20 01:09:18 +01:00
Yann Collet
70e4577179 minor code refactoring 2016-03-19 18:08:32 +01:00
Yann Collet
c6eea2b2f2 Improved speed for small files 2016-03-19 17:18:00 +01:00
Yann Collet
e74215e3c7 performance fix for small files 2016-03-19 16:09:09 +01:00
inikep
1269e2f7db working kSlotNew for ZSTD_btopt 2016-03-19 15:32:21 +01:00
Yann Collet
bb604486f1 fixed clang warning 2016-03-19 15:18:42 +01:00
Yann Collet
37f3d1b2ab minor refactor 2016-03-19 15:11:42 +01:00
Yann Collet
90a03a95a1 minor performance fix (decompression) 2016-03-19 14:47:31 +01:00
Yann Collet
01e5b95199 code refactor 2016-03-19 14:14:31 +01:00
Yann Collet
d1d210f3fb minor code refactor 2016-03-19 12:12:07 +01:00
Yann Collet
4db09efde4 simplifications 2016-03-18 22:23:49 +01:00
Yann Collet
39c596ca58 minor simplifications 2016-03-18 21:40:56 +01:00
inikep
fade9b7ae3 ZSTD_btopt with 4 repcodes 2016-03-18 16:10:35 +01:00
inikep
1c1e6916e4 improved timer accuracy 2016-03-18 12:49:44 +01:00
Yann Collet
a910dc8238 minor code refactoring 2016-03-18 12:37:45 +01:00
inikep
5b15865413 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into repcodes 
# Conflicts:
#	lib/zstd_decompress.c
#	programs/bench.c
 2016-03-18 11:14:58 +01:00
inikep
eca2ec0b3b introduced ZSTD_REP_MOVE and ZSTD_REP_INIT 2016-03-18 11:03:43 +01:00
inikep
c56e32e782 4 repcodes when litLen==0 2016-03-17 19:53:38 +01:00
inikep
3e743052cd kSlotNew 2016-03-17 17:24:17 +01:00
Yann Collet
3fd164e656 minor refactoring 2016-03-17 13:43:32 +01:00
Yann Collet
b40287aaef Improved decompression speed (by @inikep) 2016-03-17 12:11:16 +01:00
inikep
7d1742825b support for ZSTD_REP_NUM == 1 and ZSTD_REP_NUM == 4 2016-03-17 11:21:00 +01:00
inikep
5945308983 working 4 reps 2016-03-16 15:35:14 +01:00
Yann Collet
569b81adb1 changed BLOCKSIZE into ZSTD_BLOCKSIZE_MAX 2016-03-16 15:26:51 +01:00
inikep
62d694795d ZSTD_compressBlock_lazy supports more reps 2016-03-16 12:57:07 +01:00
inikep
0ac4a551b9 ZSTD_REP_NUM removed from ZSTD_storeSeq 2016-03-16 12:03:07 +01:00
inikep
df164e6d35 introduced U32 rep[ZSTD_REP_NUM] 2016-03-16 11:56:59 +01:00
inikep
4569667c85 ZSTD_REP_NUM 2016-03-16 11:40:09 +01:00
Yann Collet
6cb41356e8 update doc / comments 2016-03-15 15:47:38 +01:00
Yann Collet
346bffbc68 Added frame content size test within fuzzer 2016-03-15 15:24:52 +01:00
Yann Collet
09b21ee2f6 fix fileio bug with new zbuff 
simplified zbuff decoding
 2016-03-15 12:56:03 +01:00
inikep
d700a1a8b9 ZSTD_setAdditionalParam 2016-03-15 12:18:44 +01:00
Yann Collet
1c2c2bcaff minor refactoring 2016-03-15 01:33:36 +01:00
Yann Collet
d1b26849e5 Frame content size supported 2016-03-15 01:24:33 +01:00
inikep
5fdd0b46d3 added additionalParam 2016-03-14 19:51:11 +01:00
Yann Collet
fb7973580f minor comments update 2016-03-13 11:08:40 +01:00
Yann Collet
03ea59b17b Decompression can decode frame content size 2016-03-12 01:25:40 +01:00
Yann Collet
0e491c01fe created ZSTD_frameParams 2016-03-11 21:58:04 +01:00
inikep
cc6c283633 removed hashLog3 from ZSTD_parameters 2016-03-10 15:17:24 +01:00
inikep
260a9e3886 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 2016-03-10 15:04:12 +01:00
inikep
b0c640d934 small ratio improvement with MINMATCH=3 2016-03-10 13:18:36 +01:00
Yann Collet
da5be72fda update compression level parameters 2016-03-10 11:01:44 +01:00
Yann Collet
dcc000f0eb minor doc clarifications 2016-03-10 01:09:41 +01:00
Yann Collet
e3193c4081 update compresson levels 2016-03-09 16:57:09 +01:00
Yann Collet
4f013a2281 Merge pull request #142 from inikep/dev 
Dev
 2016-03-09 16:29:42 +01:00
inikep
b5a519f43c improved compression speed 
introduced cache in ZSTD_setLog2Prices
 2016-03-09 15:45:01 +01:00
Yann Collet
95cd0c2e26 frame format can support window sizes up to 128 MB 2016-03-08 18:24:21 +01:00
Yann Collet
ff14b351f5 Merge pull request #141 from inikep/dev 
fix update for zstd_opt
 2016-03-08 14:41:04 +01:00
Yann Collet
d3a6660ad1 Update clevel table for <= 128K 2016-03-08 14:12:46 +01:00
Yann Collet
2e91dde43e improved memory size evaluation by paramgrill 2016-03-08 12:22:11 +01:00
inikep
48413765b4 fixed update of hashTable for matches with length of 4+ 2016-03-08 11:03:55 +01:00
Yann Collet
dd54bbc184 Fixed large NbSeq > 32 K 
Added a test in Fuzzer to check NbSeq > 32 K
 2016-03-08 02:35:34 +01:00
Yann Collet
370b08e840 fix compression ratio for blocks <= 16 KB 2016-03-08 00:03:59 +01:00
Yann Collet
fb7ab8b2b8 Merge pull request #140 from inikep/dev 
mml3
 2016-03-07 18:06:48 +01:00
inikep
4f7f79ef9d fixed update of nextToUpdate 2016-03-07 16:14:58 +01:00
inikep
e2446b0e36 added support for MINMATCH=3 on big endian architecture 
changed ZSTD_HASHLOG3_MIN to 2 (4 bytes for MINMACH > 3)
 2016-03-07 10:07:08 +01:00
inikep
89c9e1a847 added missing "case 3:" in ZSTD_BtGetAllMatches 2016-03-06 23:21:52 +01:00
Yann Collet
1df2594bfe new compression level scale for <= 16KB 2016-03-05 18:43:21 +01:00
inikep
01060bc477 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev060' into dev 
# Conflicts:
#	lib/zstd_compress.c
 2016-03-04 20:10:09 +01:00
inikep
f3c6503e55 resolve conflict in zstd_internal.h 2016-03-04 20:04:25 +01:00
inikep
e29caf7d1c code cleanup 2016-03-04 19:52:23 +01:00
inikep
805d2a7465 removed zstd_opt_internal.h 2016-03-04 19:31:57 +01:00
Yann Collet
944d0d2285 updated fse version 2016-03-04 19:26:59 +01:00
inikep
4ab9c913a5 MEM_readMINMATCH replaced with inline function 2016-03-04 19:17:31 +01:00
Yann Collet
4c01580362 finer searchLength validation 2016-03-04 17:07:54 +01:00
Yann Collet
d409db68d3 Support for nbSeq > 32767 
tamed -Wstrict-aliasing warning
 2016-03-04 14:45:31 +01:00
Yann Collet
1435335342 Merge pull request #138 from inikep/dev 
reduce stack usage with optimal parsing mode
 2016-03-03 02:08:34 +01:00
inikep
5cccd77c55 fix clang -Wcast-align warning (part 2) 2016-03-02 20:37:49 +01:00
inikep
51bb9a0064 fix clang -Wcast-align warning 2016-03-02 19:17:13 +01:00
inikep
87d4f3daa4 priceTable moved to heap 2016-03-02 15:56:24 +01:00
inikep
338533f741 improved price function initialization 2016-03-02 14:36:41 +01:00
Yann Collet
a997654670 Merge pull request #136 from inikep/dev 
merging mml3 mode
 2016-03-01 17:31:03 +01:00
inikep
a4dde25498 fixed repcode before lowLimit 2016-03-01 14:14:35 +01:00
inikep
2d55563b92 better logs 2016-02-29 22:07:40 +01:00
inikep
f647d99d61 fixed bug with 3 or more succesive rep codes 2016-02-29 12:33:08 +01:00
inikep
f414647c6b fixed ZSTD_copyCCtx 2016-02-25 22:31:07 +01:00
inikep
59493e8669 fixed paramgrill 2016-02-25 19:15:08 +01:00
inikep
afa9c9f5ab turn off RTF optimization 2016-02-25 16:47:57 +01:00
Yann Collet
ee0229287c changed version to v0.6.0 2016-02-25 14:53:35 +01:00
inikep
02137f8c42 more accurate gain function 2016-02-24 18:09:36 +01:00
inikep
ee55628c9d combined price function 2016-02-24 14:40:30 +01:00
Yann Collet
3152a8c87a finer dictionary completion 2016-02-23 21:28:59 +01:00
inikep
4a981f7937 improved ZSTD_GETPRICE 2016-02-23 19:08:20 +01:00
inikep
e0010e9baf priceFunc 2016-02-23 16:25:04 +01:00
inikep
15174b0cfb statistics of encoded sequences 2016-02-23 12:41:56 +01:00
inikep
9f754d23dc dummy levels 22-25 2016-02-22 17:00:04 +01:00
inikep
6b3739c8e5 MINMATCH=3 or 4 selected in ZSTD frame 2016-02-22 15:53:42 +01:00
inikep
84f43e2359 support for searchLength from 3 to 7 for optimal parser 2016-02-22 11:34:07 +01:00
inikep
c950b78ce2 removed backward match extension 2016-02-22 10:11:39 +01:00
inikep
d3b8d7a4e5 removed ZSTD_opt parser 2016-02-22 10:06:17 +01:00
inikep
6291c54006 litlen bounded to 128 bytes 2016-02-19 18:24:14 +01:00
Yann Collet
f8482131d7 better compatibility with Visual <= 2012 2016-02-19 17:33:43 +01:00
Yann Collet
191b52a8dc silence array subscript warning 2016-02-19 15:16:15 +01:00
inikep
fcafb26b2e speed optimiaztions 2016-02-19 11:59:44 +01:00
inikep
c3a9a9ca56 ZSTD_rescaleFreqs and removed minml 2016-02-19 11:05:25 +01:00
inikep
cc52a97214 added hashLog3 2016-02-19 10:09:35 +01:00
Yann Collet
4c64d51425 modified level 18 2016-02-18 01:01:42 +01:00
Yann Collet
d56a4192f6 Added : README within /lib directory, to help file selection 2016-02-17 17:47:29 +01:00
Yann Collet
6062b15a07 performance fix for hash chain mode 2016-02-16 17:41:03 +01:00
Yann Collet
accfd80c37 Added : option -t to test compressed files integrity 2016-02-15 19:33:16 +01:00
Yann Collet
0068be94d8 Faster literals cost evaluation (suggested by @inikep) 2016-02-15 18:42:13 +01:00
Yann Collet
e93add0439 Removed if (matchIndex >= current) return 0; as recommended by @inikep 2016-02-15 17:44:14 +01:00
Yann Collet
b8a6f6824b refactored fix to be more generic and applicable to both btlazy2 and btopt 2016-02-15 17:06:29 +01:00
Yann Collet
b37e0a9241 performance tweak : bt search w/ long repeated segments 2016-02-15 14:07:07 +01:00
Yann Collet
cefef8cde7 Change : zstd_opt_bt => zstd_btopt 2016-02-15 07:21:54 +01:00
Yann Collet
0d78a33374 updated CLevel confs for 256K 2016-02-15 06:49:09 +01:00
Yann Collet
422fd563f3 modified : compression Levels for blocks <= 16KB 2016-02-13 15:07:24 +01:00
Yann Collet
38fba56483 Changed compression level settings 2016-02-13 11:20:23 +01:00
Yann Collet
e8bd73df80 Fixed : error when compressing one file 2016-02-13 03:09:48 +01:00
Yann Collet
23e8aee224 Introduced new compression level settings 
Changed position of parameter TargetLength
 2016-02-13 02:08:46 +01:00
Yann Collet
82f4d7da03 Fixed Visual solution 2016-02-12 22:04:49 +01:00
Yann Collet
6f3acbac0d notificationLevel into ZDICT_param_t 2016-02-12 20:19:48 +01:00
Yann Collet
09ab681328 fixed libzstd install 2016-02-12 19:00:30 +01:00
Yann Collet
62ae5fb0d2 zstd_buffered => zbuff 2016-02-12 18:59:11 +01:00
Yann Collet
f4c9d75370 dictBuilder => zdict 2016-02-12 18:45:02 +01:00
Yann Collet
f062436fa6 cli : added ability to compress multiple files into stdout (-c) 2016-02-12 15:56:46 +01:00
Yann Collet
71eafdd23f Integrated dictionary builder into zstd cli 2016-02-12 02:31:57 +01:00
Yann Collet
fb92a78ac1 Fixed clang warnings 2016-02-12 00:32:38 +01:00
Yann Collet
7d36028078 Integrate dictBuilder into libzstd 2016-02-12 00:10:25 +01:00
Yann Collet
82260ddd8a fixed asan read error in ZSTD_HcGetAllMatches 2016-02-11 07:14:25 +01:00
Yann Collet
04b12d8bcb paramgrill can select manually targetLength 
new compression levels
 2016-02-11 06:23:24 +01:00
Yann Collet
bd828d9cd1 changed : sufficientLength => targetLength 2016-02-11 04:38:55 +01:00
Yann Collet
4488661678 minor cosmetics 2016-02-11 04:17:50 +01:00
Yann Collet
490aa68b18 restored fast zstd_opt variant 2016-02-10 17:18:24 +01:00
Yann Collet
b79a0b34f3 paramgrill supports sufficientLength 
stronger variation for ZSTD_opt
 2016-02-10 17:07:37 +01:00
Yann Collet
62ae262771 restored currentMl>0 condition, as suggested by @inikep 2016-02-10 15:44:07 +01:00
Yann Collet
3b63f7fbb0 change zstd_opt.c into zstd_opt.h, since it's included 2016-02-10 15:05:12 +01:00
Yann Collet
0db4115f35 fixed debug print macros on Windows 2016-02-10 14:50:22 +01:00
Yann Collet
2f94d2f166 minor refactor 2016-02-10 14:26:30 +01:00
Yann Collet
768c6bc705 fix : inlining difficulties with gcc 4.6 (travis) 2016-02-10 14:01:49 +01:00
Yann Collet
70e8c389f1 Refactoring for portability 2016-02-10 13:37:52 +01:00
Yann Collet
302177a0c8 Merge pull request #125 from inikep/dev 
Dev
 2016-02-10 00:57:00 +01:00
inikep
ce6730a740 fix for ZSTD_HcGetAllMatches_generic 2016-02-09 23:26:30 +01:00
inikep
be77f3342d Revert "Revert "ZSTD_compressBlock_opt_extDict_generic"" 
This reverts commit 2b714dfd26.
 2016-02-09 23:00:41 +01:00
inikep
2b714dfd26 Revert "ZSTD_compressBlock_opt_extDict_generic" 
This reverts commit b498fc1c99.
 2016-02-09 22:47:51 +01:00
inikep
b498fc1c99 ZSTD_compressBlock_opt_extDict_generic 2016-02-09 22:23:48 +01:00
inikep
e75621f98d dumpsEnd 2016-02-09 21:12:23 +01:00
inikep
31314ae0d1 int to U32 2016-02-09 20:47:39 +01:00
Yann Collet
6bff748e43 Fixed minor warnings in legacy modes 2016-02-09 17:55:01 +01:00
Yann Collet
4f0a3931c8 added missing patches 2016-02-07 04:00:27 +01:00
inikep
da730ff092 a bugfix for backward matches 2016-02-06 15:18:26 +01:00
inikep
dbade7c263 remove ZSTD_compressBlock_opt2_generic 2016-02-05 20:29:31 +01:00
inikep
f2fee4c316 added params.sufficientLength 2016-02-05 19:45:25 +01:00
inikep
5b0749715d backward match extension 2016-02-05 19:25:40 +01:00
inikep
2712b69582 Merge remote-tracking branch 'refs/remotes/origin/v05x' into dev 2016-02-05 09:22:10 +01:00
inikep
3379c5df36 ZSTD_OPT_DEBUG 2016-02-05 09:21:20 +01:00
inikep
f967753bc2 ssize_t -> int 2016-02-05 09:07:22 +01:00
Yann Collet
982ffc773d changed enum type name to ZSTD_ErrorCode 2016-02-05 02:33:10 +01:00
inikep
55a444764b sufficient_len = 32 2016-02-04 22:22:47 +01:00
inikep
34608aaa29 ssize_t 2016-02-04 22:18:02 +01:00
inikep
17712afdbc Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into v05x 2016-02-04 21:49:45 +01:00
inikep
b3df71e538 code cleaning 2016-02-04 21:45:26 +01:00
inikep
4fa4098628 last match will start at a minimum distance of 8 from oend 2016-02-04 21:16:07 +01:00
inikep
08f500ca10 cur_min 2016-02-04 20:53:54 +01:00
inikep
6f07186745 clean code 2016-02-04 17:57:50 +01:00
inikep
59c9146258 40099096 2016-02-04 17:48:33 +01:00
inikep
baff8d6c1f gitignore 2016-02-04 16:02:05 +01:00
inikep
40dd4739e3 removed llen 2016-02-04 15:52:35 +01:00
inikep
c9483f0982 clean code 2016-02-04 15:28:55 +01:00
Yann Collet
953ce72f4a minor comments refactoring 2016-02-04 15:28:14 +01:00
inikep
158677c16a new copy rep 2016-02-04 15:25:37 +01:00
inikep
d1ff7bf89a cur_rep 2016-02-04 14:57:11 +01:00
inikep
eb72f5992a i != litlen 2016-02-04 14:55:23 +01:00
inikep
997fb064a6 anchor 2016-02-04 13:01:56 +01:00
inikep
75be1a412c fixes 2016-02-04 11:48:06 +01:00
inikep
cb5cd16d6b 40392170 integer 2016-02-04 11:19:06 +01:00
inikep
27e1c6a51f ZSTD_resetFreqs 2016-02-04 11:11:08 +01:00
inikep
4196923dc9 40370570 ZSTD_getPriceReal 2016-02-04 10:04:33 +01:00
inikep
70b054559b MaxLit 2016-02-03 22:56:55 +01:00
inikep
a98149df59 ZSTD_getLiteralPrice 2 2016-02-03 21:26:18 +01:00
inikep
dbbedfbd95 ZSTD_getLiteralPrice 2016-02-03 19:53:29 +01:00
inikep
892776efd6 ZSTD_get_price2 2016-02-03 19:27:44 +01:00
inikep
1292509987 ZSTD_get_price 2016-02-03 19:13:28 +01:00
inikep
749460d9ab ZSTD_updatePrice 2016-02-03 19:08:48 +01:00
inikep
3bfcfc7bb0 ZSTD_updatePrice 2016-02-03 18:47:30 +01:00
inikep
53721f134c Merge remote-tracking branch 'refs/remotes/Cyan4973/v05x' into v05x 
# Conflicts:
#	lib/zstd_compress.c
#	lib/zstd_static.h
 2016-02-03 17:29:40 +01:00
inikep
78e5ea3a32 ZSTD_getPrice 2016-02-03 17:25:42 +01:00
inikep
d02506c186 40699855 2016-02-03 16:54:23 +01:00
inikep
85cc6b509b minml 2016-02-03 15:58:08 +01:00
inikep
7f6680e658 clean some code 2016-02-03 13:30:34 +01:00
inikep
4cb2009821 40901629 2016-02-03 13:25:34 +01:00
inikep
1b8b1f9d53 auto check of matches 2016-02-03 13:17:28 +01:00
inikep
e9f30eae4b 40970636 2016-02-03 12:53:07 +01:00
Yann Collet
b010b3b627 Fixed decoding error (afl) 2016-02-03 12:39:34 +01:00
Yann Collet
72bff50ecf Updated error API, following suggestions by @nemequ (#116) 2016-02-03 12:06:24 +01:00
Yann Collet
ae7aa06650 lots of minor refactorings 2016-02-03 02:46:46 +01:00
Yann Collet
7d968c7bdd move _usingDict() to stable API (zstd.h) 2016-02-03 02:11:32 +01:00
Yann Collet
464fa996e9 added v0.4 legacy support 2016-02-03 01:09:46 +01:00
inikep
5391a718a4 fixed 2016-02-03 00:04:14 +01:00
inikep
55bc9a868a 41374379 2016-02-02 19:47:09 +01:00
inikep
5d4131067b update 2016-02-02 17:41:21 +01:00
Yann Collet
7d8e6bddb9 changed long length format 2016-02-02 17:30:37 +01:00
inikep
d6e8cc5f0c 41403384 2016-02-02 16:43:31 +01:00
Yann Collet
7a6343fb1b fixed encoding bugs 2016-02-02 16:00:50 +01:00
inikep
e1b8e53e84 better logging 2016-02-02 15:41:16 +01:00
Yann Collet
2ce4923238 modified buffer management rules 2016-02-02 14:36:49 +01:00
inikep
59140c5480 turn off: last match must start at a minimum distance of 8 from oend 2016-02-02 12:36:09 +01:00
inikep
ef2706ed12 in progress 2016-01-31 15:32:01 +01:00
inikep
6b0715b8fd no "start" 2016-01-31 15:17:50 +01:00
inikep
d4c3e8c595 fixed logs 2016-01-31 14:28:26 +01:00
inikep
c0dc4c361d best_off 2016-01-31 12:36:41 +01:00
inikep
9bfb1a3357 working 2016-01-31 12:17:16 +01:00
inikep
ee4be5e981 ZSTD_compressBlock_opt_generic 2016-01-31 11:57:01 +01:00
inikep
e2bfe2407a GetAllMatches 2016-01-31 11:26:05 +01:00
inikep
c71568fd44 ZSTD_opt 2016-01-31 11:26:04 +01:00
Yann Collet
35f7de52c8 fixed Visual warnings 2016-01-31 02:51:03 +01:00
Yann Collet
61e16ce07c minor optimization for small files 2016-01-31 02:04:15 +01:00
Yann Collet
e93d6ced17 minor compression gain 2016-01-31 00:58:06 +01:00
Yann Collet
fd416f1eaf added : ZSTD_compress_usingPreparedCCtx() 2016-01-30 03:14:15 +01:00
Yann Collet
863ec40f1e minor compression speed improvement 2016-01-28 17:56:33 +01:00
Yann Collet
9cadd0853c add basic tests 2016-01-28 15:39:52 +01:00
Yann Collet
4ddb1f5566 optimized counting of small segments 2016-01-28 03:24:53 +01:00
Yann Collet
fb810d6053 changed dictionary format 2016-01-28 00:21:13 +01:00
Yann Collet
1c8e194297 modified streaming compression API 2016-01-26 16:31:22 +01:00
Yann Collet
7b51a2948f modified streaming decompression API 2016-01-26 15:58:49 +01:00
Yann Collet
b923f65076 introduced dictionary format 2016-01-26 03:14:20 +01:00
Yann Collet
bc4c8aa4b7 fixed new MIN_CBLOCK_SIZE 2016-01-25 17:26:01 +01:00
Yann Collet
3742219b4e fixed decompression bugs 2016-01-25 16:54:05 +01:00
Yann Collet
a1249dc8da fix visual warnings 2016-01-25 04:22:03 +01:00
Yann Collet
afe0709964 stream control 2016-01-25 04:10:46 +01:00
Yann Collet
59d1f79768 variable litblock header size 2016-01-23 19:28:41 +01:00
Yann Collet
a768a301ba fixed VS project 2016-01-21 16:04:35 +01:00
Yann Collet
ffec740d37 fixed visual and clang errors 2016-01-21 15:50:11 +01:00
Yann Collet
977f1f3600 modified error system, following suggestions by @nemequ 2016-01-21 15:38:47 +01:00
Yann Collet
74ce8e812c fixed VS 2015 'ERROR already defined', reported by Rich Geldreich 2016-01-21 14:32:41 +01:00
Yann Collet
7beaa05d81 fixed bt bug 2016-01-21 11:57:45 +01:00
Yann Collet
1840d6af33 bench pay attention to variable block sizes 2016-01-20 15:39:06 +01:00
Yann Collet
fba6aed175 changed : ZSTD_LEGACY_SUPPORT set to 0 by default within zstd_decompress.c (#113) 2016-01-18 12:03:54 +01:00
Yann Collet
a87278a4f0 small compression speed 2016-01-17 00:12:55 +01:00
Yann Collet
3a3b72f25a Implemented ZSTD_HEAPMODE for zstd_decompress.c 
Reduce cmake version requirement to 2.8.7
 2016-01-11 12:56:11 +01:00
Yann Collet
523b594233 fixed read error 2016-01-09 02:10:40 +01:00
Yann Collet
b012510d9f dictionary with block API 2016-01-09 02:00:10 +01:00
Yann Collet
bf42c8e5d8 block-level API 2016-01-09 01:08:23 +01:00
Yann Collet
c64c100658 fixed : unused variable 2016-01-08 17:54:02 +01:00
Yann Collet
600962711d added duplication tests 2016-01-08 17:27:50 +01:00
Yann Collet
6e1c4c6c65 fixed context copy 2016-01-07 23:07:44 +01:00
Yann Collet
ecd651bd91 added ZSTD_duplicateCCtx() 2016-01-07 15:35:18 +01:00
Yann Collet
494c786a3b fixed outstanding strict aliasing warnings 2016-01-06 12:54:02 +01:00
Yann Collet
1fdd82312d Fixed : strict aliasing warnings in legacy decoders 2016-01-06 12:35:42 +01:00
Yann Collet
218bd31942 minor strict aliasing warning 2016-01-06 02:19:55 +01:00
Yann Collet
3b994cb465 updated fse 2016-01-06 01:58:37 +01:00
Yann Collet
96ffa424eb minor fast mode improvement 2016-01-02 01:16:28 +01:00
Yann Collet
2cc12cb3cb minor refactoring 2016-01-01 07:47:58 +01:00
Yann Collet
1358f91ec4 minor refactoring 2016-01-01 07:29:39 +01:00
Yann Collet
72e84cfd0f minor simplification 2015-12-31 19:08:44 +01:00
Yann Collet
722504c2a0 update version to 0.4.6 2015-12-31 18:32:15 +01:00
Yann Collet
5f9f98768a Makefile version automatically follows zstd.h 2015-12-30 19:52:08 +01:00
Yann Collet
48da164f14 fixed clang warnings 2015-12-29 23:40:02 +01:00
Yann Collet
ee3f4515c1 faster hc mode with repetitive data (experimental) 2015-12-29 22:26:09 +01:00
Yann Collet
d94efbf95c fix fast mode on Windows 2015-12-29 14:29:08 +01:00
Yann Collet
1ac364d60b Merge pull request #99 from KrzysFR/dev 
Update version to 0.4.5 in zstdlib resources
 2015-12-21 10:19:26 +01:00
Yann Collet
ed73ae8c67 fix #101 2015-12-19 19:51:42 +01:00
Christophe Chevalier
8972f13b4a Added missing ZSTDLIB_API for ZSTD_decompress_usingDict and ZSTD_decompressDCtx 2015-12-18 11:20:37 +01:00
Yann Collet
31683c0b28 benchmark can use dictionary 2015-12-18 01:26:48 +01:00
Yann Collet
fdcad6d3e1 added ZSTD_compress_usingDict() 2015-12-17 23:50:15 +01:00
Yann Collet
deb078b9b1 added : -m : decompress multiple files 2015-12-17 20:30:14 +01:00
Yann Collet
f54f57023c update params for 256K 2015-12-16 19:38:54 +01:00
Yann Collet
0700585fb9 fixed asan warning 2015-12-12 12:54:48 +01:00
Yann Collet
188b6ed6c7 fixed g++ warning 2015-12-12 12:44:44 +01:00
Yann Collet
3a061a4c9f extended static dictionary to buffered mode 2015-12-12 11:17:42 +01:00
Yann Collet
6c3e2e7855 fixed : high compression modes for Windows 32 bits 2015-12-11 10:44:07 +01:00
Yann Collet
9f5ab1a4f5 fix 32-bits windows decoding on high-compression archives 2015-12-11 00:27:41 +01:00
Christophe Chevalier
2abb04d90d Added ZSTDLIB_API to new ZSTD_maxCLevel method 2015-12-09 23:55:23 +01:00
Christophe Chevalier
7b05324a71 Only define ZSTDLIB_API when building a Windows DLL 2015-12-09 17:12:12 +01:00
Christophe Chevalier
844baf4fc5 Merge remote-tracking branch 'upstream/dev' into zstdlib_api 2015-12-09 15:45:57 +01:00
Yann Collet
d608088ca3 added : ZSTD_maxCLevel() 
added : 256KB blocks mode
 2015-12-09 09:05:22 +01:00
Yann Collet
0cde77bc5e fixed comment (reported by @annulen) 2015-12-08 14:47:46 +01:00
Christophe Chevalier
d2199e7250 Added ZSTDLIB_API definition to zstd_buffered.h which does not seem to include zstd.h 2015-12-07 19:01:41 +01:00
Christophe Chevalier
c6e845398a Add ZSTDLIB_API macro to prefix all exported methods with__declspec(dllexport) when building a DLL (on windows) 2015-12-07 17:48:53 +01:00
Yann Collet
4bfe415fa7 validated external dictionary 2015-12-06 13:18:37 +01:00
Yann Collet
e47c4e5f8e strengthened bufferless streaming decompression 2015-12-05 09:23:53 +01:00
Yann Collet
417890cec2 strengthened streaming bufferless compression 2015-12-04 17:16:37 +01:00
Yann Collet
6a2f03295c update version to 0.4.2 2015-12-02 15:05:22 +01:00
Yann Collet
120230b895 compress uses less memory for small blocks 2015-12-02 14:00:45 +01:00
Yann Collet
a308259afa new : parameters for <= 16KB 2015-12-02 13:38:48 +01:00
Peter Harris
f06e238d6f Fix compression on Big Endian CPUs 2015-12-01 14:58:24 -05:00
Yann Collet
c620b48489 revert install to normal zstd due to travis limitation (likely gcc 4.6) 2015-12-01 01:56:02 +01:00
Yann Collet
44287a3bf4 refactored decompress (#82) 2015-11-30 23:13:56 +01:00
Yann Collet
aead1bf86b changed to ZSTD_LEGACY_SUPPORT=0, as suggested within #80 2015-11-30 14:18:47 +01:00
Yann Collet
daa1c65500 fixed #81 : removed dead file (reported by @inikep) 2015-11-30 12:25:48 +01:00
Yann Collet
a85a8dd94a Fixed #80 (reported by @luben) 
Added : tests for ZSTD_LEGACY=disable build mode
extended ZSTD_LEGACY=disable to dll build
 2015-11-30 11:53:11 +01:00
Yann Collet
375727310c fixed compression bug 2015-11-29 03:17:04 +01:00
Yann Collet
7fe531e7ec fixed address space overflow (32-bits mode) 2015-11-29 02:38:09 +01:00
Yann Collet
52732b505d fixed huff0 decoder (32-bits mode) 2015-11-28 22:03:08 +01:00
Yann Collet
bf7aa3c14e fixed decoder error (rollbuffer mode) 2015-11-28 18:19:44 +01:00
Yann Collet
ad50c59bb7 fixed decoder error (32-bits mode, malicious input) 2015-11-28 17:09:28 +01:00
Yann Collet
00fd7a2110 protect 32-bits mode 2015-11-28 16:03:22 +01:00
Yann Collet
53fbf01370 minor level change 2015-11-28 14:08:01 +01:00
Yann Collet
800fa6c378 stronger tests for zbuff decompression 2015-11-27 14:30:23 +01:00
Yann Collet
31d1806a77 zbuff properly supports small dst buffers 2015-11-27 14:07:36 +01:00
Yann Collet
944042ab7d coalesce block+header size (buffered mode) 2015-11-27 13:26:38 +01:00
Yann Collet
d248aa1c62 fixed hang bug (infinite loop) in decoder, buffered mode, malicious input 2015-11-26 22:39:29 +01:00
Yann Collet
661b94fb5d fixed potential warning on undefine, reported by Maciej Adamczyk 2015-11-26 16:42:06 +01:00
Yann Collet
29a58c94f6 fixed lib makefile 2015-11-26 16:25:56 +01:00
Yann Collet
26fa696713 minor API change 2015-11-26 16:07:08 +01:00
Yann Collet
29a2c838c5 support for legacy 0.3 format 2015-11-26 16:02:04 +01:00
Yann Collet
c2ce890ac2 fixed bug in legacy decoder v0.2, reported by Maciej Adamczyk 2015-11-26 14:12:23 +01:00
Yann Collet
26415d3ff2 fixed leak 2015-11-26 12:43:28 +01:00
Yann Collet
6bcdeac954 fixed asan warning 2015-11-26 11:43:00 +01:00
Yann Collet
5f2ec63852 fixed decompression bug (buffered mode) 2015-11-26 10:32:17 +01:00
Yann Collet
91a97960f7 Added zbufftest 2015-11-26 09:59:49 +01:00
Yann Collet
e4fdad55dc fixed minor decompression bug in buffered mode 2015-11-25 21:09:17 +01:00
Yann Collet
d3cb690156 fixed leak issue 2015-11-25 15:26:55 +01:00
Yann Collet
377b9e8b94 fixed : potential malloc error 2015-11-25 15:04:37 +01:00
Yann Collet
7b2c31043c fixed : potential malloc error 2015-11-25 15:02:46 +01:00
Yann Collet
88fcd2916e Added : zstd buffered API 2015-11-25 14:42:45 +01:00
Yann Collet
c36521571e fixed large buffers bug 2015-11-24 14:06:07 +01:00
Yann Collet
225179dd05 more tests (bt modes) 2015-11-23 16:52:22 +01:00
Yann Collet
239cc289d3 fixed asan bugs 2015-11-23 16:17:21 +01:00
Yann Collet
03526e18fa preliminary rollbuffer support for bt mode 2015-11-23 15:29:15 +01:00
Yann Collet
c1e52f0f05 hc simplication 2015-11-23 14:37:59 +01:00
Yann Collet
06eade5102 comments 2015-11-23 14:23:47 +01:00
Yann Collet
5054ee0cc0 rollbuffer refactor 2015-11-23 13:34:21 +01:00
Yann Collet
d7233d6e14 more compression tests 2015-11-22 14:40:51 +01:00
Yann Collet
287b7d9607 small hc speed improvement 2015-11-22 13:24:05 +01:00
Yann Collet
428619ccd7 small cLevel update 2015-11-22 12:46:30 +01:00
Yann Collet
a85c77b9d3 fixed roll buffer lazy2 mode 2015-11-22 12:22:04 +01:00
Yann Collet
b7fc88eec7 fixed roll buffer lazy mode 2015-11-22 03:12:28 +01:00
Yann Collet
734aa92ed5 fixed roll buffer greedy mode 2015-11-22 03:01:33 +01:00
Yann Collet
9a24e5986b fixed roll buffer in fast mode 2015-11-22 02:53:43 +01:00
Yann Collet
007c1c6ff0 minor cr improvement 2015-11-22 02:42:28 +01:00
Yann Collet
7a23179779 small fast compression improvement 2015-11-21 15:27:35 +01:00
Yann Collet
402fdcf1a3 fixed asan 2015-11-20 12:46:08 +01:00
Yann Collet
55aa7f94e3 modified format 2015-11-20 12:04:52 +01:00
Yann Collet
743402cb23 modified format 2015-11-20 12:03:53 +01:00
Yann Collet
7dfd56be4a restored fix small blocks 2015-11-19 17:46:29 +01:00
Yann Collet
c95f899725 refactored greedy 2015-11-19 17:28:35 +01:00
Yann Collet
b3a2af970e refactored decompression 2015-11-19 17:13:19 +01:00
Yann Collet
110cc14bab fixed bug #73 notified by @nemequ 2015-11-19 12:02:28 +01:00
Yann Collet
dcac00e7a6 update library builder 2015-11-19 11:23:44 +01:00
Yann Collet
b2549846ba more tests 2015-11-18 11:29:32 +01:00
Yann Collet
94b9d8ec8d fix compression bug in rollbuffer mode 2015-11-17 16:14:55 +01:00
Yann Collet
138e89c9c4 fix compression bug (extDict mode) 2015-11-17 14:26:54 +01:00
Yann Collet
786eb7d42e fixed repcode bug 2015-11-17 12:23:53 +01:00
Yann Collet
82368cfa1b comment 2015-11-16 19:10:56 +01:00
Yann Collet
eaa5f75b99 fixed decompression bug 2015-11-15 11:13:41 +01:00
Yann Collet
93a823ccb7 fixed asan warnings 2015-11-13 15:08:43 +01:00
Yann Collet
89db5e001d rolling buffer preliminary support for fast mode 2015-11-13 11:27:46 +01:00
Yann Collet
0f366c61ed fixed streaming decoder 2015-11-12 16:19:30 +01:00
Yann Collet
5b78d2f20c basic rolling buffer support in decoder 2015-11-12 15:36:05 +01:00
Yann Collet
14983e7aed refactored compression code 2015-11-11 21:38:21 +01:00
Yann Collet
7f6e91fa98 fixed library compilation 2015-11-11 14:39:50 +01:00
Yann Collet
5be2dd25f2 Changed Library source tree 
- no more zstdhc ; zstd.h is enough
- ZSTD_compress() now needs a compressionLevel
- separated zstd_compress.c and zstd_decompress.c
- updated zstdcli, fullbench, fuzzer with new API
 2015-11-11 13:43:58 +01:00
Yann Collet
530918b409 level tuning 2015-11-10 10:52:58 +01:00
Yann Collet
9f432926b4 fileio can use small blocks params 2015-11-09 17:42:17 +01:00
Yann Collet
43e0cd512a small blocks params 2015-11-09 16:38:17 +01:00
Yann Collet
4baee50728 fix bug 2015-11-09 03:19:33 +01:00
Yann Collet
d1ade5a61d fix 32-bits mode 2015-11-08 15:49:20 +01:00
Yann Collet
e9eba608c2 simplified bt 2015-11-08 15:08:03 +01:00
Yann Collet
31d00ba4b3 fix bt bug 2015-11-08 14:11:29 +01:00
Yann Collet
72a8e751f4 level tuning 2015-11-07 22:24:46 +01:00
Yann Collet
f48e35c206 minor bt improvements 2015-11-07 01:13:31 +01:00
Yann Collet
a81d9ac42d minor bt CSpeed 2015-11-06 19:03:59 +01:00
Yann Collet
628065cf98 revert 2015-11-06 18:44:54 +01:00
Yann Collet
9b8e453713 minor CRatio improvement 2015-11-06 18:28:02 +01:00
Yann Collet
47cfa9a985 minor CRatio improvement 2015-11-06 17:13:46 +01:00
Yann Collet
eb283710dd minor greedy CRatio improvement 2015-11-06 16:33:11 +01:00
Yann Collet
fc2afcfa64 small Cspeed increase 2015-11-06 15:40:14 +01:00
Yann Collet
444873f494 minor greedy mod 2015-11-06 11:48:56 +01:00
Yann Collet
805a52a773 fixed asan warning 2015-11-06 10:52:17 +01:00
Yann Collet
8ad12ec343 level tuning 2015-11-06 10:19:00 +01:00
Yann Collet
f12c1304a8 integrate fast mode 2015-11-05 18:16:59 +01:00
Yann Collet
1f44b3f778 added fast mode 2015-11-05 17:32:18 +01:00
Yann Collet
47b68908b2 rename lazy2 2015-11-05 15:14:17 +01:00
Yann Collet
9036105b62 simplified source 2015-11-05 15:03:12 +01:00
Yann Collet
5106a76dc9 merged strats 2015-11-05 15:00:24 +01:00
Yann Collet
43ae161fe9 level tuning 2015-11-05 11:58:02 +01:00
Yann Collet
92309cb37e level tuning 2015-11-05 00:48:37 +01:00
Yann Collet
3137d1a059 fix fuzzer32 litCSize limit condition 2015-11-04 23:36:36 +01:00
Yann Collet
50c5cdb44c fixed issues reported by Maciej Adamczyk 2015-11-04 20:35:33 +01:00
Yann Collet
3e3582719c Fixed issue #62, reported by @luben 2015-11-04 18:19:39 +01:00
Yann Collet
e8455f51ea fix clang warning 2015-11-04 17:41:20 +01:00
Yann Collet
b241e9deb7 small compression improvement 2015-11-04 13:57:24 +01:00
Yann Collet
59d7063fbc fix and level tuning 2015-11-04 12:05:27 +01:00
Yann Collet
96b9f0ba4d btlazy2 2015-11-04 03:52:54 +01:00
Yann Collet
588d1e5fa0 Fixed asan issue reported by Maciej Adamczyk 2015-11-03 10:48:42 +01:00
Yann Collet
050efba81b level tuning 2015-11-03 09:49:30 +01:00
Yann Collet
3564487614 lazydeep 2015-11-02 16:14:46 +01:00
Yann Collet
7671f393be v0.3.2 2015-11-02 12:17:39 +01:00
Yann Collet
4e049bca8c Fixed Visual Studio issue #60, reported by @mailagentrus 2015-11-02 11:09:54 +01:00
Yann Collet
56e1092cb0 level tuning 2015-11-02 01:02:43 +01:00
Yann Collet
9b11b46f8a Fixed speed regression 2015-11-01 12:40:22 +01:00
Yann Collet
be2010ea1b lazy strategy 2015-10-31 12:57:14 +01:00
Yann Collet
4b100f43b0 level tuning 2015-10-30 15:49:48 +01:00
Yann Collet
ec43ba4a95 updated visual studio projects 2015-10-30 11:51:26 +01:00
Yann Collet
aa074051d6 decoding support for legacy v0.2 format 2015-10-30 11:21:50 +01:00
Yann Collet
fc04bcc6d7 fixed minor gcc warning 2015-10-30 07:11:07 +01:00
Yann Collet
8f9d49c092 fixed asan warning 2015-10-30 07:00:37 +01:00
Yann Collet
4114f95ce9 fix Visual Studio projects 2015-10-30 06:40:22 +01:00
Yann Collet
44fe99116e fixed hc alloc 2015-10-29 22:02:40 +01:00
Yann Collet
712def905b one less hc malloc 2015-10-29 18:41:45 +01:00
Yann Collet
3d9cf7acf2 fix include 2015-10-29 17:15:14 +01:00
Yann Collet
71bcdb5c1f sync paramgrill with HC starting at level 2 2015-10-29 17:08:03 +01:00
Yann Collet
2acb5d3d48 HC modes can be selected from command line 2015-10-29 16:49:43 +01:00
Yann Collet
76c5c2a4a0 minor tuning 2015-10-28 19:07:05 +01:00
Yann Collet
81e49e6654 fine tune & messages 2015-10-28 15:56:48 +01:00
Yann Collet
b315bc86e1 srcLog 2015-10-27 13:12:25 +01:00
Yann Collet
2c6992e1dc blocksize & T 2015-10-27 12:18:00 +01:00
Yann Collet
d55a7e0e4d simplified chain 2015-10-27 03:06:40 +01:00
Yann Collet
8b91abec18 can test param conf 2015-10-27 02:59:12 +01:00
Yann Collet
342892c3ae repcode+1 2015-10-26 17:44:04 +01:00
Yann Collet
786f5b554a skip slow conf faster 2015-10-26 15:45:58 +01:00
Yann Collet
3a787da419 Fixed : minor clang warning 2015-10-25 19:42:03 +01:00
Yann Collet
083fcc82fc HC params 2015-10-25 14:06:35 +01:00
Yann Collet
53fff6cd9d Individual memlog values 2015-10-24 13:48:37 +01:00
Yann Collet
ed0a781246 repfirst 2015-10-23 19:25:06 +01:00
Yann Collet
83fae977e6 Added asan tests 2015-10-23 15:21:53 +01:00
Yann Collet
fc60883d42 fixed issue #49, reported by Hanno Böck (@hannob) 2015-10-23 12:45:02 +01:00
Yann Collet
eeb8ba18d1 uncoupled maxdlog and chainlog 2015-10-22 16:55:40 +01:00
Yann Collet
f3eca25322 quick first zstdhc version 2015-10-22 15:31:46 +01:00
Yann Collet
353c5d26cf unified ctx naming convention 2015-10-21 14:39:26 +01:00
Yann Collet
f42803e5b4 restored ZSTD_compressCCtx() def 2015-10-21 13:31:46 +01:00
Yann Collet
8eb50b8acf Fixed minor gcc5+ warning 2015-10-21 10:01:09 +01:00
Yann Collet
7010c27a13 Added advanced compression functions 2015-10-21 09:07:25 +01:00
Yann Collet
c8a7254192 clang optimizer bug workaround 2015-10-20 03:01:38 +01:00
Yann Collet
4caefaf62f refactored literal segment 2015-10-19 19:25:44 +01:00
Yann Collet
1fbee1f483 Fixed sanitizer warning 2015-10-19 15:32:47 +01:00
Yann Collet
4d1a0ef515 minor speed improvement 2015-10-19 03:12:23 +01:00
Yann Collet
c7ae9ffde3 small decompression speed improvement 2015-10-19 02:36:34 +01:00
Yann Collet
107c5755f4 small decompression speed improvement 2015-10-19 02:14:43 +01:00
Yann Collet
50174c77fd missed updated files 2015-10-18 22:26:26 +01:00
Yann Collet
b1f3f4b565 update huff0 2015-10-18 22:18:32 +01:00
Yann Collet
8f86c700cd Fix uninitialized warnings reported by @nemequ 2015-10-14 17:53:44 +01:00
Yann Collet
59aac5f467 Clarified comments 2015-10-14 16:28:19 +01:00
xaphier
77ee44c7b6 Remove one malloc/free from compression 
By making the buffer part of the zstd context structure, one malloc/free
can be removed from the compression.
 2015-10-10 12:14:51 +02:00
Yann Collet
be50aaa0ee Added : Frame concatenation ability 2015-09-10 23:26:09 +01:00
Yann Collet
5abd8203cb Fix : decompression i/o detects flush write errors 2015-08-27 03:16:04 +01:00
Yann Collet
866a3c3158 Fixed minor visual warning 2015-08-26 23:10:09 +01:00
Yann Collet
352ab88aea fixed minor issue on decoding invalid data 2015-08-26 22:45:23 +01:00
Yann Collet
6b5198fe3a Updated FSE 2015-08-26 19:22:01 +01:00
Yann Collet
bf50409e05 Refactor Makefile and .travis tests 2015-08-26 00:19:06 +01:00
Yann Collet
f44b2b02b4 Zstd cli : --version and --help write to stdout 2015-08-25 23:32:45 +01:00
Yann Collet
c705567522 Fixed : compression error on some specific distribution 2015-08-25 22:56:42 +01:00
Yann Collet
1885029ba1 Fixed a few issues found by AFL (American Fuzzy Lop) 2015-08-24 20:17:11 +01:00
Yann Collet
fee8e240c7 Fixed decoding error #11 (reported by @magv) 2015-08-24 15:47:04 +01:00
Yann Collet
d5d9bc3f82 Added : ZSTD decompression : ability to resist to noisy/faulty data source 2015-08-23 23:13:49 +01:00
Yann Collet
d02114e0e1 updated fse 2015-08-21 03:59:31 +01:00
Yann Collet
997f9ee2ef Added : fuzzer tests : decompressing noisy src 2015-08-21 02:44:20 +01:00
Yann Collet
7d20acd340 Fix : decoder issue in exceptionnal circumstances (dst buffer too small) 2015-08-20 15:55:50 +01:00
Yann Collet
602834f794 Fixed : bug in compression in specific conditions (too small dst size) 2015-08-20 07:46:10 +01:00
Yann Collet
23743530e0 Updated fse 2015-08-19 23:53:56 +01:00
Yann Collet
f4ce8913a3 Added more strigent tests : compresson into too small buffer 2015-08-11 14:18:45 +01:00
Yann Collet
f90314610d Minor decompression simplification 2015-08-08 02:54:26 +01:00
Yann Collet
8a46c159c5 minor compress sequence refactor 2015-08-08 02:16:11 +01:00
Yann Collet
e9853b2cdb Fixed : ZSTD_compress() can attempt compression on a too small buffer 2015-08-07 19:07:32 +01:00
Yann Collet
96f3b2c0f7 zstd_compress() no longer requires maxDstSize >= ZSTD_compressBound(srcSize) 2015-08-07 15:50:42 +01:00
Yann Collet
a787550d1c Updated FSE lib 2015-08-07 15:21:00 +01:00
Yann Collet
77c82b680b updated fse 2015-08-02 01:19:09 +01:00
Yann Collet
138db2187d Fixed some minor sanitizer warnings within huff0 2015-07-27 20:19:21 +01:00
Yann Collet
fb8296f55c Updated fse (faster huff0 32-bits decoding) 2015-07-27 19:34:27 +01:00
Yann Collet
e8c6bb1e42 Integrated huff0 (breaking format change) 2015-07-26 00:23:57 +01:00
Yann Collet
968f275981 Changed allocation strategy to reduce stack usage of ZSTD_compressSequences() 2015-07-07 16:54:25 -08:00
Yann Collet
858db1291f Fixed : ZSTD_compress() corrently prevents buffer overflow 2015-07-07 01:02:04 -08:00
Yann Collet
dc6c1d920a Fixed clang warning 2015-07-07 00:44:22 -08:00
Yann Collet
674d91b3da Added sanitize=undefine test 2015-07-07 00:36:49 -08:00
Yann Collet
17867ced03 Fixed minor clang warnings 2015-07-07 00:14:27 -08:00
Yann Collet
7083b8a54a Travis CI : added ARM compilation tests 2015-07-06 23:58:59 -08:00
Yann Collet
ee28a0def4 Fixed issue #34 and #31 2015-07-06 23:45:35 -08:00
Yann Collet
94f998b1fc Fixed a few visual analyzer warnings 2015-07-04 23:10:40 -08:00
Yann Collet
7393c5a51d Advanced API change : Cctx & DCtx are now incomplete types for stricter type checking 2015-07-04 18:20:42 -08:00
Yann Collet
1efa31fad8 Updated FSE lib 2015-07-04 15:56:41 -08:00
Yann Collet
078a9a2804 Added Wcast-qual compilation flag 
Updated xxHash
Removed log traces
 2015-06-20 19:37:53 -08:00
Yann Collet
213089c078 Updated FSE 
Added clang and g++ tests
 2015-06-18 07:43:16 -08:00
Yann Collet
bbfa7d77c9 Fixed issue #28 2015-03-23 23:15:22 +01:00
Yann Collet
ad68a0e5f7 "type redef fix" for gcc <= 4.4 2015-02-26 00:29:15 +01:00
Yann Collet
26aa1ec946 updated FSE 2015-02-24 09:05:58 +01:00
Yann Collet
a3c75bad5a Updated FSE 
Faster speed on barely compressible data
 2015-02-21 03:31:59 +01:00
Yann Collet
00be3437a8 Some comments, to explain streaming decompression API 2015-02-20 18:53:31 +01:00
Yann Collet
1db6f22f34 Removed useless constants 2015-02-17 13:38:44 +01:00
Yann Collet
c5d46b5c90 New streaming API behavior, to solve issue 19 (https://github.com/Cyan4973/zstd/issues/19) 2015-02-16 18:06:26 +01:00
Yann Collet
fb98fd0bd4 datagen refactoring 2015-02-10 18:15:20 +01:00
Yann Collet
e30c39cd1b internal refactoring - better separation between match & entropy 2015-02-09 03:50:11 +01:00
Yann Collet
2ddf7e9a28 Fixed : FSE issue with corner case distributions 2015-02-08 20:26:47 +01:00
Yann Collet
d6914bfd5a fix minor Visual warning 2015-02-01 12:09:46 +01:00
Yann Collet
aacace37f7 Fixed a few clang warnings 2015-02-01 11:57:30 +01:00
Yann Collet
b0802817c0 avx2 mode solely detected using __AVX2__ macro 2015-02-01 10:13:22 +01:00
Yann Collet
e5c25e0ebc minor fix for Visual 2015-01-31 20:10:53 +01:00
Yann Collet
936e678dda Fixed : issue 8 regarding huge input buffer (> 4 GB) 2015-01-31 15:09:51 +01:00
Yann Collet
439eb77088 Normalize all the line endings 2015-01-31 10:52:59 +01:00
Yann Collet
1cc58def2d Restored cast for C++ (fse) 2015-01-29 07:13:54 +01:00
Yann Collet
6434adcf54 Fix : no AVX2 for Visual 2008 and older 2015-01-29 07:00:43 +01:00
Yann Collet
565b81d0ba fixed : corner case (FSE) 2015-01-29 06:51:30 +01:00
Yann Collet
e98ea20c37 Fix for VS2005 (AVX2 intrinsics) 2015-01-28 01:23:14 +01:00
r-lyeh
0da7941a72 Bugfixed Visual Studio errors when compiled as C++ sources 2015-01-26 10:24:04 +01:00
Yann Collet
fb81417323 Fixed : minor warnings under Visual 32 bits 2015-01-25 13:19:12 +01:00
Yann Collet
759433d679 Updated fullbench tests 2015-01-24 13:31:55 +01:00
Yann Collet
04d8e15e57 fixed : file permissions 2015-01-24 11:22:13 +01:00
Yann Collet
4856a00164 Initial release 2015-01-24 01:58:16 +01:00