townforge/lz4
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
2,868 Commits 1 Branch 0 Tags 5.8 MiB
cc
Commit Graph

1037 Commits

Author SHA1 Message Date
Alexander Mohr
9a42a9db94 dict-size: make lz4 context const 
change the context to const to make
clear that the context is not modified
 2022-06-11 23:58:03 +02:00
Alexander Mohr
3c57d2f185 lz4frame: fix different linkage error 
Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
 2022-06-11 22:47:50 +02:00
Alexander Mohr
1738b50443 fuzz-test: add fuzz test for uncompressed api 
add a fuzzing test for uncompressed frame api

Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
 2022-06-11 22:47:44 +02:00
Alexander Mohr
5c7382798e
review: Fix review findings 
This commit fixes the review findings

Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
 2022-06-10 06:37:57 +00:00
Alexander Mohr
62f6cef564
review: Fix review findings 
This commit fixes the review findings

Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
 2022-06-10 06:00:38 +00:00
Alexander Mohr
4aeb5020c3
frame-api: add method to insert uncomressed data 
new method `uncompressed_update` allows to insert blocks without
compression into the lz4 stream.
The usage is documented in the frameCompress example

Signed-off-by: Alexander Mohr <alexander.m.mohr@mercedes-benz.com>
 2022-06-09 17:08:01 +02:00
Qi Wang
b1e84b38ef API: add LZ4_decompress_safe_partial_usingDict API 
feature request: #1051

Signed-off-by: Qi Wang <wangqi@linux.alibaba.com>
 2022-06-07 17:12:19 +08:00
Yann Collet
97d60acd89 updated documentation of LZ4F_freeCompressionContext 
to answer #1090
 2022-06-05 11:21:28 -07:00
Yann Collet
3bf71d35d4 introduced FORWARD_IF_ERROR() 
and removed prefix from RETURN_ERROR(_IF)
to improve brevity
as it's just a local macro (no bleeding in user's namespace).
 2022-05-09 21:45:07 -07:00
Yann Collet
5d0adabd28 introduced macro LZ4F_RETURN_ERROR_IF 2022-05-06 18:45:41 -07:00
Yann Collet
80ececaaed introduced macro LZ4F_RETURN_ERROR 2022-05-06 18:35:06 -07:00
Yann Collet
ebfad2da84 removed ERROR_GENERIC from lz4frame.h 
created a new error code in the process : LZ4F_ERROR_compressionState_uninitialized
 2022-05-06 18:19:58 -07:00
Yann Collet
ce8ee024b2 update library version to v1.9.4 
to reduce confusion in traces between dev branch and latest v1.9.3 release.
 2022-04-17 10:06:18 -07:00
Yann Collet
da75a4096e updated one error code 2022-04-06 06:38:21 -07:00
Yann Collet
90d68e3709 updated documentation around liblz4 granularity 2022-03-21 16:48:40 -07:00
anjiahao
f5625f260e add file operation and examples 
operate lz4 compressed files as a general files

Signed-off-by: anjiahao <anjiahao@xiaomi.com>
 2022-03-10 13:28:34 +08:00
Yann Collet
4f984e45a5 added target test-compile-with-lz4-memory-usage 
and run it in GA CI
 2022-01-31 14:51:20 -08:00
Yann Collet
379c1a10ca Introduce MIN and MAX bounds to LZ4_MEMORY_USAGE 
ensure that `frametest` works fine with these values,
notably with low LZ4_MEMORY_USAGE (dict test notably)

following suggestions from @t-mat at #1016
 2022-01-30 23:02:55 -08:00
Yann Collet
756693083a minor comments improvements to lz4frame 2022-01-30 21:59:56 -08:00
Yann Collet
b3a7ca21ce
Merge pull request #1057 from lz4/lz4_unalign 
renamed unaligned -> LZ4_unaligned
 2022-01-28 22:35:31 -08:00
Yann Collet
77bef09693 renamed unaligned -> LZ4_unaligned 
better name space isolation
suggested by @boris-kolpackov in #1053
 2022-01-28 21:24:12 -08:00
Yann Collet
e4f5a34d16 fixed bug in optimal parser 
discovered by @yoniko.
 2022-01-28 16:51:28 -08:00
Ben Niu
c2d00cb97b
Add comments 2022-01-27 11:04:06 -08:00
Ben Niu
bee421b93d
Disable _tzcnt_u64 for ARM64EC 
The ARM64EC is a new Microsoft-designed ARM64 ABI that is compatible with AMD64 code. However, not all AMD64 intrinsic functions are supported. For, intrinsics that are lowered to AVX, AVX2 and AVX512 instructions are not supported, including the _tzcnt_u64. To make sure this file compiles for ARM64EC, the use of _tzcnt_u64 should be neutered.
 2022-01-26 18:02:31 -08:00
Dimitri Papadopoulos
4bdfb08b95
Fix typos found by codespell 2021-11-25 15:32:37 +01:00
Arseny Kapoulkine
22e232dada Enable fast decoding on Apple/AArch64 builds 
This makes decoding significantly faster on M1; measured on compressed source
code across 8 hardware threads, decompressing 294 MB to 1301 MB takes 513 ms
of cumulative work (2.53 GB/s) before, and 406 ms (3.2 GB/s) after this change
on M1 Pro.

There's no way to check if the target architecture is M1 specifically but the
gains are likely to be similar on recent iterations on Apple processors, and
the original performance issue was probably more specific to Qualcomm.
 2021-11-22 11:23:41 -08:00
a1346054
3e99d07bc0 trim excess whitespace 2021-08-21 22:24:52 +00:00
Zeyi (Rice) Fan
1a2e8f4e30 fix clang-cl _tzcnt_u64 not defined issue 2021-08-09 11:30:02 -07:00
Eddy Jansson
0a0922067f Don't reuse state memory that's too small. 
Ensure that the memory block we're trying
to reuse is large enough for the new state.

Fixes #974
 2021-07-31 19:05:29 +02:00
Eddy Jansson
80e3e7034e Define LZ4_STREAMSIZE in terms of LZ4_MEMORY_USAGE 
This is required to correctly size a static member
to hold the hash table, whose size is derived from
LZ4_MEMORY_USAGE.
 2021-07-31 18:59:15 +02:00
Yann Collet
4de56b3da3
Merge pull request #1012 from eloj/print-install-dirs 
Print target directories during 'make install'.
 2021-07-25 00:44:40 +02:00
Eddy Jansson
eba110ad5b Print target directories during 'make install'. 
This takes #975 to its logical conclusion.
 2021-07-24 17:20:37 +02:00
Eddy Jansson
c1f514f3db Expand use of pkg-config variables. 
Change pkg-config generation such that the path variables,
not their values, are used in the definitions of Libs and Cflags,
and that $prefix is substituted into libdir and includedir iff
they start with its value.

This makes it easier to modify the already installed file if necessary.
 2021-07-24 16:05:15 +02:00
TotalJustice
2216d45e04 fix LZ4HC_HEAPMODE macro guard 2021-06-25 15:16:22 +01:00
Gabe Jones
b3d46ae07f Cast ALLOC return value to satisfy C++ 2021-06-15 12:09:14 -05:00
Yann Collet
dfc431fb3d fix NULL ptr arithmetic at lz4:2299 2021-05-28 01:10:41 -07:00
Yann Collet
539c783c98 fix NULL ptr arithmetic in lz4:1680 
only do arithmetic if offset > 0
 2021-05-28 01:08:18 -07:00
Yann Collet
59273b7127 fix UB lz4:988 and lz4:1178 
ensure `dictBase` is only used
when there is an actual dictionary content.
 2021-05-28 00:56:26 -07:00
Yann Collet
c2c0c47d5f fix NULL ptr arithmetic of lz4:1572 
was blindly adding an offset (0) to `dictionary` which could be `NULL`.
 2021-05-27 23:20:28 -07:00
Yann Collet
8e46846287 fix UB of lz4frame:907 
now line 912
by ensuring pointer arithmetic is only performed
if there is a reason for an internal buffer to be used.
 2021-05-27 22:59:22 -07:00
Yann Collet
7a966c1511
Merge pull request #972 from jasperla/memmove_crash 
Fix potential memory corruption with negative memmove() size
 2021-04-30 08:56:25 -07:00
klebertosantos
29a6a1f494 fix null pointer dereference 2021-03-10 09:37:57 -03:00
Jasper Lievisse Adriaanse
8301a21773 Fix potential memory corruption with negative memmove() size 2021-02-26 15:21:20 +01:00
Yann Collet
a8e888774a
Merge pull request #964 from sigiesec/fix-ubsan-resetStreamHC_fast 
Don't trigger UBSan warning in LZ4_resetStreamHC_fast if LZ4_streamHC…
 2021-01-19 02:54:18 -08:00
Thomas Waldmann
909aae8260 fix some typos (work by Andrea Gelmini) 2021-01-07 18:39:57 +01:00
Simon Giesecke
06b07e175e Don't trigger UBSan warning in LZ4_resetStreamHC_fast if LZ4_streamHCPtr->internal_donotuse.end is NULL. 2021-01-07 10:21:48 +01:00
Yann Collet
50ff8b3ae8 fix minor header date 2020-11-30 18:16:00 -08:00
Yann Collet
e585a438c7 refactor Makefile 
remove usage of include Makefile.inc in too Makefile
as it seems to somehow unexport CFLAGS ...
 2020-11-30 16:06:50 -08:00
Yann Collet
87a80acbe7 updated license & header dates 2020-11-25 14:45:14 -08:00
Yann Collet
0760724577 update doc 2020-11-15 01:31:09 -08:00
Yann Collet
f61b034cd7 fix minor win32 warning 2020-11-14 20:39:23 -08:00
Yann Collet
e0f85f2fc8 better visual conformance 
only include <intrin.h> on vs2005+ (#947)
remove some useless #pragma
fix a few minor Visual warnings
 2020-11-14 16:27:57 -08:00
Yann Collet
468011c346 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev 2020-11-14 14:26:48 -08:00
Yann Collet
581c459b4e restrict BitScanForward() to VS2005+ 
suggested by @aqrit in #947
 2020-11-14 14:26:08 -08:00
Yann Collet
b7c6fc9f61 minor refactor of lz4io 
for readability.
Mostly around --list capability
 2020-11-13 10:57:40 -08:00
Yann Collet
86f5e9d0f4 minor : lz4frame include stdlib only if needed 2020-11-09 21:23:56 -08:00
Yann Collet
a296839802 changed LZ4_calloc() to a 2-arguments signature 
to remain similar to stdlib's calloc().

Updated test to use c++ compiler for stricter signature check.
 2020-11-09 08:44:26 -08:00
Yann Collet
44b13db532 Merge branch 'dev' into customMem 2020-11-08 23:26:19 -08:00
Yann Collet
52646e8d75 first proposal for LZ4_USER_MEMORY_FUNCTIONS 
makes it possible to replace at link time
malloc, calloc and free
by user-provided functions
which must be named LZ4_malloc(), LZ4_calloc() and LZ4_free().

answer #937
 2020-11-08 21:17:32 -08:00
Yann Collet
9cf3f106a8
Merge pull request #944 from lz4/fix874 
fix #874
 2020-11-08 20:48:20 -08:00
Yann Collet
b8cd01183f
Merge pull request #945 from lz4/fix926 
fixed remaining ubsan warnings
 2020-11-08 20:47:19 -08:00
Yann Collet
2a2b10f192 fixed remaining ubsan warnings 2020-11-08 18:08:43 -08:00
Yann Collet
2964b8a6f6 fix #874 
coverity reported a warning regarding a memcpy() overwrite.
This is a false positive (the memory area is large enough),
but it's true that it's not trivial to determine (encompassing struct),
and it's proper anyway to only memcpy() just the right amount of data.
 2020-11-08 13:21:58 -08:00
Yann Collet
be634559e3 update obsolete section documentation 
and update manuals.
fix #712
 2020-11-08 12:54:59 -08:00
Yann Collet
e251a84025 fix minor UBs 
- check alignment before casting a pointer
- saveDict : don't memmove() on NULL dst
 2020-11-07 19:42:57 -08:00
Yann Collet
a2222a879f fix #926 
fix incorrect behavior of LZ4_saveDictHC()
when invoked right after initialization.
 2020-11-07 18:12:26 -08:00
Yann Collet
1d02141bf8
Merge pull request #941 from lz4/revertinline 
Revert "Replace "static" to "LZ4_FORCE_INLINE" for small functions"
 2020-11-07 17:18:04 -08:00
Yann Collet
f61eeb7793 Revert "Replace "static" to "LZ4_FORCE_INLINE" for small functions" 
This reverts commit 0e3933edd4.
 2020-11-07 11:02:30 -08:00
Yann Collet
d4bfcf8489 fix #935 
minor: identical declaration and prototypes of `LZ4HC_compress_optimal()`

also :
very minor optimization of `LZ4_memcpy_using_offset()`
 2020-11-07 10:06:52 -08:00
Yann Collet
27959b43ec LZ4F_decompress requires a valid dctx state 
This is now explicitly documented and asserted.
fix #927
 2020-11-06 20:46:35 -08:00
Yann Collet
b5e2a4acd9
Merge pull request #936 from lz4/alignTest 
More alignment tests
 2020-11-06 20:27:42 -08:00
Yann Collet
67e661a2ad static state size 
for better inter-version compatibility
 2020-11-06 18:26:13 -08:00
Yann Collet
211d653ff8 re-enable alignment test on all targets 2020-11-06 16:43:14 -08:00
Yann Collet
0c56f838ed unified internal state declaration 
align on `void*` instead : there is no `long long` inside the structure
 2020-11-06 16:15:51 -08:00
Yann Collet
78433070ab document LZ4_ALIGN_TEST 2020-11-06 14:48:43 -08:00
Yann Collet
e968a24129 unified alignment test 
across lz4.c and lz4hc.c
 2020-11-06 14:46:48 -08:00
Yann Collet
664427aa8f
Merge pull request #930 from remittor-pr/fix_msvc 
Fix: The "inline" specifier do not use for LZ4_wildCopy8 and LZ4_wildCopy32
 2020-10-31 01:44:21 -07:00
remittor
0e3933edd4 Replace "static" to "LZ4_FORCE_INLINE" for small functions 
The "static" specifier does not guarantee that the function will be inlined.
 2020-10-07 09:52:40 +03:00
remittor
749bd91a06 Replace define LZ4_FORCE_O2_INLINE_GCC_PPC64LE to LZ4_FORCE_INLINE 
There is no reason to separate these two definitions!
 2020-10-07 09:51:08 +03:00
remittor
c4792cdfa9 Fix: The "inline" specifier do not use for LZ4_wildCopy8 and LZ4_wildCopy32 
This problem was reproduced on MSVC 2015 (32-bit). Both functions were called using the operator "call".
 2020-10-06 17:16:43 +03:00
remittor
c24e979063 [lz4hc] Made function LZ4HC_encodeSequence a human readable 2020-10-03 18:32:34 +03:00
Yann Collet
2631002f74 preserver alignment test on Visual Studio x64 
this it works fine in this environment
(only x86 is suspicious)
 2020-10-02 15:51:14 -07:00
Yann Collet
1e5ff76b36 add LZ4F_decompress() tests with (NULL,0) input and output 
fix one (rare & complex) issue discovered by this test
 2020-10-01 21:10:08 -07:00
Yann Collet
03f006a7ea make scan-build accept assert() 2020-10-01 10:48:22 -07:00
Yann Collet
1e31f1d25c fix bad init scenario 2020-09-30 22:51:17 -07:00
Yann Collet
fe74eb2031 added memcpy() related SA warning fixes 
memcpy() on NULL is UB, even if length is 0.
 2020-09-30 21:20:07 -07:00
Yann Collet
7d21f761c3 fix conversion warning 2020-09-29 21:53:42 -07:00
Yann Collet
40cc3aa9a2 Merge branch 'dev' into safixes 2020-09-29 17:21:59 -07:00
Yann Collet
ad2d2764c7 fix minor static analyzer warnings 
detected by scan-build, cppcheck and advanved compilation flags
fix #786
 2020-09-29 17:20:52 -07:00
Yann Collet
293713a4fa bump version number 
to v1.9.3
 2020-09-29 14:27:39 -07:00
Yann Collet
78f4fdbb89
Merge pull request #923 from lz4/fix784 
fix efficiency of LZ4_compress_HC_destSize()
 2020-09-28 14:04:56 -07:00
Yann Collet
ab89dda91d improved last literals run on LZ4_compress_destSize 
applying new more accurate formula from LZ4_compress_HC_destSize()

also : fix some minor display issue in tests/frametest
 2020-09-28 11:39:00 -07:00
Yann Collet
89736e4e27 ensure last match not too close to end 
must respect MFLIMIT distance from oend
 2020-09-27 23:59:56 -07:00
Yann Collet
a13c79d56d fix incorrect counting 
after truncation of last sequence
 2020-09-27 22:47:05 -07:00
Yann Collet
8a362a8ac8
Merge pull request #921 from lz4/doubleNull 
fix compressing into NULL
 2020-09-27 21:09:06 -07:00
Yann Collet
e7fe105ac6 fix efficiency of LZ4_compress_HC_destSize() 
LZ4_compress_HC_destSize() had a tendency
to discard its last match when this match overflowed specified dstBuffer limit.
The impact is generally moderate,
but occasionally huge,
typically when this last match is very large
(such as compressing a bunch of zeroes).

Issue #784 fixed for both Chain and Opt implementations.

Added a unit test suggested by @remittor checking this topic.
 2020-09-27 21:04:40 -07:00
Anton Kochkov
9730d91110 Fix compilation with TinyCC 2020-09-27 17:07:51 +08:00
Yann Collet
ee4f37d284 fix compressing into NULL 
fails properly
bug discovered by oss-fuzz
 2020-09-26 11:31:57 -07:00
Yann Collet
dc27b7ae41 comment bug on older versions of ZSTD_compress_destSize() 
following investigation in #859
 2020-09-17 20:59:01 -07:00
Yann Collet
10d2e1c694 fixed lz4frame with blocks of size 1 
properly track history
 2020-09-17 14:43:02 -07:00