Commit Graph

612 Commits

Author SHA1 Message Date
Yann Collet
392809d66e fix minor static analyzer warnings
detected by scan-build and cppcheck
fix #786
2020-09-29 17:09:58 -07:00
Niklas Hambüchen
a58b7ac6ce fixup: Portable fileno() 2020-08-25 03:37:41 +02:00
Niklas Hambüchen
86edeb6784 fixup: C90 compliance, clean up stat() types/calls 2020-08-25 03:36:57 +02:00
Niklas Hambüchen
9a7658070a Use fstat() to determine file size.
This allows us to get the file size even when the input file is passed
via stdin. This fixes `--content-size` not working in situations like

    $ lz4 -v --content-size < /tmp/test > /tmp/test.lz4
    Warning : cannot determine input content size

With this change, it works.

Also helps with #904.
2020-08-24 06:27:28 +02:00
Yann Collet
3dd34df751 added target lz4-wlib
variant of lz4 linking to liblz4 dynamic library

requires the dynamic library to expose static-only symbols (experimental API)

Example for #888
2020-08-11 14:07:51 -07:00
Filipe Calasans
781417a369 Implement -m option with legacy format on cli 2020-02-07 17:03:13 -08:00
Filipe Calasans
ac4940cd98 Fix lz4cli --list option 2020-02-06 21:39:54 -08:00
Jerome Duval
935b432be1 bring support for Haiku up to date. 2019-11-07 14:32:38 +01:00
Reto Koradi
cc91777c98 Make benchmark compatible with dictionary compression
Support the -D command line option for running benchmarks. The
benchmark code was slightly restructured to factor out the calls
that need to be different for each benchmark scenario. Since there
are now 4 scenarios (all combinations of fast/HC and with/without
dictionary), the logic was getting somewhat convoluted otherwise.

This was done by extending the compressionParameters struct that
previously contained just a single function pointer. It now
contains 4 function pointers for init/reset/compress/cleanup,
with the related state. The functions get a pointer to the
structure as their first argument (inspired by C++), so that they
can access the state values in the struct.
2019-11-05 23:38:00 -08:00
Rosen Penev
a55095ddd6
util.h: Remove deprecated utime for non-Windows
utime was deprecated in POSIX 2008.
2019-09-10 11:29:05 -07:00
WHR
eee8cc79e7 lz4cli: add option '--best' as an alias of '-12' 2019-07-23 13:37:11 +08:00
Yann Collet
bb5c34a875 bumped version number to v1.9.2
to reduce risks that future bug reports in `dev` branch report `v1.9.1` as the failing version.
2019-07-01 09:01:43 -07:00
Yann Collet
baf9b0e043 fix #734 : --version should output to stdout
instead of stderr
2019-06-24 16:08:30 -07:00
gabrielstedman
729eef61a1 Handle file not existing case #704 2019-05-26 15:04:45 +01:00
gstedman
81081f9b2d List mode improvements.
Improve formatting
  Include static assert
  Use UTIL_fseek to handle potential fseek limitation
  Be explicit when refusing to read from stdin
  Properly free dctx after use
  Include valgrind tests
2019-05-16 20:53:29 +01:00
gstedman
98a86c8ef6 Add multiframe report to --list command 2019-05-15 21:13:19 +01:00
JPeterMugaas
7136489d33 More build imrpvements
Moved a few other tests to Makefiles.inc.  Other things might need to go there.
Made a test for symlink appropriateness.  Windows can NOT handle them the same way Unix-like operating systems do (if at all).  This is mostly the same as the Visual C projects.
embed version info into .dll and .exes that are redistributed.
2019-04-23 07:44:00 -04:00
Yann Collet
291b3d8b7e
Merge pull request #691 from lz4/list
--list
2019-04-22 17:43:01 -07:00
Yann Collet
35b83a921f fix: no leak when LZ4F_dctx creation fails
strange, because it previous implementation, it would `exit()`,
so it should not matter ...
2019-04-22 16:06:22 -07:00
Yann Collet
c4fe7a2047
Merge pull request #689 from JPeterMugaas/jpm-makefile
Jpm makefile - as described in https://github.com/lz4/lz4/issues/688
2019-04-22 15:29:50 -07:00
Yann Collet
5a50247d5f updated code comments 2019-04-22 15:24:44 -07:00
Yann Collet
a685d5cffc --list gives block type 2019-04-22 15:14:53 -07:00
Yann Collet
f401f1de7e --list can resume after a failed file 2019-04-22 15:01:20 -07:00
JPeterMugaas
5e6807fd95 Make programs/Makefile use the includes. 2019-04-22 17:38:43 -04:00
Yann Collet
0d4c885abb refactored --list function
better handling of special conditions,
better scoping of variables.

Also : updated man page
2019-04-22 14:01:19 -07:00
Yann Collet
ae5cea9112 fixed C90 compliance
re-structure code, have everything into a single section of lz4io.c
2019-04-22 12:27:25 -07:00
JPeterMugaas
773b66547f Initial commits from diff I submitted earlier 2019-04-22 13:48:59 -04:00
gabrielstedman
84f978a2f3 FR #598 - Correctly initialize cfinfo & cast malloc res to (char*) 2019-04-22 11:07:08 +01:00
gabrielstedman
e31b6dc03c FR #598 - Make fileSize unsigned long long 2019-04-22 10:35:29 +01:00
gabrielstedman
0acebbe53d FR #598 - Move LZ4F_compFileInfo_t def to lz4frame.h discard output_file
assert if in om_list mode
2019-04-22 10:02:46 +01:00
gabrielstedman
9d0960c354 Merge https://github.com/lz4/lz4 into dev 2019-04-22 09:56:51 +01:00
gabrielstedman
2133366da0 FR #598 - Make LZ4IO_getCompressedFileInfo internal and reword func 2019-04-22 09:00:20 +01:00
Yann Collet
22fae16c6f ensure tests work when stdout is not the console
ensure this case is continuously tested on travis.

Update documentation on implicit output,
invite to not rely on implicit output in scripts.
2019-04-21 17:01:03 -07:00
gabrielstedman
5fdc6d201e FR #598 Improve initial design, test mallocs, support C90. 2019-04-21 18:20:40 +01:00
gabrielstedman
55484191c4 correctly use unisgned int for index 2019-04-21 00:20:38 +01:00
gabrielstedman
55ab3c48ae Add --list option to display compressed file information. 2019-04-21 00:01:03 +01:00
Yann Collet
5f8ac02b77 cli: display a warning whenever default output is stdout while input != stdin
This behavior has been preserved for compatibility with existing ecosystem.
But it's problematic, as some environment start `lz4` without identifying stdout as console by default,
leading to a change of behavior for a same line of script.

A more sensible policy would be to default to stdout only when input is stdin.

Soft change for the time being : keep the behavior, just print a warning message.
User should prefer `-c` to explicitly select `stdout`.

Also : updated tests in Makefile to explicitly select `stdout` with `-c`.
2019-04-19 17:08:40 -07:00
Yann Collet
4e4f1ad623 ensure list of names is large enough 2019-04-18 17:26:01 -07:00
Yann Collet
4f6de46f60 fix several minor static analyzer warnings 2019-04-18 12:05:50 -07:00
Yann Collet
63bfb64b80 and even more visual conversion warnings 2019-04-12 17:40:23 -07:00
Yann Collet
8c2429ed1f some more Visual conversion warning 2019-04-12 17:06:53 -07:00
Yann Collet
c7554c3004 fixed minor Visual conversion warnings 2019-04-12 17:03:28 -07:00
Yann Collet
9c49e3ca07 added tests for -d -m -c
seems to work properly
2019-04-12 16:10:55 -07:00
Yann Collet
e865a3e7ec fixed lz4 -m -c
can compress multiple files into stdout
2019-04-12 15:17:48 -07:00
Yann Collet
20c2a5fc94 added command -BI for "Independent Blocks"
This is the reverse of `-BD`, and the current default.

This command can be useful to reverse a previous `-BD` command.
It may in the future be more important
if `lz4` switches to generating dependent blocks by default.
2019-04-12 13:27:11 -07:00
Yann Collet
352bbdb98e updated man page
with --favor-decSpeed
2019-04-12 13:15:05 -07:00
Yann Collet
21ff1a839a updated doc to underline difference between block and frame
as this is a very frequent source of confusion for new users.
2019-04-12 11:27:44 -07:00
Yann Collet
bcb26565a7 improved documentation for LZ4 dictionary compression 2019-04-11 15:20:46 -07:00
Yann Collet
8d76c8a44a introduce LZ4_DISTANCE_MAX build macro
make it possible to generate LZ4-compressed block
with a controlled maximum offset (necessarily <= 65535).

This could be useful for compatibility with decoders
using a very limited memory budget (<64 KB).

Answer #154
2019-04-11 14:15:33 -07:00
Yann Collet
c438548312
Merge pull request #652 from vtorri/dev
Allow installation of lz4 for Windows 10 with MSYS2
2019-03-03 13:40:45 -08:00
Vincent Torri
0360981f68 Allow installation of lz4 for Windows 10 with MSYS2 2019-03-03 22:06:38 +01:00
Tim Zakian
c1610690b1 Add cast around malloc 2019-01-11 09:49:26 -08:00
Tim Zakian
416916146f Add constant pointer annotations 2019-01-10 20:40:00 -08:00
Tim Zakian
5822e667cc Remove a bunch of global variables that tracked settings for the IO module, and move them in to a struct 2019-01-10 15:27:47 -08:00
Tim Zakian
9028682e7a Fix pass-through mode 2019-01-10 10:20:17 -08:00
Ruben O. Chiavone
e6905b5812 Fix lz4 extension in input filename not causing decompression 2019-01-08 22:56:04 -03:00
Vincent Torri
0314027051 Allow installation of lz4 for Windows (MSYS2 or when cross-compiling) 2018-11-20 21:08:23 +01:00
Blaise Sanouillet
32501603c7 support custom block sizes: fix usage message & remove unused functions 2018-09-28 09:53:14 -07:00
Blaise Sanouillet
718d16ef29 support custom block sizes - fix types 2018-09-28 09:02:36 -07:00
Blaise Sanouillet
64ad395d27 support custom block sizes 2018-09-28 08:02:49 -07:00
Yann Collet
b2215f2a89 tried to clean another bunch of cppcheck warnings
so "funny" thing with cppcheck
is that no 2 versions give the same list of warnings.

On Mac, I'm using v1.81, which had all warnings fixed.
On Travis CI, it's v1.61, and it complains about a dozen more/different things.
On Linux, it's v1.72, and it finds a completely different list of a half dozen warnings.

Some of these seems to be bugs/limitations in cppcheck itself.
The TravisCI version v1.61 seems unable to understand %zu correctly, and seems to assume it means %u.
2018-09-19 12:12:49 -07:00
Yann Collet
e34a27d291 fixed minor cppcheck warnings in programs 2018-09-18 16:08:17 -07:00
Lucas Holt
3318d573ba Add support for MidnightBSD 2018-09-08 14:46:54 -04:00
Yann Collet
0f08c22c31
Merge pull request #563 from lz4/docDict
updated documentation for dictionary compression
2018-09-06 12:43:29 -07:00
Yann Collet
cd3945f532
Merge pull request #564 from lz4/cTime
fixed compression time
2018-09-05 17:48:40 -07:00
Yann Collet
d136118ea9
Merge pull request #556 from kou/add-missing-exeext
Add missing $(EXT)
2018-09-05 15:55:34 -07:00
Yann Collet
b61991491a fixed compression time
displayed at the end (#555)
2018-09-05 15:16:03 -07:00
Yann Collet
0fea528e3a updated documentation regarding dictionary compression
following suggestion from @stbrumme (#558)

Also : bumped version number, regenerated man page and html doc
2018-09-05 14:05:08 -07:00
Yann Collet
2e4847c2d5 fixed #560
it was a fairly complex scenario,
involving source files > 64K
and some extraordinary conditions related to specific layout of ranges of zeroes.
and only on level 9.
2018-09-04 18:21:40 -07:00
Kouhei Sutou
9c457ccb7a Add missing $(EXT) 2018-08-13 14:17:54 +09:00
Jennifer Liu
e778db373b Fixed bugs about incorrect acceleration calculation and benchmarking negative compresion level 2018-06-27 13:36:38 -07:00
Jennifer Liu
8745638d7c Fixed invalid argument test and reformatted else 2018-06-26 17:53:12 -07:00
Jennifer Liu
9ee5183d9b Fixed code based on comments from pull request 2018-06-26 17:31:15 -07:00
Jennifer Liu
9f8fd09852 delete commented out code in bench.c 2018-06-26 10:49:39 -07:00
Jennifer Liu
536b79afd9 Added --fast command to cli 2018-06-25 17:46:39 -07:00
fbrosson
2149b1a8f6 Add Haiku as a validated target.
lz4 1.8.2 works fine on Haiku and passes all tests.
2018-05-17 16:52:53 +00:00
Nick Terrell
a7cc0b590a Fix make install
* Uninstall didn't remove the pkg-config correctly.
* Fix `mandir`
* Allow overriding either upper- or lower-case location variables, but
  always use the lower case variables.
* Add test case that ensures overriding both upper- and lower-case
  variables is the same, and that the directory is empty after uninstall.
2018-05-04 13:50:23 -07:00
Cyan4973
5a2501a90d added a test case for LZ4_decompress_fast_usingDict
with a separated dictionary
since a joined dictionary is now detected as prefix64K.

Also : fixed a minor warning under msys
2018-04-29 07:42:24 -07:00
Yann Collet
5c7d3812d9 fasterDecSpeed can be triggered from cli with --favor-decSpeed 2018-04-26 15:49:32 -07:00
W. Felix Handte
2dfc7cbe82 Change Over Includes in the Project 2018-04-24 16:22:28 -04:00
Cyan4973
644b7bd2b6 fixed minor declaration issue with clang on msys 2018-04-23 15:52:44 -07:00
Yann Collet
133a50b780 fixed DISPLAYUPDATE()
wrong comparison, which was always overflowing (hence was always true)
except when it was not (i386, reported by pmc)
in which case it would never show any information.
2018-04-06 14:16:23 -07:00
Yann Collet
1faa7e2698 bench: introduced hidden command -S
to benchmark multiple files with separate results
2018-03-19 17:19:25 -07:00
Yann Collet
5b67c7d185 Merge branch 'dev' of github.com:Cyan4973/lz4 into dev 2018-03-19 16:18:25 -07:00
Yann Collet
d6711a7cff minor man fix on clevels 2018-03-19 16:18:10 -07:00
W. Felix Handte
9dcd9abc14 Make LZ4F_compressFrame_usingCDict Take a Compression Context 2018-03-12 14:58:43 -04:00
Yann Collet
179670f32f use TIMELOOP_NANOSEC,
as suggested by @terrelln
2018-02-20 15:26:59 -08:00
Yann Collet
34c1634d4b fixed minor conversion warning 2018-02-20 14:13:13 -08:00
Yann Collet
ae3dededed ensure bench speed measurement is more accurate for small inputs
Previous method would produce too many time() invocations,
becoming a significant fraction of workload measured.

The new strategy is to use time() only once per batch,
and dynamically resize batch size so that each round lasts approximately 1 second.

This only matters for small inputs.
Measurement for large files (such as silesia.tar) are much less impacted
(though decoding speed is so fast that even medium-size files will notice an improvement).
2018-02-20 13:09:13 -08:00
Yann Collet
1a233c5f0f update bench.c to use less time invocations
translating into more accurate speed measurements for small sources
2018-02-20 11:37:19 -08:00
hobomind
b202c67234
fix: missed semicolon at programs/lz4io.c:954 2018-02-14 18:47:56 +03:00
Yann Collet
7dba09af47 use more restrictive conditions for clock_gettime() 2018-01-16 10:21:37 -08:00
Yann Collet
75e22d133e minor : try to tell static analyzer that we don't care if fseek() fails
as already explained in comments.
2018-01-13 22:52:17 -08:00
Yann Collet
8e69328d61 programs/datagen : attempt to please static analyzer
with an assert()
to reduce false positive
2018-01-13 22:24:32 -08:00
Yann Collet
fe66e78b96 lz4io: fixed minor ressource leak 2018-01-13 22:18:04 -08:00
Yann Collet
52a76ff682 update man page 2018-01-13 19:06:33 -08:00
Yann Collet
ac38ffa9af updated manuals to v1.8.1 2018-01-13 18:46:13 -08:00
Nick Terrell
00eac87dda [lz4io] Refuse to set file stat for non-regular files 2018-01-05 11:54:35 -08:00
Nick Terrell
9474b706a5 [lz4io] Fix decompression file stat with --rm 2018-01-05 11:27:24 -08:00
Yann Collet
506ae9c333
Merge pull request #439 from eli-b/doc-nohup-behavior
Docs: describe behavior under nohup
2018-01-02 00:19:59 +01:00