redonkable
/
remarkable-linux
1
0
Fork 0
Code Releases Activity

lib: update LZ4 compressor module 

Patch series "Update LZ4 compressor module", v7.

This patchset updates the LZ4 compression module to a version based on
LZ4 v1.7.3 allowing to use the fast compression algorithm aka LZ4 fast
which provides an "acceleration" parameter as a tradeoff between high
compression ratio and high compression speed.

We want to use LZ4 fast in order to support compression in lustre and
(mostly, based on that) investigate data reduction techniques in behalf
of storage systems.

Also, it will be useful for other users of LZ4 compression, as with LZ4
fast it is possible to enable applications to use fast and/or high
compression depending on the usecase.  For instance, ZRAM is offering a
LZ4 backend and could benefit from an updated LZ4 in the kernel.

LZ4 homepage: http://www.lz4.org/
LZ4 source repository: https://github.com/lz4/lz4 Source version: 1.7.3

Benchmark (taken from [1], Core i5-4300U @1.9GHz):
----------------|--------------|----------------|----------
Compressor      | Compression  | Decompression  | Ratio
----------------|--------------|----------------|----------
memcpy          |  4200 MB/s   |  4200 MB/s     | 1.000
LZ4 fast 50     |  1080 MB/s   |  2650 MB/s     | 1.375
LZ4 fast 17     |   680 MB/s   |  2220 MB/s     | 1.607
LZ4 fast 5      |   475 MB/s   |  1920 MB/s     | 1.886
LZ4 default     |   385 MB/s   |  1850 MB/s     | 2.101

[1] http://fastcompression.blogspot.de/2015/04/sampling-or-faster-lz4.html

[PATCH 1/5] lib: Update LZ4 compressor module
[PATCH 2/5] lib/decompress_unlz4: Change module to work with new LZ4 module version
[PATCH 3/5] crypto: Change LZ4 modules to work with new LZ4 module version
[PATCH 4/5] fs/pstore: fs/squashfs: Change usage of LZ4 to work with new LZ4 version
[PATCH 5/5] lib/lz4: Remove back-compat wrappers

This patch (of 5):

Update the LZ4 kernel module to LZ4 v1.7.3 by Yann Collet.  The kernel
module is inspired by the previous work by Chanho Min.  The updated LZ4
module will not break existing code since the patchset contains
appropriate changes.

API changes:

New method LZ4_compress_fast which differs from the variant available in
kernel by the new acceleration parameter, allowing to trade compression
ratio for more compression speed and vice versa.

LZ4_decompress_fast is the respective decompression method, featuring a
very fast decoder (multiple GB/s per core), able to reach RAM speed in
multi-core systems.  The decompressor allows to decompress data
compressed with LZ4 fast as well as the LZ4 HC (high compression)
algorithm.

Also the useful functions LZ4_decompress_safe_partial and
LZ4_compress_destsize were added.  The latter reverses the logic by
trying to compress as much data as possible from source to dest while
the former aims to decompress partial blocks of data.

A bunch of streaming functions were also added which allow
compressig/decompressing data in multiple steps (so called "streaming
mode").

The methods lz4_compress and lz4_decompress_unknownoutputsize are now
known as LZ4_compress_default respectivley LZ4_decompress_safe.  The old
methods will be removed since there's no callers left in the code.

[arnd@arndb.de: fix KERNEL_LZ4 support]
  Link: http://lkml.kernel.org/r/20170208211946.2839649-1-arnd@arndb.de
[akpm@linux-foundation.org: simplify]
[akpm@linux-foundation.org: fix the simplification]
[4sschmid@informatik.uni-hamburg.de: fix performance regressions]
  Link: http://lkml.kernel.org/r/1486898178-17125-2-git-send-email-4sschmid@informatik.uni-hamburg.de
[4sschmid@informatik.uni-hamburg.de: v8]
  Link: http://lkml.kernel.org/r/1487182598-15351-2-git-send-email-4sschmid@informatik.uni-hamburg.de
Link: http://lkml.kernel.org/r/1486321748-19085-2-git-send-email-4sschmid@informatik.uni-hamburg.de
Signed-off-by: Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Cc: Bongkyu Kim <bongkyu.kim@lge.com>
Cc: Rui Salvaterra <rsalvaterra@gmail.com>
Cc: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: David S. Miller <davem@davemloft.net>
Cc: Anton Vorontsov <anton@enomsg.org>
Cc: Colin Cross <ccross@android.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Tony Luck <tony.luck@intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
zero-colors
Sven Schmidt 2017-02-24 15:01:12 -08:00 committed by Linus Torvalds
parent 8893f51933
commit 4e1a33b105
6 changed files with 2788 additions and 1107 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

762
include/linux/lz4.h
View File

@ -1,87 +1,717 @@
#ifndef __LZ4_H__
#define __LZ4_H__
/*
* LZ4 Kernel Interface
/* LZ4 Kernel Interface
*
* Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
* Copyright (C) 2016, Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*
* This file is based on the original header file
* for LZ4 - Fast LZ compression algorithm.
*
* LZ4 - Fast LZ compression algorithm
* Copyright (C) 2011-2016, Yann Collet.
* BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the
* distribution.
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
* You can contact the author at :
* - LZ4 homepage : http://www.lz4.org
* - LZ4 source repository : https://github.com/lz4/lz4
*/
#define LZ4_MEM_COMPRESS (16384)
#define LZ4HC_MEM_COMPRESS (262144 + (2 * sizeof(unsigned char *)))
#ifndef __LZ4_H__
#define __LZ4_H__
#include <linux/types.h>
#include <linux/string.h> /* memset, memcpy */
/*-************************************************************************
* CONSTANTS
**************************************************************************/
/*
* LZ4_MEMORY_USAGE :
* Memory usage formula : N->2^N Bytes
* (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.)
* Increasing memory usage improves compression ratio
* Reduced memory usage can improve speed, due to cache effect
* Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache
*/
#define LZ4_MEMORY_USAGE 14
#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */
#define LZ4_COMPRESSBOUND(isize) (\
(unsigned int)(isize) > (unsigned int)LZ4_MAX_INPUT_SIZE \
? 0 \
: (isize) + ((isize)/255) + 16)
#define LZ4_ACCELERATION_DEFAULT 1
#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2)
#define LZ4_HASHTABLESIZE (1 << LZ4_MEMORY_USAGE)
#define LZ4_HASH_SIZE_U32 (1 << LZ4_HASHLOG)
#define LZ4HC_MIN_CLEVEL 3
#define LZ4HC_DEFAULT_CLEVEL 9
#define LZ4HC_MAX_CLEVEL 16
#define LZ4HC_DICTIONARY_LOGSIZE 16
#define LZ4HC_MAXD (1<<LZ4HC_DICTIONARY_LOGSIZE)
#define LZ4HC_MAXD_MASK (LZ4HC_MAXD - 1)
#define LZ4HC_HASH_LOG (LZ4HC_DICTIONARY_LOGSIZE - 1)
#define LZ4HC_HASHTABLESIZE (1 << LZ4HC_HASH_LOG)
#define LZ4HC_HASH_MASK (LZ4HC_HASHTABLESIZE - 1)
/*-************************************************************************
* STREAMING CONSTANTS AND STRUCTURES
**************************************************************************/
#define LZ4_STREAMSIZE_U64 ((1 << (LZ4_MEMORY_USAGE - 3)) + 4)
#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U64 * sizeof(unsigned long long))
#define LZ4_STREAMHCSIZE 262192
#define LZ4_STREAMHCSIZE_SIZET (262192 / sizeof(size_t))
#define LZ4_STREAMDECODESIZE_U64 4
#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U64 * \
sizeof(unsigned long long))
/*
* lz4_compressbound()
* Provides the maximum size that LZ4 may output in a "worst case" scenario
* (input data not compressible)
* LZ4_stream_t - information structure to track an LZ4 stream.
*/
static inline size_t lz4_compressbound(size_t isize)
typedef struct {
uint32_t hashTable[LZ4_HASH_SIZE_U32];
uint32_t currentOffset;
uint32_t initCheck;
const uint8_t *dictionary;
uint8_t *bufferStart;
uint32_t dictSize;
} LZ4_stream_t_internal;
typedef union {
unsigned long long table[LZ4_STREAMSIZE_U64];
LZ4_stream_t_internal internal_donotuse;
} LZ4_stream_t;
/*
* LZ4_streamHC_t - information structure to track an LZ4HC stream.
*/
typedef struct {
unsigned int hashTable[LZ4HC_HASHTABLESIZE];
unsigned short chainTable[LZ4HC_MAXD];
/* next block to continue on current prefix */
const unsigned char *end;
/* All index relative to this position */
const unsigned char *base;
/* alternate base for extDict */
const unsigned char *dictBase;
/* below that point, need extDict */
unsigned int dictLimit;
/* below that point, no more dict */
unsigned int lowLimit;
/* index from which to continue dict update */
unsigned int nextToUpdate;
unsigned int compressionLevel;
} LZ4HC_CCtx_internal;
typedef union {
size_t table[LZ4_STREAMHCSIZE_SIZET];
LZ4HC_CCtx_internal internal_donotuse;
} LZ4_streamHC_t;
/*
* LZ4_streamDecode_t - information structure to track an
* LZ4 stream during decompression.
*
* init this structure using LZ4_setStreamDecode (or memset()) before first use
*/
typedef struct {
const uint8_t *externalDict;
size_t extDictSize;
const uint8_t *prefixEnd;
size_t prefixSize;
} LZ4_streamDecode_t_internal;
typedef union {
unsigned long long table[LZ4_STREAMDECODESIZE_U64];
LZ4_streamDecode_t_internal internal_donotuse;
} LZ4_streamDecode_t;
/*-************************************************************************
* SIZE OF STATE
**************************************************************************/
#define LZ4_MEM_COMPRESS LZ4_STREAMSIZE
#define LZ4HC_MEM_COMPRESS LZ4_STREAMHCSIZE
/*-************************************************************************
* Compression Functions
**************************************************************************/
/**
* LZ4_compressBound() - Max. output size in worst case szenarios
* @isize: Size of the input data
*
* Return: Max. size LZ4 may output in a "worst case" szenario
* (data not compressible)
*/
static inline int LZ4_compressBound(size_t isize)
{
return isize + (isize / 255) + 16;
return LZ4_COMPRESSBOUND(isize);
}
/*
* lz4_compress()
* src : source address of the original data
* src_len : size of the original data
* dst : output buffer address of the compressed data
* This requires 'dst' of size LZ4_COMPRESSBOUND.
* dst_len : is the output size, which is returned after compress done
* workmem : address of the working memory.
* This requires 'workmem' of size LZ4_MEM_COMPRESS.
* return : Success if return 0
* Error if return (< 0)
* note : Destination buffer and workmem must be already allocated with
* the defined size.
/**
* lz4_compressbound() - For backwards compatibility; see LZ4_compressBound
* @isize: Size of the input data
*
* Return: Max. size LZ4 may output in a "worst case" szenario
* (data not compressible)
*/
int lz4_compress(const unsigned char *src, size_t src_len,
unsigned char *dst, size_t *dst_len, void *wrkmem);
static inline int lz4_compressbound(size_t isize)
{
return LZ4_COMPRESSBOUND(isize);
}
/*
* lz4hc_compress()
* src : source address of the original data
* src_len : size of the original data
* dst : output buffer address of the compressed data
* This requires 'dst' of size LZ4_COMPRESSBOUND.
* dst_len : is the output size, which is returned after compress done
* workmem : address of the working memory.
* This requires 'workmem' of size LZ4HC_MEM_COMPRESS.
* return : Success if return 0
* Error if return (< 0)
* note : Destination buffer and workmem must be already allocated with
* the defined size.
*/
int lz4hc_compress(const unsigned char *src, size_t src_len,
unsigned char *dst, size_t *dst_len, void *wrkmem);
/*
* lz4_decompress()
* src : source address of the compressed data
* src_len : is the input size, whcih is returned after decompress done
* dest : output buffer address of the decompressed data
* actual_dest_len: is the size of uncompressed data, supposing it's known
* return : Success if return 0
* Error if return (< 0)
* note : Destination buffer must be already allocated.
* slightly faster than lz4_decompress_unknownoutputsize()
/**
* LZ4_compress_default() - Compress data from source to dest
* @source: source address of the original data
* @dest: output buffer address of the compressed data
* @inputSize: size of the input data. Max supported value is LZ4_MAX_INPUT_SIZE
* @maxOutputSize: full or partial size of buffer 'dest'
* which must be already allocated
* @wrkmem: address of the working memory.
* This requires 'workmem' of LZ4_MEM_COMPRESS.
*
* Compresses 'sourceSize' bytes from buffer 'source'
* into already allocated 'dest' buffer of size 'maxOutputSize'.
* Compression is guaranteed to succeed if
* 'maxOutputSize' >= LZ4_compressBound(inputSize).
* It also runs faster, so it's a recommended setting.
* If the function cannot compress 'source' into a more limited 'dest' budget,
* compression stops *immediately*, and the function result is zero.
* As a consequence, 'dest' content is not valid.
*
* Return: Number of bytes written into buffer 'dest'
* (necessarily <= maxOutputSize) or 0 if compression fails
*/
int lz4_decompress(const unsigned char *src, size_t *src_len,
unsigned char *dest, size_t actual_dest_len);
int LZ4_compress_default(const char *source, char *dest, int inputSize,
int maxOutputSize, void *wrkmem);
/**
* LZ4_compress_fast() - As LZ4_compress_default providing an acceleration param
* @source: source address of the original data
* @dest: output buffer address of the compressed data
* @inputSize: size of the input data. Max supported value is LZ4_MAX_INPUT_SIZE
* @maxOutputSize: full or partial size of buffer 'dest'
* which must be already allocated
* @acceleration: acceleration factor
* @wrkmem: address of the working memory.
* This requires 'workmem' of LZ4_MEM_COMPRESS.
*
* Same as LZ4_compress_default(), but allows to select an "acceleration"
* factor. The larger the acceleration value, the faster the algorithm,
* but also the lesser the compression. It's a trade-off. It can be fine tuned,
* with each successive value providing roughly +~3% to speed.
* An acceleration value of "1" is the same as regular LZ4_compress_default()
* Values <= 0 will be replaced by LZ4_ACCELERATION_DEFAULT, which is 1.
*
* Return: Number of bytes written into buffer 'dest'
* (necessarily <= maxOutputSize) or 0 if compression fails
*/
int LZ4_compress_fast(const char *source, char *dest, int inputSize,
int maxOutputSize, int acceleration, void *wrkmem);
/**
* LZ4_compress_destSize() - Compress as much data as possible
* from source to dest
* @source: source address of the original data
* @dest: output buffer address of the compressed data
* @sourceSizePtr: will be modified to indicate how many bytes where read
* from 'source' to fill 'dest'. New value is necessarily <= old value.
* @targetDestSize: Size of buffer 'dest' which must be already allocated
* @wrkmem: address of the working memory.
* This requires 'workmem' of LZ4_MEM_COMPRESS.
*
* Reverse the logic, by compressing as much data as possible
* from 'source' buffer into already allocated buffer 'dest'
* of size 'targetDestSize'.
* This function either compresses the entire 'source' content into 'dest'
* if it's large enough, or fill 'dest' buffer completely with as much data as
* possible from 'source'.
*
* Return: Number of bytes written into 'dest' (necessarily <= targetDestSize)
* or 0 if compression fails
*/
int LZ4_compress_destSize(const char *source, char *dest, int *sourceSizePtr,
int targetDestSize, void *wrkmem);
/*
* lz4_decompress_unknownoutputsize()
* src : source address of the compressed data
* src_len : is the input size, therefore the compressed size
* dest : output buffer address of the decompressed data
* dest_len: is the max size of the destination buffer, which is
* returned with actual size of decompressed data after
* decompress done
* return : Success if return 0
* Error if return (< 0)
* note : Destination buffer must be already allocated.
* lz4_compress() - For backward compatibility, see LZ4_compress_default
* @src: source address of the original data
* @src_len: size of the original data
* @dst: output buffer address of the compressed data. This requires 'dst'
* of size LZ4_COMPRESSBOUND
* @dst_len: is the output size, which is returned after compress done
* @workmem: address of the working memory.
*
* Return: Success if return 0, Error if return < 0
*/
int lz4_compress(const unsigned char *src, size_t src_len, unsigned char *dst,
size_t *dst_len, void *wrkmem);
/*-************************************************************************
* Decompression Functions
**************************************************************************/
/**
* LZ4_decompress_fast() - Decompresses data from 'source' into 'dest'
* @source: source address of the compressed data
* @dest: output buffer address of the uncompressed data
* which must be already allocated with 'originalSize' bytes
* @originalSize: is the original and therefore uncompressed size
*
* Decompresses data from 'source' into 'dest'.
* This function fully respect memory boundaries for properly formed
* compressed data.
* It is a bit faster than LZ4_decompress_safe().
* However, it does not provide any protection against intentionally
* modified data stream (malicious input).
* Use this function in trusted environment only
* (data to decode comes from a trusted source).
*
* Return: number of bytes read from the source buffer
* or a negative result if decompression fails.
*/
int LZ4_decompress_fast(const char *source, char *dest, int originalSize);
/**
* LZ4_decompress_safe() - Decompression protected against buffer overflow
* @source: source address of the compressed data
* @dest: output buffer address of the uncompressed data
* which must be already allocated
* @compressedSize: is the precise full size of the compressed block
* @maxDecompressedSize: is the size of 'dest' buffer
*
* Decompresses data fom 'source' into 'dest'.
* If the source stream is detected malformed, the function will
* stop decoding and return a negative result.
* This function is protected against buffer overflow exploits,
* including malicious data packets. It never writes outside output buffer,
* nor reads outside input buffer.
*
* Return: number of bytes decompressed into destination buffer
* (necessarily <= maxDecompressedSize)
* or a negative result in case of error
*/
int LZ4_decompress_safe(const char *source, char *dest, int compressedSize,
int maxDecompressedSize);
/**
* LZ4_decompress_safe_partial() - Decompress a block of size 'compressedSize'
* at position 'source' into buffer 'dest'
* @source: source address of the compressed data
* @dest: output buffer address of the decompressed data which must be
* already allocated
* @compressedSize: is the precise full size of the compressed block.
* @targetOutputSize: the decompression operation will try
* to stop as soon as 'targetOutputSize' has been reached
* @maxDecompressedSize: is the size of destination buffer
*
* This function decompresses a compressed block of size 'compressedSize'
* at position 'source' into destination buffer 'dest'
* of size 'maxDecompressedSize'.
* The function tries to stop decompressing operation as soon as
* 'targetOutputSize' has been reached, reducing decompression time.
* This function never writes outside of output buffer,
* and never reads outside of input buffer.
* It is therefore protected against malicious data packets.
*
* Return: the number of bytes decoded in the destination buffer
* (necessarily <= maxDecompressedSize)
* or a negative result in case of error
*
*/
int LZ4_decompress_safe_partial(const char *source, char *dest,
int compressedSize, int targetOutputSize, int maxDecompressedSize);
/*
* lz4_decompress_unknownoutputsize() - For backwards compatibility,
* see LZ4_decompress_safe
* @src: source address of the compressed data
* @src_len: is the input size, therefore the compressed size
* @dest: output buffer address of the decompressed data
* which must be already allocated
* @dest_len: is the max size of the destination buffer, which is
* returned with actual size of decompressed data after decompress done
*
* Return: Success if return 0, Error if return (< 0)
*/
int lz4_decompress_unknownoutputsize(const unsigned char *src, size_t src_len,
unsigned char *dest, size_t *dest_len);
unsigned char *dest, size_t *dest_len);
/**
* lz4_decompress() - For backwards cocmpatibility, see LZ4_decompress_fast
* @src: source address of the compressed data
* @src_len: is the input size, which is returned after decompress done
* @dest: output buffer address of the decompressed data,
* which must be already allocated
* @actual_dest_len: is the size of uncompressed data, supposing it's known
*
* Return: Success if return 0, Error if return (< 0)
*/
int lz4_decompress(const unsigned char *src, size_t *src_len,
unsigned char *dest, size_t actual_dest_len);
/*-************************************************************************
* LZ4 HC Compression
**************************************************************************/
/**
* LZ4_compress_HC() - Compress data from `src` into `dst`, using HC algorithm
* @src: source address of the original data
* @dst: output buffer address of the compressed data
* @srcSize: size of the input data. Max supported value is LZ4_MAX_INPUT_SIZE
* @dstCapacity: full or partial size of buffer 'dst',
* which must be already allocated
* @compressionLevel: Recommended values are between 4 and 9, although any
* value between 1 and LZ4HC_MAX_CLEVEL will work.
* Values >LZ4HC_MAX_CLEVEL behave the same as 16.
* @wrkmem: address of the working memory.
* This requires 'wrkmem' of size LZ4HC_MEM_COMPRESS.
*
* Compress data from 'src' into 'dst', using the more powerful
* but slower "HC" algorithm. Compression is guaranteed to succeed if
* `dstCapacity >= LZ4_compressBound(srcSize)
*
* Return : the number of bytes written into 'dst' or 0 if compression fails.
*/
int LZ4_compress_HC(const char *src, char *dst, int srcSize, int dstCapacity,
int compressionLevel, void *wrkmem);
/**
* lz4hc_compress() - For backwards compatibility, see LZ4_compress_HC
* @src: source address of the original data
* @src_len: size of the original data
* @dst: output buffer address of the compressed data. This requires 'dst'
* of size LZ4_COMPRESSBOUND.
* @dst_len: is the output size, which is returned after compress done
* @wrkmem: address of the working memory.
* This requires 'workmem' of size LZ4HC_MEM_COMPRESS.
*
* Return : Success if return 0, Error if return (< 0)
*/
int lz4hc_compress(const unsigned char *src, size_t src_len, unsigned char *dst,
size_t *dst_len, void *wrkmem);
/**
* LZ4_resetStreamHC() - Init an allocated 'LZ4_streamHC_t' structure
* @streamHCPtr: pointer to the 'LZ4_streamHC_t' structure
* @compressionLevel: Recommended values are between 4 and 9, although any
* value between 1 and LZ4HC_MAX_CLEVEL will work.
* Values >LZ4HC_MAX_CLEVEL behave the same as 16.
*
* An LZ4_streamHC_t structure can be allocated once
* and re-used multiple times.
* Use this function to init an allocated `LZ4_streamHC_t` structure
* and start a new compression.
*/
void LZ4_resetStreamHC(LZ4_streamHC_t *streamHCPtr, int compressionLevel);
/**
* LZ4_loadDictHC() - Load a static dictionary into LZ4_streamHC
* @streamHCPtr: pointer to the LZ4HC_stream_t
* @dictionary: dictionary to load
* @dictSize: size of dictionary
*
* Use this function to load a static dictionary into LZ4HC_stream.
* Any previous data will be forgotten, only 'dictionary'
* will remain in memory.
* Loading a size of 0 is allowed.
*
* Return : dictionary size, in bytes (necessarily <= 64 KB)
*/
int LZ4_loadDictHC(LZ4_streamHC_t *streamHCPtr, const char *dictionary,
int dictSize);
/**
* LZ4_compress_HC_continue() - Compress 'src' using data from previously
* compressed blocks as a dictionary using the HC algorithm
* @streamHCPtr: Pointer to the previous 'LZ4_streamHC_t' structure
* @src: source address of the original data
* @dst: output buffer address of the compressed data,
* which must be already allocated
* @srcSize: size of the input data. Max supported value is LZ4_MAX_INPUT_SIZE
* @maxDstSize: full or partial size of buffer 'dest'
* which must be already allocated
*
* These functions compress data in successive blocks of any size, using
* previous blocks as dictionary. One key assumption is that previous
* blocks (up to 64 KB) remain read-accessible while
* compressing next blocks. There is an exception for ring buffers,
* which can be smaller than 64 KB.
* Ring buffers scenario is automatically detected and handled by
* LZ4_compress_HC_continue().
* Before starting compression, state must be properly initialized,
* using LZ4_resetStreamHC().
* A first "fictional block" can then be designated as
* initial dictionary, using LZ4_loadDictHC() (Optional).
* Then, use LZ4_compress_HC_continue()
* to compress each successive block. Previous memory blocks
* (including initial dictionary when present) must remain accessible
* and unmodified during compression.
* 'dst' buffer should be sized to handle worst case scenarios, using
* LZ4_compressBound(), to ensure operation success.
* If, for any reason, previous data blocks can't be preserved unmodified
* in memory during next compression block,
* you must save it to a safer memory space, using LZ4_saveDictHC().
* Return value of LZ4_saveDictHC() is the size of dictionary
* effectively saved into 'safeBuffer'.
*
* Return: Number of bytes written into buffer 'dst' or 0 if compression fails
*/
int LZ4_compress_HC_continue(LZ4_streamHC_t *streamHCPtr, const char *src,
char *dst, int srcSize, int maxDstSize);
/**
* LZ4_saveDictHC() - Save static dictionary from LZ4HC_stream
* @streamHCPtr: pointer to the 'LZ4HC_stream_t' structure
* @safeBuffer: buffer to save dictionary to, must be already allocated
* @maxDictSize: size of 'safeBuffer'
*
* If previously compressed data block is not guaranteed
* to remain available at its memory location,
* save it into a safer place (char *safeBuffer).
* Note : you don't need to call LZ4_loadDictHC() afterwards,
* dictionary is immediately usable, you can therefore call
* LZ4_compress_HC_continue().
*
* Return : saved dictionary size in bytes (necessarily <= maxDictSize),
* or 0 if error.
*/
int LZ4_saveDictHC(LZ4_streamHC_t *streamHCPtr, char *safeBuffer,
int maxDictSize);
/*-*********************************************
* Streaming Compression Functions
***********************************************/
/**
* LZ4_resetStream() - Init an allocated 'LZ4_stream_t' structure
* @LZ4_stream: pointer to the 'LZ4_stream_t' structure
*
* An LZ4_stream_t structure can be allocated once
* and re-used multiple times.
* Use this function to init an allocated `LZ4_stream_t` structure
* and start a new compression.
*/
void LZ4_resetStream(LZ4_stream_t *LZ4_stream);
/**
* LZ4_loadDict() - Load a static dictionary into LZ4_stream
* @streamPtr: pointer to the LZ4_stream_t
* @dictionary: dictionary to load
* @dictSize: size of dictionary
*
* Use this function to load a static dictionary into LZ4_stream.
* Any previous data will be forgotten, only 'dictionary'
* will remain in memory.
* Loading a size of 0 is allowed.
*
* Return : dictionary size, in bytes (necessarily <= 64 KB)
*/
int LZ4_loadDict(LZ4_stream_t *streamPtr, const char *dictionary,
int dictSize);
/**
* LZ4_saveDict() - Save static dictionary from LZ4_stream
* @streamPtr: pointer to the 'LZ4_stream_t' structure
* @safeBuffer: buffer to save dictionary to, must be already allocated
* @dictSize: size of 'safeBuffer'
*
* If previously compressed data block is not guaranteed
* to remain available at its memory location,
* save it into a safer place (char *safeBuffer).
* Note : you don't need to call LZ4_loadDict() afterwards,
* dictionary is immediately usable, you can therefore call
* LZ4_compress_fast_continue().
*
* Return : saved dictionary size in bytes (necessarily <= dictSize),
* or 0 if error.
*/
int LZ4_saveDict(LZ4_stream_t *streamPtr, char *safeBuffer, int dictSize);
/**
* LZ4_compress_fast_continue() - Compress 'src' using data from previously
* compressed blocks as a dictionary
* @streamPtr: Pointer to the previous 'LZ4_stream_t' structure
* @src: source address of the original data
* @dst: output buffer address of the compressed data,
* which must be already allocated
* @srcSize: size of the input data. Max supported value is LZ4_MAX_INPUT_SIZE
* @maxDstSize: full or partial size of buffer 'dest'
* which must be already allocated
* @acceleration: acceleration factor
*
* Compress buffer content 'src', using data from previously compressed blocks
* as dictionary to improve compression ratio.
* Important : Previous data blocks are assumed to still
* be present and unmodified !
* If maxDstSize >= LZ4_compressBound(srcSize),
* compression is guaranteed to succeed, and runs faster.
*
* Return: Number of bytes written into buffer 'dst' or 0 if compression fails
*/
int LZ4_compress_fast_continue(LZ4_stream_t *streamPtr, const char *src,
char *dst, int srcSize, int maxDstSize, int acceleration);
/**
* LZ4_setStreamDecode() - Instruct where to find dictionary
* @LZ4_streamDecode: the 'LZ4_streamDecode_t' structure
* @dictionary: dictionary to use
* @dictSize: size of dictionary
*
* Use this function to instruct where to find the dictionary.
* Setting a size of 0 is allowed (same effect as reset).
*
* Return: 1 if OK, 0 if error
*/
int LZ4_setStreamDecode(LZ4_streamDecode_t *LZ4_streamDecode,
const char *dictionary, int dictSize);
/**
* LZ4_decompress_fast_continue() - Decompress blocks in streaming mode
* @LZ4_streamDecode: the 'LZ4_streamDecode_t' structure
* @source: source address of the compressed data
* @dest: output buffer address of the uncompressed data
* which must be already allocated
* @compressedSize: is the precise full size of the compressed block
* @maxDecompressedSize: is the size of 'dest' buffer
*
* These decoding function allows decompression of multiple blocks
* in "streaming" mode.
* Previously decoded blocks *must* remain available at the memory position
* where they were decoded (up to 64 KB)
* In the case of a ring buffers, decoding buffer must be either :
* - Exactly same size as encoding buffer, with same update rule
* (block boundaries at same positions) In which case,
* the decoding & encoding ring buffer can have any size,
* including very small ones ( < 64 KB).
* - Larger than encoding buffer, by a minimum of maxBlockSize more bytes.
* maxBlockSize is implementation dependent.
* It's the maximum size you intend to compress into a single block.
* In which case, encoding and decoding buffers do not need
* to be synchronized, and encoding ring buffer can have any size,
* including small ones ( < 64 KB).
* - _At least_ 64 KB + 8 bytes + maxBlockSize.
* In which case, encoding and decoding buffers do not need to be
* synchronized, and encoding ring buffer can have any size,
* including larger than decoding buffer. W
* Whenever these conditions are not possible, save the last 64KB of decoded
* data into a safe buffer, and indicate where it is saved
* using LZ4_setStreamDecode()
*
* Return: number of bytes decompressed into destination buffer
* (necessarily <= maxDecompressedSize)
* or a negative result in case of error
*/
int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode,
const char *source, char *dest, int compressedSize,
int maxDecompressedSize);
/**
* LZ4_decompress_fast_continue() - Decompress blocks in streaming mode
* @LZ4_streamDecode: the 'LZ4_streamDecode_t' structure
* @source: source address of the compressed data
* @dest: output buffer address of the uncompressed data
* which must be already allocated with 'originalSize' bytes
* @originalSize: is the original and therefore uncompressed size
*
* These decoding function allows decompression of multiple blocks
* in "streaming" mode.
* Previously decoded blocks *must* remain available at the memory position
* where they were decoded (up to 64 KB)
* In the case of a ring buffers, decoding buffer must be either :
* - Exactly same size as encoding buffer, with same update rule
* (block boundaries at same positions) In which case,
* the decoding & encoding ring buffer can have any size,
* including very small ones ( < 64 KB).
* - Larger than encoding buffer, by a minimum of maxBlockSize more bytes.
* maxBlockSize is implementation dependent.
* It's the maximum size you intend to compress into a single block.
* In which case, encoding and decoding buffers do not need
* to be synchronized, and encoding ring buffer can have any size,
* including small ones ( < 64 KB).
* - _At least_ 64 KB + 8 bytes + maxBlockSize.
* In which case, encoding and decoding buffers do not need to be
* synchronized, and encoding ring buffer can have any size,
* including larger than decoding buffer. W
* Whenever these conditions are not possible, save the last 64KB of decoded
* data into a safe buffer, and indicate where it is saved
* using LZ4_setStreamDecode()
*
* Return: number of bytes decompressed into destination buffer
* (necessarily <= maxDecompressedSize)
* or a negative result in case of error
*/
int LZ4_decompress_fast_continue(LZ4_streamDecode_t *LZ4_streamDecode,
const char *source, char *dest, int originalSize);
/**
* LZ4_decompress_safe_usingDict() - Same as LZ4_setStreamDecode()
* followed by LZ4_decompress_safe_continue()
* @source: source address of the compressed data
* @dest: output buffer address of the uncompressed data
* which must be already allocated
* @compressedSize: is the precise full size of the compressed block
* @maxDecompressedSize: is the size of 'dest' buffer
* @dictStart: pointer to the start of the dictionary in memory
* @dictSize: size of dictionary
*
* These decoding function works the same as
* a combination of LZ4_setStreamDecode() followed by
* LZ4_decompress_safe_continue()
* It is stand-alone, and don'tn eed a LZ4_streamDecode_t structure.
*
* Return: number of bytes decompressed into destination buffer
* (necessarily <= maxDecompressedSize)
* or a negative result in case of error
*/
int LZ4_decompress_safe_usingDict(const char *source, char *dest,
int compressedSize, int maxDecompressedSize, const char *dictStart,
int dictSize);
/**
* LZ4_decompress_fast_usingDict() - Same as LZ4_setStreamDecode()
* followed by LZ4_decompress_fast_continue()
* @source: source address of the compressed data
* @dest: output buffer address of the uncompressed data
* which must be already allocated with 'originalSize' bytes
* @originalSize: is the original and therefore uncompressed size
* @dictStart: pointer to the start of the dictionary in memory
* @dictSize: size of dictionary
*
* These decoding function works the same as
* a combination of LZ4_setStreamDecode() followed by
* LZ4_decompress_safe_continue()
* It is stand-alone, and don'tn eed a LZ4_streamDecode_t structure.
*
* Return: number of bytes decompressed into destination buffer
* (necessarily <= maxDecompressedSize)
* or a negative result in case of error
*/
int LZ4_decompress_fast_usingDict(const char *source, char *dest,
int originalSize, const char *dictStart, int dictSize);
#endif

2
lib/lz4/Makefile
View File

@ -1,3 +1,5 @@
ccflags-y += -O3
obj-$(CONFIG_LZ4_COMPRESS) += lz4_compress.o
obj-$(CONFIG_LZ4HC_COMPRESS) += lz4hc_compress.o
obj-$(CONFIG_LZ4_DECOMPRESS) += lz4_decompress.o

1229
lib/lz4/lz4_compress.c
View File

File diff suppressed because it is too large Load Diff

715
lib/lz4/lz4_decompress.c
View File

@ -1,25 +1,16 @@
/*
* LZ4 Decompressor for Linux kernel
*
* Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
*
* Based on LZ4 implementation by Yann Collet.
*
* LZ4 - Fast LZ compression algorithm
* Copyright (C) 2011-2012, Yann Collet.
* BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
*
* Copyright (C) 2011 - 2016, Yann Collet.
* BSD 2 - Clause License (http://www.opensource.org/licenses/bsd - license.php)
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the
* distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
@ -31,313 +22,529 @@
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
* You can contact the author at :
* - LZ4 homepage : http://www.lz4.org
* - LZ4 source repository : https://github.com/lz4/lz4
*
* You can contact the author at :
* - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
* - LZ4 source repository : http://code.google.com/p/lz4/
* Changed for kernel usage by:
* Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
*/
#ifndef STATIC
/*-************************************
* Dependencies
**************************************/
#include <linux/lz4.h>
#include "lz4defs.h"
#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>
#endif
#include <linux/lz4.h>
#include <asm/unaligned.h>
#include "lz4defs.h"
static const int dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0};
#if LZ4_ARCH64
static const int dec64table[] = {0, 0, 0, -1, 0, 1, 2, 3};
#endif
static int lz4_uncompress(const char *source, char *dest, int osize)
/*-*****************************
* Decompression functions
*******************************/
/* LZ4_decompress_generic() :
* This generic decompression function cover all use cases.
* It shall be instantiated several times, using different sets of directives
* Note that it is important this generic function is really inlined,
* in order to remove useless branches during compilation optimization.
*/
static FORCE_INLINE int LZ4_decompress_generic(
const char * const source,
char * const dest,
int inputSize,
/*
* If endOnInput == endOnInputSize,
* this value is the max size of Output Buffer.
*/
int outputSize,
/* endOnOutputSize, endOnInputSize */
int endOnInput,
/* full, partial */
int partialDecoding,
/* only used if partialDecoding == partial */
int targetOutputSize,
/* noDict, withPrefix64k, usingExtDict */
int dict,
/* == dest when no prefix */
const BYTE * const lowPrefix,
/* only if dict == usingExtDict */
const BYTE * const dictStart,
/* note : = 0 if noDict */
const size_t dictSize
)
{
/* Local Variables */
const BYTE *ip = (const BYTE *) source;
const BYTE *ref;
const BYTE * const iend = ip + inputSize;
BYTE *op = (BYTE *) dest;
BYTE * const oend = op + osize;
BYTE * const oend = op + outputSize;
BYTE *cpy;
unsigned token;
size_t length;
BYTE *oexit = op + targetOutputSize;
const BYTE * const lowLimit = lowPrefix - dictSize;
const BYTE * const dictEnd = (const BYTE *)dictStart + dictSize;
const unsigned int dec32table[] = { 0, 1, 2, 1, 4, 4, 4, 4 };
const int dec64table[] = { 0, 0, 0, -1, 0, 1, 2, 3 };
const int safeDecode = (endOnInput == endOnInputSize);
const int checkOffset = ((safeDecode) && (dictSize < (int)(64 * KB)));
/* Special cases */
/* targetOutputSize too high => decode everything */
if ((partialDecoding) && (oexit > oend - MFLIMIT))
oexit = oend - MFLIMIT;
/* Empty output buffer */
if ((endOnInput) && (unlikely(outputSize == 0)))
return ((inputSize == 1) && (*ip == 0)) ? 0 : -1;
if ((!endOnInput) && (unlikely(outputSize == 0)))
return (*ip == 0 ? 1 : -1);
/* Main Loop : decode sequences */
while (1) {
size_t length;
const BYTE *match;
size_t offset;
/* get literal length */
unsigned int const token = *ip++;
length = token>>ML_BITS;
/* get runlength */
token = *ip++;
length = (token >> ML_BITS);
if (length == RUN_MASK) {
size_t len;
unsigned int s;
len = *ip++;
for (; len == 255; length += 255)
len = *ip++;
if (unlikely(length > (size_t)(length + len)))
do {
s = *ip++;
length += s;
} while (likely(endOnInput
? ip < iend - RUN_MASK
: 1) & (s == 255));
if ((safeDecode)
&& unlikely(
(size_t)(op + length) < (size_t)(op))) {
/* overflow detection */
goto _output_error;
length += len;
}
if ((safeDecode)
&& unlikely(
(size_t)(ip + length) < (size_t)(ip))) {
/* overflow detection */
goto _output_error;
}
}
/* copy literals */
cpy = op + length;
if (unlikely(cpy > oend - COPYLENGTH)) {
/*
* Error: not enough place for another match
* (min 4) + 5 literals
*/
if (cpy != oend)
goto _output_error;
if (((endOnInput) && ((cpy > (partialDecoding ? oexit : oend - MFLIMIT))
|| (ip + length > iend - (2 + 1 + LASTLITERALS))))
|| ((!endOnInput) && (cpy > oend - WILDCOPYLENGTH))) {
if (partialDecoding) {
if (cpy > oend) {
/*
* Error :
* write attempt beyond end of output buffer
*/
goto _output_error;
}
if ((endOnInput)
&& (ip + length > iend)) {
/*
* Error :
* read attempt beyond
* end of input buffer
*/
goto _output_error;
}
} else {
if ((!endOnInput)
&& (cpy != oend)) {
/*
* Error :
* block decoding must
* stop exactly there
*/
goto _output_error;
}
if ((endOnInput)
&& ((ip + length != iend)
|| (cpy > oend))) {
/*
* Error :
* input must be consumed
*/
goto _output_error;
}
}
memcpy(op, ip, length);
ip += length;
break; /* EOF */
op += length;
/* Necessarily EOF, due to parsing restrictions */
break;
}
LZ4_WILDCOPY(ip, op, cpy);
ip -= (op - cpy);
LZ4_wildCopy(op, ip, cpy);
ip += length;
op = cpy;
/* get offset */
LZ4_READ_LITTLEENDIAN_16(ref, cpy, ip);
offset = LZ4_readLE16(ip);
ip += 2;
match = op - offset;
/* Error: offset create reference outside destination buffer */
if (unlikely(ref < (BYTE *const) dest))
if ((checkOffset) && (unlikely(match < lowLimit))) {
/* Error : offset outside buffers */
goto _output_error;
}
/* costs ~1%; silence an msan warning when offset == 0 */
LZ4_write32(op, (U32)offset);
/* get matchlength */
length = token & ML_MASK;
if (length == ML_MASK) {
for (; *ip == 255; length += 255)
ip++;
if (unlikely(length > (size_t)(length + *ip)))
unsigned int s;
do {
s = *ip++;
if ((endOnInput) && (ip > iend - LASTLITERALS))
goto _output_error;
length += s;
} while (s == 255);
if ((safeDecode)
&& unlikely(
(size_t)(op + length) < (size_t)op)) {
/* overflow detection */
goto _output_error;
length += *ip++;
}
}
/* copy repeated sequence */
if (unlikely((op - ref) < STEPSIZE)) {
#if LZ4_ARCH64
int dec64 = dec64table[op - ref];
#else
const int dec64 = 0;
#endif
op[0] = ref[0];
op[1] = ref[1];
op[2] = ref[2];
op[3] = ref[3];
op += 4;
ref += 4;
ref -= dec32table[op-ref];
PUT4(ref, op);
op += STEPSIZE - 4;
ref -= dec64;
} else {
LZ4_COPYSTEP(ref, op);
}
cpy = op + length - (STEPSIZE - 4);
if (cpy > (oend - COPYLENGTH)) {
length += MINMATCH;
/* Error: request to write beyond destination buffer */
if (cpy > oend)
goto _output_error;
#if LZ4_ARCH64
if ((ref + COPYLENGTH) > oend)
#else
if ((ref + COPYLENGTH) > oend ||
(op + COPYLENGTH) > oend)
#endif
goto _output_error;
LZ4_SECURECOPY(ref, op, (oend - COPYLENGTH));
while (op < cpy)
*op++ = *ref++;
op = cpy;
/*
* Check EOF (should never happen, since last 5 bytes
* are supposed to be literals)
*/
if (op == oend)
/* check external dictionary */
if ((dict == usingExtDict) && (match < lowPrefix)) {
if (unlikely(op + length > oend - LASTLITERALS)) {
/* doesn't respect parsing restriction */
goto _output_error;
}
if (length <= (size_t)(lowPrefix - match)) {
/*
* match can be copied as a single segment
* from external dictionary
*/
memmove(op, dictEnd - (lowPrefix - match),
length);
op += length;
} else {
/*
* match encompass external
* dictionary and current block
*/
size_t const copySize = (size_t)(lowPrefix - match);
size_t const restSize = length - copySize;
memcpy(op, dictEnd - copySize, copySize);
op += copySize;
if (restSize > (size_t)(op - lowPrefix)) {
/* overlap copy */
BYTE * const endOfMatch = op + restSize;
const BYTE *copyFrom = lowPrefix;
while (op < endOfMatch)
*op++ = *copyFrom++;
} else {
memcpy(op, lowPrefix, restSize);
op += restSize;
}
}
continue;
}
LZ4_SECURECOPY(ref, op, cpy);
/* copy match within block */
cpy = op + length;
if (unlikely(offset < 8)) {
const int dec64 = dec64table[offset];
op[0] = match[0];
op[1] = match[1];
op[2] = match[2];
op[3] = match[3];
match += dec32table[offset];
memcpy(op + 4, match, 4);
match -= dec64;
} else {
LZ4_copy8(op, match);
match += 8;
}
op += 8;
if (unlikely(cpy > oend - 12)) {
BYTE * const oCopyLimit = oend - (WILDCOPYLENGTH - 1);
if (cpy > oend - LASTLITERALS) {
/*
* Error : last LASTLITERALS bytes
* must be literals (uncompressed)
*/
goto _output_error;
}
if (op < oCopyLimit) {
LZ4_wildCopy(op, match, oCopyLimit);
match += oCopyLimit - op;
op = oCopyLimit;
}
while (op < cpy)
*op++ = *match++;
} else {
LZ4_copy8(op, match);
if (length > 16)
LZ4_wildCopy(op + 8, match + 8, cpy);
}
op = cpy; /* correction */
}
/* end of decoding */
return (int) (((char *)ip) - source);
/* write overflow error detected */
/* end of decoding */
if (endOnInput) {
/* Nb of output bytes decoded */
return (int) (((char *)op) - dest);
} else {
/* Nb of input bytes read */
return (int) (((const char *)ip) - source);
}
/* Overflow error detected */
_output_error:
return -1;
}
static int lz4_uncompress_unknownoutputsize(const char *source, char *dest,
int isize, size_t maxoutputsize)
int LZ4_decompress_safe(const char *source, char *dest,
int compressedSize, int maxDecompressedSize)
{
const BYTE *ip = (const BYTE *) source;
const BYTE *const iend = ip + isize;
const BYTE *ref;
return LZ4_decompress_generic(source, dest, compressedSize,
maxDecompressedSize, endOnInputSize, full, 0,
noDict, (BYTE *)dest, NULL, 0);
}
int LZ4_decompress_safe_partial(const char *source, char *dest,
int compressedSize, int targetOutputSize, int maxDecompressedSize)
{
return LZ4_decompress_generic(source, dest, compressedSize,
maxDecompressedSize, endOnInputSize, partial,
targetOutputSize, noDict, (BYTE *)dest, NULL, 0);
}
BYTE *op = (BYTE *) dest;
BYTE * const oend = op + maxoutputsize;
BYTE *cpy;
int LZ4_decompress_fast(const char *source, char *dest, int originalSize)
{
return LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0, withPrefix64k,
(BYTE *)(dest - 64 * KB), NULL, 64 * KB);
}
/* Main Loop */
while (ip < iend) {
int LZ4_setStreamDecode(LZ4_streamDecode_t *LZ4_streamDecode,
const char *dictionary, int dictSize)
{
LZ4_streamDecode_t_internal *lz4sd = (LZ4_streamDecode_t_internal *) LZ4_streamDecode;
unsigned token;
size_t length;
lz4sd->prefixSize = (size_t) dictSize;
lz4sd->prefixEnd = (const BYTE *) dictionary + dictSize;
lz4sd->externalDict = NULL;
lz4sd->extDictSize = 0;
return 1;
}
/* get runlength */
token = *ip++;
length = (token >> ML_BITS);
if (length == RUN_MASK) {
int s = 255;
while ((ip < iend) && (s == 255)) {
s = *ip++;
if (unlikely(length > (size_t)(length + s)))
goto _output_error;
length += s;
}
}
/* copy literals */
cpy = op + length;
if ((cpy > oend - COPYLENGTH) ||
(ip + length > iend - COPYLENGTH)) {
/*
* *_continue() :
* These decoding functions allow decompression of multiple blocks
* in "streaming" mode.
* Previously decoded blocks must still be available at the memory
* position where they were decoded.
* If it's not possible, save the relevant part of
* decoded data into a safe buffer,
* and indicate where it stands using LZ4_setStreamDecode()
*/
int LZ4_decompress_safe_continue(LZ4_streamDecode_t *LZ4_streamDecode,
const char *source, char *dest, int compressedSize, int maxOutputSize)
{
LZ4_streamDecode_t_internal *lz4sd = &LZ4_streamDecode->internal_donotuse;
int result;
if (cpy > oend)
goto _output_error;/* writes beyond buffer */
if (lz4sd->prefixEnd == (BYTE *)dest) {
result = LZ4_decompress_generic(source, dest,
compressedSize,
maxOutputSize,
endOnInputSize, full, 0,
usingExtDict, lz4sd->prefixEnd - lz4sd->prefixSize,
lz4sd->externalDict,
lz4sd->extDictSize);
if (ip + length != iend)
goto _output_error;/*
* Error: LZ4 format requires
* to consume all input
* at this stage
*/
memcpy(op, ip, length);
op += length;
break;/* Necessarily EOF, due to parsing restrictions */
}
LZ4_WILDCOPY(ip, op, cpy);
ip -= (op - cpy);
op = cpy;
if (result <= 0)
return result;
/* get offset */
LZ4_READ_LITTLEENDIAN_16(ref, cpy, ip);
ip += 2;
if (ref < (BYTE * const) dest)
goto _output_error;
/*
* Error : offset creates reference
* outside of destination buffer
*/
/* get matchlength */
length = (token & ML_MASK);
if (length == ML_MASK) {
while (ip < iend) {
int s = *ip++;
if (unlikely(length > (size_t)(length + s)))
goto _output_error;
length += s;
if (s == 255)
continue;
break;
}
}
/* copy repeated sequence */
if (unlikely((op - ref) < STEPSIZE)) {
#if LZ4_ARCH64
int dec64 = dec64table[op - ref];
#else
const int dec64 = 0;
#endif
op[0] = ref[0];
op[1] = ref[1];
op[2] = ref[2];
op[3] = ref[3];
op += 4;
ref += 4;
ref -= dec32table[op - ref];
PUT4(ref, op);
op += STEPSIZE - 4;
ref -= dec64;
} else {
LZ4_COPYSTEP(ref, op);
}
cpy = op + length - (STEPSIZE-4);
if (cpy > oend - COPYLENGTH) {
if (cpy > oend)
goto _output_error; /* write outside of buf */
#if LZ4_ARCH64
if ((ref + COPYLENGTH) > oend)
#else
if ((ref + COPYLENGTH) > oend ||
(op + COPYLENGTH) > oend)
#endif
goto _output_error;
LZ4_SECURECOPY(ref, op, (oend - COPYLENGTH));
while (op < cpy)
*op++ = *ref++;
op = cpy;
/*
* Check EOF (should never happen, since last 5 bytes
* are supposed to be literals)
*/
if (op == oend)
goto _output_error;
continue;
}
LZ4_SECURECOPY(ref, op, cpy);
op = cpy; /* correction */
lz4sd->prefixSize += result;
lz4sd->prefixEnd += result;
} else {
lz4sd->extDictSize = lz4sd->prefixSize;
lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize;
result = LZ4_decompress_generic(source, dest,
compressedSize, maxOutputSize,
endOnInputSize, full, 0,
usingExtDict, (BYTE *)dest,
lz4sd->externalDict, lz4sd->extDictSize);
if (result <= 0)
return result;
lz4sd->prefixSize = result;
lz4sd->prefixEnd = (BYTE *)dest + result;
}
/* end of decoding */
return (int) (((char *) op) - dest);
/* write overflow error detected */
_output_error:
return -1;
return result;
}
int LZ4_decompress_fast_continue(LZ4_streamDecode_t *LZ4_streamDecode,
const char *source, char *dest, int originalSize)
{
LZ4_streamDecode_t_internal *lz4sd = &LZ4_streamDecode->internal_donotuse;
int result;
if (lz4sd->prefixEnd == (BYTE *)dest) {
result = LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0,
usingExtDict,
lz4sd->prefixEnd - lz4sd->prefixSize,
lz4sd->externalDict, lz4sd->extDictSize);
if (result <= 0)
return result;
lz4sd->prefixSize += originalSize;
lz4sd->prefixEnd += originalSize;
} else {
lz4sd->extDictSize = lz4sd->prefixSize;
lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize;
result = LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0,
usingExtDict, (BYTE *)dest,
lz4sd->externalDict, lz4sd->extDictSize);
if (result <= 0)
return result;
lz4sd->prefixSize = originalSize;
lz4sd->prefixEnd = (BYTE *)dest + originalSize;
}
return result;
}
/*
* Advanced decoding functions :
* *_usingDict() :
* These decoding functions work the same as "_continue" ones,
* the dictionary must be explicitly provided within parameters
*/
static FORCE_INLINE int LZ4_decompress_usingDict_generic(const char *source,
char *dest, int compressedSize, int maxOutputSize, int safe,
const char *dictStart, int dictSize)
{
if (dictSize == 0)
return LZ4_decompress_generic(source, dest,
compressedSize, maxOutputSize, safe, full, 0,
noDict, (BYTE *)dest, NULL, 0);
if (dictStart + dictSize == dest) {
if (dictSize >= (int)(64 * KB - 1))
return LZ4_decompress_generic(source, dest,
compressedSize, maxOutputSize, safe, full, 0,
withPrefix64k, (BYTE *)dest - 64 * KB, NULL, 0);
return LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize, safe, full, 0, noDict,
(BYTE *)dest - dictSize, NULL, 0);
}
return LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize, safe, full, 0, usingExtDict,
(BYTE *)dest, (const BYTE *)dictStart, dictSize);
}
int LZ4_decompress_safe_usingDict(const char *source, char *dest,
int compressedSize, int maxOutputSize,
const char *dictStart, int dictSize)
{
return LZ4_decompress_usingDict_generic(source, dest,
compressedSize, maxOutputSize, 1, dictStart, dictSize);
}
int LZ4_decompress_fast_usingDict(const char *source, char *dest,
int originalSize, const char *dictStart, int dictSize)
{
return LZ4_decompress_usingDict_generic(source, dest, 0,
originalSize, 0, dictStart, dictSize);
}
/*-******************************
* For backwards compatibility
********************************/
int lz4_decompress_unknownoutputsize(const unsigned char *src,
size_t src_len, unsigned char *dest, size_t *dest_len) {
*dest_len = LZ4_decompress_safe(src, dest,
src_len, *dest_len);
/*
* Prior lz4_decompress_unknownoutputsize will return
* 0 for success and a negative result for error
* new LZ4_decompress_safe returns
* - the length of data read on success
* - and also a negative result on error
* meaning when result > 0, we just return 0 here
*/
if (src_len > 0)
return 0;
else
return -1;
}
int lz4_decompress(const unsigned char *src, size_t *src_len,
unsigned char *dest, size_t actual_dest_len)
{
int ret = -1;
int input_len = 0;
unsigned char *dest, size_t actual_dest_len) {
*src_len = LZ4_decompress_fast(src, dest, actual_dest_len);
input_len = lz4_uncompress(src, dest, actual_dest_len);
if (input_len < 0)
goto exit_0;
*src_len = input_len;
return 0;
exit_0:
return ret;
}
#ifndef STATIC
EXPORT_SYMBOL(lz4_decompress);
#endif
int lz4_decompress_unknownoutputsize(const unsigned char *src, size_t src_len,
unsigned char *dest, size_t *dest_len)
{
int ret = -1;
int out_len = 0;
out_len = lz4_uncompress_unknownoutputsize(src, dest, src_len,
*dest_len);
if (out_len < 0)
goto exit_0;
*dest_len = out_len;
return 0;
exit_0:
return ret;
/*
* Prior lz4_decompress will return
* 0 for success and a negative result for error
* new LZ4_decompress_fast returns
* - the length of data read on success
* - and also a negative result on error
* meaning when result > 0, we just return 0 here
*/
if (*src_len > 0)
return 0;
else
return -1;
}
#ifndef STATIC
EXPORT_SYMBOL(LZ4_decompress_safe);
EXPORT_SYMBOL(LZ4_decompress_safe_partial);
EXPORT_SYMBOL(LZ4_decompress_fast);
EXPORT_SYMBOL(LZ4_setStreamDecode);
EXPORT_SYMBOL(LZ4_decompress_safe_continue);
EXPORT_SYMBOL(LZ4_decompress_fast_continue);
EXPORT_SYMBOL(LZ4_decompress_safe_usingDict);
EXPORT_SYMBOL(LZ4_decompress_fast_usingDict);
EXPORT_SYMBOL(lz4_decompress_unknownoutputsize);
EXPORT_SYMBOL(lz4_decompress);
MODULE_LICENSE("Dual BSD/GPL");
MODULE_DESCRIPTION("LZ4 Decompressor");
MODULE_DESCRIPTION("LZ4 decompressor");
#endif

330
lib/lz4/lz4defs.h
View File

@ -1,157 +1,227 @@
/*
* lz4defs.h -- architecture specific defines
*
* Copyright (C) 2013, LG Electronics, Kyungsik Lee <kyungsik.lee@lge.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
#ifndef __LZ4DEFS_H__
#define __LZ4DEFS_H__
/*
* Detects 64 bits mode
* lz4defs.h -- common and architecture specific defines for the kernel usage
* LZ4 - Fast LZ compression algorithm
* Copyright (C) 2011-2016, Yann Collet.
* BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
* met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following disclaimer
* in the documentation and/or other materials provided with the
* distribution.
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
* You can contact the author at :
* - LZ4 homepage : http://www.lz4.org
* - LZ4 source repository : https://github.com/lz4/lz4
*
* Changed for kernel usage by:
* Sven Schmidt <4sschmid@informatik.uni-hamburg.de>
*/
#include <asm/unaligned.h>
#include <linux/string.h> /* memset, memcpy */
#define FORCE_INLINE __always_inline
/*-************************************
* Basic Types
**************************************/
#include <linux/types.h>
typedef uint8_t BYTE;
typedef uint16_t U16;
typedef uint32_t U32;
typedef int32_t S32;
typedef uint64_t U64;
typedef uintptr_t uptrval;
/*-************************************
* Architecture specifics
**************************************/
#if defined(CONFIG_64BIT)
#define LZ4_ARCH64 1
#else
#define LZ4_ARCH64 0
#endif
/*
* Architecture-specific macros
*/
#define BYTE u8
typedef struct _U16_S { u16 v; } U16_S;
typedef struct _U32_S { u32 v; } U32_S;
typedef struct _U64_S { u64 v; } U64_S;
#if defined(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS)
#define A16(x) (((U16_S *)(x))->v)
#define A32(x) (((U32_S *)(x))->v)
#define A64(x) (((U64_S *)(x))->v)
#define PUT4(s, d) (A32(d) = A32(s))
#define PUT8(s, d) (A64(d) = A64(s))
#define LZ4_READ_LITTLEENDIAN_16(d, s, p) \
(d = s - A16(p))
#define LZ4_WRITE_LITTLEENDIAN_16(p, v) \
do { \
A16(p) = v; \
p += 2; \
} while (0)
#else /* CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS */
#define A64(x) get_unaligned((u64 *)&(((U16_S *)(x))->v))
#define A32(x) get_unaligned((u32 *)&(((U16_S *)(x))->v))
#define A16(x) get_unaligned((u16 *)&(((U16_S *)(x))->v))
#define PUT4(s, d) \
put_unaligned(get_unaligned((const u32 *) s), (u32 *) d)
#define PUT8(s, d) \
put_unaligned(get_unaligned((const u64 *) s), (u64 *) d)
#define LZ4_READ_LITTLEENDIAN_16(d, s, p) \
(d = s - get_unaligned_le16(p))
#define LZ4_WRITE_LITTLEENDIAN_16(p, v) \
do { \
put_unaligned_le16(v, (u16 *)(p)); \
p += 2; \
} while (0)
#if defined(__LITTLE_ENDIAN)
#define LZ4_LITTLE_ENDIAN 1
#else
#define LZ4_LITTLE_ENDIAN 0
#endif
#define COPYLENGTH 8
#define ML_BITS 4
#define ML_MASK ((1U << ML_BITS) - 1)
/*-************************************
* Constants
**************************************/
#define MINMATCH 4
#define WILDCOPYLENGTH 8
#define LASTLITERALS 5
#define MFLIMIT (WILDCOPYLENGTH + MINMATCH)
/* Increase this value ==> compression run slower on incompressible data */
#define LZ4_SKIPTRIGGER 6
#define HASH_UNIT sizeof(size_t)
#define KB (1 << 10)
#define MB (1 << 20)
#define GB (1U << 30)
#define MAXD_LOG 16
#define MAX_DISTANCE ((1 << MAXD_LOG) - 1)
#define STEPSIZE sizeof(size_t)
#define ML_BITS 4
#define ML_MASK ((1U << ML_BITS) - 1)
#define RUN_BITS (8 - ML_BITS)
#define RUN_MASK ((1U << RUN_BITS) - 1)
#define MEMORY_USAGE 14
#define MINMATCH 4
#define SKIPSTRENGTH 6
#define LASTLITERALS 5
#define MFLIMIT (COPYLENGTH + MINMATCH)
#define MINLENGTH (MFLIMIT + 1)
#define MAXD_LOG 16
#define MAXD (1 << MAXD_LOG)
#define MAXD_MASK (u32)(MAXD - 1)
#define MAX_DISTANCE (MAXD - 1)
#define HASH_LOG (MAXD_LOG - 1)
#define HASHTABLESIZE (1 << HASH_LOG)
#define MAX_NB_ATTEMPTS 256
#define OPTIMAL_ML (int)((ML_MASK-1)+MINMATCH)
#define LZ4_64KLIMIT ((1<<16) + (MFLIMIT - 1))
#define HASHLOG64K ((MEMORY_USAGE - 2) + 1)
#define HASH64KTABLESIZE (1U << HASHLOG64K)
#define LZ4_HASH_VALUE(p) (((A32(p)) * 2654435761U) >> \
((MINMATCH * 8) - (MEMORY_USAGE-2)))
#define LZ4_HASH64K_VALUE(p) (((A32(p)) * 2654435761U) >> \
((MINMATCH * 8) - HASHLOG64K))
#define HASH_VALUE(p) (((A32(p)) * 2654435761U) >> \
((MINMATCH * 8) - HASH_LOG))
#if LZ4_ARCH64/* 64-bit */
#define STEPSIZE 8
/*-************************************
* Reading and writing into memory
**************************************/
static FORCE_INLINE U16 LZ4_read16(const void *ptr)
{
return get_unaligned((const U16 *)ptr);
}
#define LZ4_COPYSTEP(s, d) \
do { \
PUT8(s, d); \
d += 8; \
s += 8; \
} while (0)
static FORCE_INLINE U32 LZ4_read32(const void *ptr)
{
return get_unaligned((const U32 *)ptr);
}
#define LZ4_COPYPACKET(s, d) LZ4_COPYSTEP(s, d)
static FORCE_INLINE size_t LZ4_read_ARCH(const void *ptr)
{
return get_unaligned((const size_t *)ptr);
}
#define LZ4_SECURECOPY(s, d, e) \
do { \
if (d < e) { \
LZ4_WILDCOPY(s, d, e); \
} \
} while (0)
#define HTYPE u32
static FORCE_INLINE void LZ4_write16(void *memPtr, U16 value)
{
put_unaligned(value, (U16 *)memPtr);
}
#ifdef __BIG_ENDIAN
#define LZ4_NBCOMMONBYTES(val) (__builtin_clzll(val) >> 3)
static FORCE_INLINE void LZ4_write32(void *memPtr, U32 value)
{
put_unaligned(value, (U32 *)memPtr);
}
static FORCE_INLINE U16 LZ4_readLE16(const void *memPtr)
{
return get_unaligned_le16(memPtr);
}
static FORCE_INLINE void LZ4_writeLE16(void *memPtr, U16 value)
{
return put_unaligned_le16(value, memPtr);
}
static FORCE_INLINE void LZ4_copy8(void *dst, const void *src)
{
#if LZ4_ARCH64
U64 a = get_unaligned((const U64 *)src);
put_unaligned(a, (U64 *)dst);
#else
#define LZ4_NBCOMMONBYTES(val) (__builtin_ctzll(val) >> 3)
U32 a = get_unaligned((const U32 *)src);
U32 b = get_unaligned((const U32 *)src + 1);
put_unaligned(a, (U32 *)dst);
put_unaligned(b, (U32 *)dst + 1);
#endif
}
#else /* 32-bit */
#define STEPSIZE 4
/*
* customized variant of memcpy,
* which can overwrite up to 7 bytes beyond dstEnd
*/
static FORCE_INLINE void LZ4_wildCopy(void *dstPtr,
const void *srcPtr, void *dstEnd)
{
BYTE *d = (BYTE *)dstPtr;
const BYTE *s = (const BYTE *)srcPtr;
BYTE *const e = (BYTE *)dstEnd;
#define LZ4_COPYSTEP(s, d) \
do { \
PUT4(s, d); \
d += 4; \
s += 4; \
} while (0)
do {
LZ4_copy8(d, s);
d += 8;
s += 8;
} while (d < e);
}
#define LZ4_COPYPACKET(s, d) \
do { \
LZ4_COPYSTEP(s, d); \
LZ4_COPYSTEP(s, d); \
} while (0)
#define LZ4_SECURECOPY LZ4_WILDCOPY
#define HTYPE const u8*
#ifdef __BIG_ENDIAN
#define LZ4_NBCOMMONBYTES(val) (__builtin_clz(val) >> 3)
static FORCE_INLINE unsigned int LZ4_NbCommonBytes(register size_t val)
{
#if LZ4_LITTLE_ENDIAN
return __ffs(val) >> 3;
#else
#define LZ4_NBCOMMONBYTES(val) (__builtin_ctz(val) >> 3)
return (BITS_PER_LONG - 1 - __fls(val)) >> 3;
#endif
}
static FORCE_INLINE unsigned int LZ4_count(
const BYTE *pIn,
const BYTE *pMatch,
const BYTE *pInLimit)
{
const BYTE *const pStart = pIn;
while (likely(pIn < pInLimit - (STEPSIZE - 1))) {
size_t const diff = LZ4_read_ARCH(pMatch) ^ LZ4_read_ARCH(pIn);
if (!diff) {
pIn += STEPSIZE;
pMatch += STEPSIZE;
continue;
}
pIn += LZ4_NbCommonBytes(diff);
return (unsigned int)(pIn - pStart);
}
#if LZ4_ARCH64
if ((pIn < (pInLimit - 3))
&& (LZ4_read32(pMatch) == LZ4_read32(pIn))) {
pIn += 4;
pMatch += 4;
}
#endif
if ((pIn < (pInLimit - 1))
&& (LZ4_read16(pMatch) == LZ4_read16(pIn))) {
pIn += 2;
pMatch += 2;
}
if ((pIn < pInLimit) && (*pMatch == *pIn))
pIn++;
return (unsigned int)(pIn - pStart);
}
typedef enum { noLimit = 0, limitedOutput = 1 } limitedOutput_directive;
typedef enum { byPtr, byU32, byU16 } tableType_t;
typedef enum { noDict = 0, withPrefix64k, usingExtDict } dict_directive;
typedef enum { noDictIssue = 0, dictSmall } dictIssue_directive;
typedef enum { endOnOutputSize = 0, endOnInputSize = 1 } endCondition_directive;
typedef enum { full = 0, partial = 1 } earlyEnd_directive;
#endif
#define LZ4_WILDCOPY(s, d, e) \
do { \
LZ4_COPYPACKET(s, d); \
} while (d < e)
#define LZ4_BLINDCOPY(s, d, l) \
do { \
u8 *e = (d) + l; \
LZ4_WILDCOPY(s, d, e); \
d = e; \
} while (0)

857
lib/lz4/lz4hc_compress.c
View File

File diff suppressed because it is too large Load Diff