Dear Group members,
I am reading a large deflated/gzipped file in chunks. And I want to
inflate/ungzip each chunk as it arrives. However, the call to first
inflate gives no error but subsequent calls give data error (-3).
I have tried all parameters i.e. Z_NFLUSH, Z_FULL_FLUSH,
Z_SYNC_FLUSH.
After the call to inflate (&stream, err)
stream.total_out is always equal to what ever the value of
stream.avail_out was set. If avail_out was set to 10000 then the
total_out was 10000, if avail_out set to 16000 then total_out is 16000.
and so on.
The code works fine if I read the whole file at once and use Z_FINISH
and write it out, but does not work in chunks.
Following is my code if anyone can help you understand my problem. I
will greatly appreciate your help.
Thanks.
arun
#include <stdio.h>
#include "zlib.h"
#include <string.h>
#include <stdlib.h>
int main(void)
{
Byte *compr, *uncompr;
uLong comprLen = 4096*sizeof(int);
uLong uncomprLen = 4096*sizeof(int);
static const char* myVersion = ZLIB_VERSIN;
FILE* inFile;
FILE* outFile;
int err;
int lenRead;
int w;
z_stream c_stream; /* decompression stream */
compr = (Byte*)calloc((uInt)comprLen, 1);
uncompr = (Byte*)calloc((uInt)uncomprLen, 1);
if (compr == Z_NULL || uncompr == Z_NULL) {
printf("out of memory\n");
exit(1);
}
inFile = fopen("deflatedfile", "rb");
outFile = fopen("undeflatedfile2", "w");
if(inFile == NULL) {
printf("File could not be opened \n");
exit(1);
}
c_stream.zalloc = (alloc_func)0;
c_stream.zfree = (free_func)0;
c_stream.opaque = (voidpf)0;
c_stream.next_in = 400;
c_stream.avail_in = comprLen; //strlen(compr);
printf("before next in is %p \n", c_stream.next_in);
err = inflateInit(&c_stream);
printf("err inflateinit is %d next in is %p \n", err,
c_stream.next_in);
c_stream.next_in = 400;
c_stream.avail_in = comprLen; //strlen(compr);
c_stream.next_out = uncompr;
c_stream.avail_out = 3000; // Enough space for inflation
printf("total out %d\n", c_stream.total_out);
while ((lenRead = fread(compr, 1, 400, inFile)) != 0) {
printf("lenread is %d \n", lenRead);
err = inflate(&c_stream, Z_SYNC_FLUSH);// I have tried all
options FULL_FLUSH,
//
NFLUSH, FINISH
// Here the toal_out parameter is always = avail_out parameter =
3000
// And the err is -3 DATA_ERRR
printf("err inflate is %d total out %d\n", err,
c_stream.total_out);
printf("avail out %d\n", err, c_stream.avail_out);
// Since total_out is more the write to file has some unwanted
bytes written to it.
w = fwrite(uncompr, 1, c_stream.total_out, outFile);
printf("w is %d\n", w);
err = inflateReset(&c_stream);
printf("err inflate reset is %d \n", err);
c_stream.next_in = compr;
c_stream.avail_in = 400;
c_stream.next_out = uncompr;
c_stream.avail_out = 3000;
}
fclose(inFile);
fclose(outFile);
free(compr);
free(uncompr);
return 0;
}

Vinay Nagrik wrote:
However, the call to first
inflate gives no error but subsequent calls give data error (-3).

You should look at http://zlib.net/zlib_how.html for an example of how
to use the zlib functions correctly. You are not updating avail_in and
next_in with the newly read input.

mark

Mark Adler wrote:
Vinay Nagrik wrote:
However, the call to first
inflate gives no error but subsequent calls give data error (-3).

You should look at http://zlib.net/zlib_how.html for an example of how
to use the zlib functions correctly. You are not updating avail_in and
next_in with the newly read input.

mark

Mark,

That site and code over there helped me a lot, but my job remains
unfinished. The code works find if the whole file is to be inflated in
a single function. However, my http stack
implementatin requires me to make function calls. Here is how the
function calls look
like.

main (void).// input/output files and decide the chunk size.
Call fn1(char *buf, FILE* fptr, int size).

fn1(buf, fptr, size) // initializes z_stream and calls fn2(buf, fptr,
size) to read from the
destinatin file so that it could then inflate the data and return to
main.

fn2(buf, fptr, size) // read required number of bytes from fptr in buf
and return.

This code works fine for first and only first inflate. Since the data
is being read in chunk
sizes the second function call fails in inflate with Z_DATA_ERRR.

I have differenct combinations.

1. Initializing everytime/(only one time) when I enter the function.
2. Calling/(not calling) inflateReset/inflateEnd after each call to
function.

None of the combination seems to work.

please help. It is very important to me.

Here is my complete code.I must be making some stupid mistake, which
you can point
out.

For read file you can create any gzip file and kindly select any small
chunk size to read
the file so that many function calls are made from main function.

Thanks.

I look forward to hear from you.

arun

code
#include <stdio.h>
#include "zlib.h"
#include <string.h>
#include <stdlib.h>
#include <iostream>

using namespace std;

int chunksize = 2048;

int readFn(char* inbuf, FILE* fPtr, int size);
int inflateData(char* inbuf, FILE* fPtr, int size);

int main(void)
{
FILE* inFile;
FILE* outFile;
char readBuf[chunksize];
int readLen;

inFile = fopen("gzipImg.jpg.gz", "rb");

if(inFile == NULL) {
printf("Input file could not be opened \n");
exit(1);
}

outFile = fopen("phoneImg.jpg", "w");

if(outFile == NULL) {
printf(" file could not be opened \n");
exit(1);
}

while (( readLen = inflateData(readBuf, inFile, chunksize)) != 0) {
fwrite(readBuf, 1, readLen, outFile);
}

}

int readFn(char* inbuf, FILE* fPtr, int size) {

int len = fread(inbuf, 1, size, fPtr);
return len;

}

int inflateData(char* inbuf, FILE* fPtr, int size) {

int decompressionError;
z_stream c_stream;
int dataLen;
static bool firsttime = true;
int outsize = size/4;
char outbuf[outsize];

if ( firsttime) {
c_stream.zalloc = (alloc_func)0;
c_stream.zfree = (free_func)0;
c_stream.opaque = (voidpf)0;

c_stream.next_in = Z_NULL;
c_stream.avail_in = 0;

decompressionError = inflateInit2(&c_stream, 47);
if ( decompressionError != ZK)
return Z_ERRN;

firsttime = false;
}
else {
c_stream.zalloc = (alloc_func)0;
c_stream.zfree = (free_func)0;
c_stream.opaque = (voidpf)0;

c_stream.next_in = Z_NULL;
c_stream.avail_in = 0;

decompressionError = inflateInit(&c_stream);
if ( decompressionError != ZK)
return Z_ERRN;

firsttime = false;

}
c_stream.avail_in = readFn(outbuf, fPtr, outsize);
cout << "avail in is " << c_stream.avail_in << endl;

if (c_stream.avail_in == 0)
return 0;
c_stream.next_in = (Byte *)outbuf;

do {

c_stream.next_out = (Byte*)inbuf;
c_stream.avail_out = size;
cout << "avail out is " << c_stream.avail_out << endl;
decompressionError = inflate(&c_stream, Z_NFLUSH);
cout << "error is " << decompressionError << endl; //
Z_DATA_ERRR after first time
dataLen = size - c_stream.avail_out;
cout << "datalen is " << dataLen << endl;
cout << "avail out is " << c_stream.avail_out << endl;
decompressionError = inflateEnd(&c_stream);
cout << "error is " << decompressionError << endl;

} while (c_stream.avail_out ==0);

return dataLen;
}

Vinay Nagrik wrote:
Mark Adler wrote:
You should look at http://zlib.net/zlib_how.html for an example of how
to use the zlib functions correctly.

That site and code over there helped me a lot,

Well, apparently not. It would appear that you did not read that site
at all. In your posted, truly perplexing code, you are not
understanding even the most basic ordering of the function calls. You
call inflateInit(), once, then you call inflate() one or more times,
then you call inflateEnd() once inflate() reports that it is done,
once.

I strongly suggest that you read http://zlib.net/zlib_how.html very
carefully. Start with that code and modify it to suit your needs,
checking frequently that the code continues to work as you modify it.

mark

Mark Adler wrote:
Vinay Nagrik wrote:
Mark Adler wrote:
You should look at http://zlib.net/zlib_how.html for an example of how
to use the zlib functions correctly.

That site and code over there helped me a lot,

Well, apparently not. It would appear that you did not read that site
at all. In your posted, truly perplexing code, you are not
understanding even the most basic ordering of the function calls. You
call inflateInit(), once, then you call inflate() one or more times,
then you call inflateEnd() once inflate() reports that it is done,
once.

I strongly suggest that you read http://zlib.net/zlib_how.html very
carefully. Start with that code and modify it to suit your needs,
checking frequently that the code continues to work as you modify it.

mark

Dear Mark,

Thank you for your quick help. Your code and advice helped me solve my
problem.
Especially the clue that init and end should only be done only once.
Rest all I did by
declaring appropriate data structure.

Thanks once again.

nagrik