Oral  Brekke

Oral Brekke

1646124780

Yauzl: Yet another Unzip Library for Node

yauzl

yet another unzip library for node. For zipping, see yazl.

Design principles:

  • Follow the spec. Don't scan for local file headers. Read the central directory for file metadata. (see No Streaming Unzip API).
  • Don't block the JavaScript thread. Use and provide async APIs.
  • Keep memory usage under control. Don't attempt to buffer entire files in RAM at once.
  • Never crash (if used properly). Don't let malformed zip files bring down client applications who are trying to catch errors.
  • Catch unsafe file names. See validateFileName().

Usage

var yauzl = require("yauzl");

yauzl.open("path/to/file.zip", {lazyEntries: true}, function(err, zipfile) {
  if (err) throw err;
  zipfile.readEntry();
  zipfile.on("entry", function(entry) {
    if (/\/$/.test(entry.fileName)) {
      // Directory file names end with '/'.
      // Note that entries for directories themselves are optional.
      // An entry's fileName implicitly requires its parent directories to exist.
      zipfile.readEntry();
    } else {
      // file entry
      zipfile.openReadStream(entry, function(err, readStream) {
        if (err) throw err;
        readStream.on("end", function() {
          zipfile.readEntry();
        });
        readStream.pipe(somewhere);
      });
    }
  });
});

See also examples/ for more usage examples.

API

The default for every optional callback parameter is:

function defaultCallback(err) {
  if (err) throw err;
}

open(path, [options], [callback])

Calls fs.open(path, "r") and reads the fd effectively the same as fromFd() would.

options may be omitted or null. The defaults are {autoClose: true, lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

autoClose is effectively equivalent to:

zipfile.once("end", function() {
  zipfile.close();
});

lazyEntries indicates that entries should be read only when readEntry() is called. If lazyEntries is false, entry events will be emitted as fast as possible to allow pipe()ing file data from all entries in parallel. This is not recommended, as it can lead to out of control memory usage for zip files with many entries. See issue #22. If lazyEntries is true, an entry or end event will be emitted in response to each call to readEntry(). This allows processing of one entry at a time, and will keep memory usage under control for zip files with many entries.

decodeStrings is the default and causes yauzl to decode strings with CP437 or UTF-8 as required by the spec. The exact effects of turning this option off are:

  • zipfile.comment, entry.fileName, and entry.fileComment will be Buffer objects instead of Strings.
  • Any Info-ZIP Unicode Path Extra Field will be ignored. See extraFields.
  • Automatic file name validation will not be performed. See validateFileName().

validateEntrySizes is the default and ensures that an entry's reported uncompressed size matches its actual uncompressed size. This check happens as early as possible, which is either before emitting each "entry" event (for entries with no compression), or during the readStream piping after calling openReadStream(). See openReadStream() for more information on defending against zip bomb attacks.

When strictFileNames is false (the default) and decodeStrings is true, all backslash (\) characters in each entry.fileName are replaced with forward slashes (/). The spec forbids file names with backslashes, but Microsoft's System.IO.Compression.ZipFile class in .NET versions 4.5.0 until 4.6.1 creates non-conformant zipfiles with backslashes in file names. strictFileNames is false by default so that clients can read these non-conformant zipfiles without knowing about this Microsoft-specific bug. When strictFileNames is true and decodeStrings is true, entries with backslashes in their file names will result in an error. See validateFileName(). When decodeStrings is false, strictFileNames has no effect.

The callback is given the arguments (err, zipfile). An err is provided if the End of Central Directory Record cannot be found, or if its metadata appears malformed. This kind of error usually indicates that this is not a zip file. Otherwise, zipfile is an instance of ZipFile.

fromFd(fd, [options], [callback])

Reads from the fd, which is presumed to be an open .zip file. Note that random access is required by the zip file specification, so the fd cannot be an open socket or any other fd that does not support random access.

options may be omitted or null. The defaults are {autoClose: false, lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

See open() for the meaning of the options and callback.

fromBuffer(buffer, [options], [callback])

Like fromFd(), but reads from a RAM buffer instead of an open file. buffer is a Buffer.

If a ZipFile is acquired from this method, it will never emit the close event, and calling close() is not necessary.

options may be omitted or null. The defaults are {lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

See open() for the meaning of the options and callback. The autoClose option is ignored for this method.

fromRandomAccessReader(reader, totalSize, [options], [callback])

This method of reading a zip file allows clients to implement their own back-end file system. For example, a client might translate read calls into network requests.

The reader parameter must be of a type that is a subclass of RandomAccessReader that implements the required methods. The totalSize is a Number and indicates the total file size of the zip file.

options may be omitted or null. The defaults are {autoClose: true, lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

See open() for the meaning of the options and callback.

dosDateTimeToDate(date, time)

Converts MS-DOS date and time data into a JavaScript Date object. Each parameter is a Number treated as an unsigned 16-bit integer. Note that this format does not support timezones, so the returned object will use the local timezone.

validateFileName(fileName)

Returns null or a String error message depending on the validity of fileName. If fileName starts with "/" or /[A-Za-z]:\// or if it contains ".." path segments or "\\", this function returns an error message appropriate for use like this:

var errorMessage = yauzl.validateFileName(fileName);
if (errorMessage != null) throw new Error(errorMessage);

This function is automatically run for each entry, as long as decodeStrings is true. See open(), strictFileNames, and Event: "entry" for more information.

Class: ZipFile

The constructor for the class is not part of the public API. Use open(), fromFd(), fromBuffer(), or fromRandomAccessReader() instead.

Event: "entry"

Callback gets (entry), which is an Entry. See open() and readEntry() for when this event is emitted.

If decodeStrings is true, entries emitted via this event have already passed file name validation. See validateFileName() and open() for more information.

If validateEntrySizes is true and this entry's compressionMethod is 0 (stored without compression), this entry has already passed entry size validation. See open() for more information.

Event: "end"

Emitted after the last entry event has been emitted. See open() and readEntry() for more info on when this event is emitted.

Event: "close"

Emitted after the fd is actually closed. This is after calling close() (or after the end event when autoClose is true), and after all stream pipelines created from openReadStream() have finished reading data from the fd.

If this ZipFile was acquired from fromRandomAccessReader(), the "fd" in the previous paragraph refers to the RandomAccessReader implemented by the client.

If this ZipFile was acquired from fromBuffer(), this event is never emitted.

Event: "error"

Emitted in the case of errors with reading the zip file. (Note that other errors can be emitted from the streams created from openReadStream() as well.) After this event has been emitted, no further entry, end, or error events will be emitted, but the close event may still be emitted.

readEntry()

Causes this ZipFile to emit an entry or end event (or an error event). This method must only be called when this ZipFile was created with the lazyEntries option set to true (see open()). When this ZipFile was created with the lazyEntries option set to true, entry and end events are only ever emitted in response to this method call.

The event that is emitted in response to this method will not be emitted until after this method has returned, so it is safe to call this method before attaching event listeners.

After calling this method, calling this method again before the response event has been emitted will cause undefined behavior. Calling this method after the end event has been emitted will cause undefined behavior. Calling this method after calling close() will cause undefined behavior.

openReadStream(entry, [options], callback)

entry must be an Entry object from this ZipFile. callback gets (err, readStream), where readStream is a Readable Stream that provides the file data for this entry. If this zipfile is already closed (see close()), the callback will receive an err.

options may be omitted or null, and has the following defaults:

{
  decompress: entry.isCompressed() ? true : null,
  decrypt: null,
  start: 0,                  // actually the default is null, see below
  end: entry.compressedSize, // actually the default is null, see below
}

If the entry is compressed (with a supported compression method), and the decompress option is true (or omitted), the read stream provides the decompressed data. Omitting the decompress option is what most clients should do.

The decompress option must be null (or omitted) when the entry is not compressed (see isCompressed()), and either true (or omitted) or false when the entry is compressed. Specifying decompress: false for a compressed entry causes the read stream to provide the raw compressed file data without going through a zlib inflate transform.

If the entry is encrypted (see isEncrypted()), clients may want to avoid calling openReadStream() on the entry entirely. Alternatively, clients may call openReadStream() for encrypted entries and specify decrypt: false. If the entry is also compressed, clients must also specify decompress: false. Specifying decrypt: false for an encrypted entry causes the read stream to provide the raw, still-encrypted file data. (This data includes the 12-byte header described in the spec.)

The decrypt option must be null (or omitted) for non-encrypted entries, and false for encrypted entries. Omitting the decrypt option (or specifying it as null) for an encrypted entry will result in the callback receiving an err. This default behavior is so that clients not accounting for encrypted files aren't surprised by bogus file data.

The start (inclusive) and end (exclusive) options are byte offsets into this entry's file data, and can be used to obtain part of an entry's file data rather than the whole thing. If either of these options are specified and non-null, then the above options must be used to obain the file's raw data. Specifying {start: 0, end: entry.compressedSize} will result in the complete file, which is effectively the default values for these options, but note that unlike omitting the options, when you specify start or end as any non-null value, the above requirement is still enforced that you must also pass the appropriate options to get the file's raw data.

It's possible for the readStream provided to the callback to emit errors for several reasons. For example, if zlib cannot decompress the data, the zlib error will be emitted from the readStream. Two more error cases (when validateEntrySizes is true) are if the decompressed data has too many or too few actual bytes compared to the reported byte count from the entry's uncompressedSize field. yauzl notices this false information and emits an error from the readStream after some number of bytes have already been piped through the stream.

This check allows clients to trust the uncompressedSize field in Entry objects. Guarding against zip bomb attacks can be accomplished by doing some heuristic checks on the size metadata and then watching out for the above errors. Such heuristics are outside the scope of this library, but enforcing the uncompressedSize is implemented here as a security feature.

It is possible to destroy the readStream before it has piped all of its data. To do this, call readStream.destroy(). You must unpipe() the readStream from any destination before calling readStream.destroy(). If this zipfile was created using fromRandomAccessReader(), the RandomAccessReader implementation must provide readable streams that implement a .destroy() method (see randomAccessReader._readStreamForRange()) in order for calls to readStream.destroy() to work in this context.

close()

Causes all future calls to openReadStream() to fail, and closes the fd, if any, after all streams created by openReadStream() have emitted their end events.

If the autoClose option is set to true (see open()), this function will be called automatically effectively in response to this object's end event.

If the lazyEntries option is set to false (see open()) and this object's end event has not been emitted yet, this function causes undefined behavior. If the lazyEntries option is set to true, you can call this function instead of calling readEntry() to abort reading the entries of a zipfile.

It is safe to call this function multiple times; after the first call, successive calls have no effect. This includes situations where the autoClose option effectively calls this function for you.

If close() is never called, then the zipfile is "kept open". For zipfiles created with fromFd(), this will leave the fd open, which may be desirable. For zipfiles created with open(), this will leave the underlying fd open, thereby "leaking" it, which is probably undesirable. For zipfiles created with fromRandomAccessReader(), the reader's close() method will never be called. For zipfiles created with fromBuffer(), the close() function has no effect whether called or not.

Regardless of how this ZipFile was created, there are no resources other than those listed above that require cleanup from this function. This means it may be desirable to never call close() in some usecases.

isOpen

Boolean. true until close() is called; then it's false.

entryCount

Number. Total number of central directory records.

comment

String. Always decoded with CP437 per the spec.

If decodeStrings is false (see open()), this field is the undecoded Buffer instead of a decoded String.

Class: Entry

Objects of this class represent Central Directory Records. Refer to the zipfile specification for more details about these fields.

These fields are of type Number:

  • versionMadeBy
  • versionNeededToExtract
  • generalPurposeBitFlag
  • compressionMethod
  • lastModFileTime (MS-DOS format, see getLastModDateTime)
  • lastModFileDate (MS-DOS format, see getLastModDateTime)
  • crc32
  • compressedSize
  • uncompressedSize
  • fileNameLength (bytes)
  • extraFieldLength (bytes)
  • fileCommentLength (bytes)
  • internalFileAttributes
  • externalFileAttributes
  • relativeOffsetOfLocalHeader

fileName

String. Following the spec, the bytes for the file name are decoded with UTF-8 if generalPurposeBitFlag & 0x800, otherwise with CP437. Alternatively, this field may be populated from the Info-ZIP Unicode Path Extra Field (see extraFields).

This field is automatically validated by validateFileName() before yauzl emits an "entry" event. If this field would contain unsafe characters, yauzl emits an error instead of an entry.

If decodeStrings is false (see open()), this field is the undecoded Buffer instead of a decoded String. Therefore, generalPurposeBitFlag and any Info-ZIP Unicode Path Extra Field are ignored. Furthermore, no automatic file name validation is performed for this file name.

extraFields

Array with each entry in the form {id: id, data: data}, where id is a Number and data is a Buffer.

This library looks for and reads the ZIP64 Extended Information Extra Field (0x0001) in order to support ZIP64 format zip files.

This library also looks for and reads the Info-ZIP Unicode Path Extra Field (0x7075) in order to support some zipfiles that use it instead of General Purpose Bit 11 to convey UTF-8 file names. When the field is identified and verified to be reliable (see the zipfile spec), the the file name in this field is stored in the fileName property, and the file name in the central directory record for this entry is ignored. Note that when decodeStrings is false, all Info-ZIP Unicode Path Extra Fields are ignored.

None of the other fields are considered significant by this library. Fields that this library reads are left unalterned in the extraFields array.

fileComment

String decoded with the charset indicated by generalPurposeBitFlag & 0x800 as with the fileName. (The Info-ZIP Unicode Path Extra Field has no effect on the charset used for this field.)

If decodeStrings is false (see open()), this field is the undecoded Buffer instead of a decoded String.

Prior to yauzl version 2.7.0, this field was erroneously documented as comment instead of fileComment. For compatibility with any code that uses the field name comment, yauzl creates an alias field named comment which is identical to fileComment.

getLastModDate()

Effectively implemented as:

return dosDateTimeToDate(this.lastModFileDate, this.lastModFileTime);

isEncrypted()

Returns is this entry encrypted with "Traditional Encryption". Effectively implemented as:

return (this.generalPurposeBitFlag & 0x1) !== 0;

See openReadStream() for the implications of this value.

Note that "Strong Encryption" is not supported, and will result in an "error" event emitted from the ZipFile.

isCompressed()

Effectively implemented as:

return this.compressionMethod === 8;

See openReadStream() for the implications of this value.

Class: RandomAccessReader

This class is meant to be subclassed by clients and instantiated for the fromRandomAccessReader() function.

An example implementation can be found in test/test.js.

randomAccessReader._readStreamForRange(start, end)

Subclasses must implement this method.

start and end are Numbers and indicate byte offsets from the start of the file. end is exclusive, so _readStreamForRange(0x1000, 0x2000) would indicate to read 0x1000 bytes. end - start will always be at least 1.

This method should return a readable stream which will be pipe()ed into another stream. It is expected that the readable stream will provide data in several chunks if necessary. If the readable stream provides too many or too few bytes, an error will be emitted. (Note that validateEntrySizes has no effect on this check, because this is a low-level API that should behave correctly regardless of the contents of the file.) Any errors emitted on the readable stream will be handled and re-emitted on the client-visible stream (returned from zipfile.openReadStream()) or provided as the err argument to the appropriate callback (for example, for fromRandomAccessReader()).

The returned stream must implement a method .destroy() if you call readStream.destroy() on streams you get from openReadStream(). If you never call readStream.destroy(), then streams returned from this method do not need to implement a method .destroy(). .destroy() should abort any streaming that is in progress and clean up any associated resources. .destroy() will only be called after the stream has been unpipe()d from its destination.

Note that the stream returned from this method might not be the same object that is provided by openReadStream(). The stream returned from this method might be pipe()d through one or more filter streams (for example, a zlib inflate stream).

randomAccessReader.read(buffer, offset, length, position, callback)

Subclasses may implement this method. The default implementation uses createReadStream() to fill the buffer.

This method should behave like fs.read().

randomAccessReader.close(callback)

Subclasses may implement this method. The default implementation is effectively setImmediate(callback);.

callback takes parameters (err).

This method is called once the all streams returned from _readStreamForRange() have ended, and no more _readStreamForRange() or read() requests will be issued to this object.

How to Avoid Crashing

When a malformed zipfile is encountered, the default behavior is to crash (throw an exception). If you want to handle errors more gracefully than this, be sure to do the following:

  • Provide callback parameters where they are allowed, and check the err parameter.
  • Attach a listener for the error event on any ZipFile object you get from open(), fromFd(), fromBuffer(), or fromRandomAccessReader().
  • Attach a listener for the error event on any stream you get from openReadStream().

Minor version updates to yauzl will not add any additional requirements to this list.

Limitations

No Streaming Unzip API

Due to the design of the .zip file format, it's impossible to interpret a .zip file from start to finish (such as from a readable stream) without sacrificing correctness. The Central Directory, which is the authority on the contents of the .zip file, is at the end of a .zip file, not the beginning. A streaming API would need to either buffer the entire .zip file to get to the Central Directory before interpreting anything (defeating the purpose of a streaming interface), or rely on the Local File Headers which are interspersed through the .zip file. However, the Local File Headers are explicitly denounced in the spec as being unreliable copies of the Central Directory, so trusting them would be a violation of the spec.

Any library that offers a streaming unzip API must make one of the above two compromises, which makes the library either dishonest or nonconformant (usually the latter). This library insists on correctness and adherence to the spec, and so does not offer a streaming API.

Here is a way to create a spec-conformant .zip file using the zip command line program (Info-ZIP) available in most unix-like environments, that is (nearly) impossible to parse correctly with a streaming parser:

$ echo -ne '\x50\x4b\x07\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' > file.txt
$ zip -q0 - file.txt | cat > out.zip

This .zip file contains a single file entry that uses General Purpose Bit 3, which means the Local File Header doesn't know the size of the file. Any streaming parser that encounters this situation will either immediately fail, or attempt to search for the Data Descriptor after the file's contents. The file's contents is a sequence of 16-bytes crafted to exactly mimic a valid Data Descriptor for an empty file, which will fool any parser that gets this far into thinking that the file is empty rather than containing 16-bytes. What follows the file's real contents is the file's real Data Descriptor, which will likely cause some kind of signature mismatch error for a streaming parser (if one hasn't occurred already).

By using General Purpose Bit 3 (and compression method 0), it's possible to create arbitrarily ambiguous .zip files that distract parsers with file contents that contain apparently valid .zip file metadata.

Limitted ZIP64 Support

For ZIP64, only zip files smaller than 8PiB are supported, not the full 16EiB range that a 64-bit integer should be able to index. This is due to the JavaScript Number type being an IEEE 754 double precision float.

The Node.js fs module probably has this same limitation.

ZIP64 Extensible Data Sector Is Ignored

The spec does not allow zip file creators to put arbitrary data here, but rather reserves its use for PKWARE and mentions something about Z390. This doesn't seem useful to expose in this library, so it is ignored.

No Multi-Disk Archive Support

This library does not support multi-disk zip files. The multi-disk fields in the zipfile spec were intended for a zip file to span multiple floppy disks, which probably never happens now. If the "number of this disk" field in the End of Central Directory Record is not 0, the open(), fromFd(), fromBuffer(), or fromRandomAccessReader() callback will receive an err. By extension the following zip file fields are ignored by this library and not provided to clients:

  • Disk where central directory starts
  • Number of central directory records on this disk
  • Disk number where file starts

Limited Encryption Handling

You can detect when a file entry is encrypted with "Traditional Encryption" via isEncrypted(), but yauzl will not help you decrypt it. See openReadStream().

If a zip file contains file entries encrypted with "Strong Encryption", yauzl emits an error.

If the central directory is encrypted or compressed, yauzl emits an error.

Local File Headers Are Ignored

Many unzip libraries mistakenly read the Local File Header data in zip files. This data is officially defined to be redundant with the Central Directory information, and is not to be trusted. Aside from checking the signature, yauzl ignores the content of the Local File Header.

No CRC-32 Checking

This library provides the crc32 field of Entry objects read from the Central Directory. However, this field is not used for anything in this library.

versionNeededToExtract Is Ignored

The field versionNeededToExtract is ignored, because this library doesn't support the complete zip file spec at any version,

No Support For Obscure Compression Methods

Regarding the compressionMethod field of Entry objects, only method 0 (stored with no compression) and method 8 (deflated) are supported. Any of the other 15 official methods will cause the openReadStream() callback to receive an err.

Data Descriptors Are Ignored

There may or may not be Data Descriptor sections in a zip file. This library provides no support for finding or interpreting them.

Archive Extra Data Record Is Ignored

There may or may not be an Archive Extra Data Record section in a zip file. This library provides no support for finding or interpreting it.

No Language Encoding Flag Support

Zip files officially support charset encodings other than CP437 and UTF-8, but the zip file spec does not specify how it works. This library makes no attempt to interpret the Language Encoding Flag.

Change History

  • 2.10.0
    • Added support for non-conformant zipfiles created by Microsoft, and added option strictFileNames to disable the workaround. issue #66, issue #88
  • 2.9.2
  • 2.9.1
    • Removed console.log() accidentally introduced in 2.9.0. issue #64
  • 2.9.0
    • Throw an exception if readEntry() is called without lazyEntries:true. Previously this caused undefined behavior. issue #63
  • 2.8.0
    • Added option validateEntrySizes. issue #53
    • Added examples/promises.js
    • Added ability to read raw file data via decompress and decrypt options. issue #11, issue #38, pull #39
    • Added start and end options to openReadStream(). issue #38
  • 2.7.0
    • Added option decodeStrings. issue #42
    • Fixed documentation for entry.fileComment and added compatibility alias. issue #47
  • 2.6.0
    • Support Info-ZIP Unicode Path Extra Field, used by WinRAR for Chinese file names. issue #33
  • 2.5.0
    • Ignore malformed Extra Field that is common in Android .apk files. issue #31
  • 2.4.3
    • Fix crash when parsing malformed Extra Field buffers. issue #31
  • 2.4.2
    • Remove .npmignore and .travis.yml from npm package.
  • 2.4.1
    • Fix error handling.
  • 2.4.0
  • 2.3.1
    • Documentation updates.
  • 2.3.0
    • Check that uncompressedSize is correct, or else emit an error. issue #13
  • 2.2.1
    • Update dependencies.
  • 2.2.0
    • Update dependencies.
  • 2.1.0
    • Remove dependency on iconv.
  • 2.0.3
    • Fix crash when trying to read a 0-byte file.
  • 2.0.2
    • Fix event behavior after errors.
  • 2.0.1
    • Fix bug with using iconv.
  • 2.0.0
    • Initial release.

Author: Thejoshwolfe
Source Code: https://github.com/thejoshwolfe/yauzl
License: MIT License

#node #library 

What is GEEK

Buddha Community

Yauzl: Yet another Unzip Library for Node
Oral  Brekke

Oral Brekke

1646124780

Yauzl: Yet another Unzip Library for Node

yauzl

yet another unzip library for node. For zipping, see yazl.

Design principles:

  • Follow the spec. Don't scan for local file headers. Read the central directory for file metadata. (see No Streaming Unzip API).
  • Don't block the JavaScript thread. Use and provide async APIs.
  • Keep memory usage under control. Don't attempt to buffer entire files in RAM at once.
  • Never crash (if used properly). Don't let malformed zip files bring down client applications who are trying to catch errors.
  • Catch unsafe file names. See validateFileName().

Usage

var yauzl = require("yauzl");

yauzl.open("path/to/file.zip", {lazyEntries: true}, function(err, zipfile) {
  if (err) throw err;
  zipfile.readEntry();
  zipfile.on("entry", function(entry) {
    if (/\/$/.test(entry.fileName)) {
      // Directory file names end with '/'.
      // Note that entries for directories themselves are optional.
      // An entry's fileName implicitly requires its parent directories to exist.
      zipfile.readEntry();
    } else {
      // file entry
      zipfile.openReadStream(entry, function(err, readStream) {
        if (err) throw err;
        readStream.on("end", function() {
          zipfile.readEntry();
        });
        readStream.pipe(somewhere);
      });
    }
  });
});

See also examples/ for more usage examples.

API

The default for every optional callback parameter is:

function defaultCallback(err) {
  if (err) throw err;
}

open(path, [options], [callback])

Calls fs.open(path, "r") and reads the fd effectively the same as fromFd() would.

options may be omitted or null. The defaults are {autoClose: true, lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

autoClose is effectively equivalent to:

zipfile.once("end", function() {
  zipfile.close();
});

lazyEntries indicates that entries should be read only when readEntry() is called. If lazyEntries is false, entry events will be emitted as fast as possible to allow pipe()ing file data from all entries in parallel. This is not recommended, as it can lead to out of control memory usage for zip files with many entries. See issue #22. If lazyEntries is true, an entry or end event will be emitted in response to each call to readEntry(). This allows processing of one entry at a time, and will keep memory usage under control for zip files with many entries.

decodeStrings is the default and causes yauzl to decode strings with CP437 or UTF-8 as required by the spec. The exact effects of turning this option off are:

  • zipfile.comment, entry.fileName, and entry.fileComment will be Buffer objects instead of Strings.
  • Any Info-ZIP Unicode Path Extra Field will be ignored. See extraFields.
  • Automatic file name validation will not be performed. See validateFileName().

validateEntrySizes is the default and ensures that an entry's reported uncompressed size matches its actual uncompressed size. This check happens as early as possible, which is either before emitting each "entry" event (for entries with no compression), or during the readStream piping after calling openReadStream(). See openReadStream() for more information on defending against zip bomb attacks.

When strictFileNames is false (the default) and decodeStrings is true, all backslash (\) characters in each entry.fileName are replaced with forward slashes (/). The spec forbids file names with backslashes, but Microsoft's System.IO.Compression.ZipFile class in .NET versions 4.5.0 until 4.6.1 creates non-conformant zipfiles with backslashes in file names. strictFileNames is false by default so that clients can read these non-conformant zipfiles without knowing about this Microsoft-specific bug. When strictFileNames is true and decodeStrings is true, entries with backslashes in their file names will result in an error. See validateFileName(). When decodeStrings is false, strictFileNames has no effect.

The callback is given the arguments (err, zipfile). An err is provided if the End of Central Directory Record cannot be found, or if its metadata appears malformed. This kind of error usually indicates that this is not a zip file. Otherwise, zipfile is an instance of ZipFile.

fromFd(fd, [options], [callback])

Reads from the fd, which is presumed to be an open .zip file. Note that random access is required by the zip file specification, so the fd cannot be an open socket or any other fd that does not support random access.

options may be omitted or null. The defaults are {autoClose: false, lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

See open() for the meaning of the options and callback.

fromBuffer(buffer, [options], [callback])

Like fromFd(), but reads from a RAM buffer instead of an open file. buffer is a Buffer.

If a ZipFile is acquired from this method, it will never emit the close event, and calling close() is not necessary.

options may be omitted or null. The defaults are {lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

See open() for the meaning of the options and callback. The autoClose option is ignored for this method.

fromRandomAccessReader(reader, totalSize, [options], [callback])

This method of reading a zip file allows clients to implement their own back-end file system. For example, a client might translate read calls into network requests.

The reader parameter must be of a type that is a subclass of RandomAccessReader that implements the required methods. The totalSize is a Number and indicates the total file size of the zip file.

options may be omitted or null. The defaults are {autoClose: true, lazyEntries: false, decodeStrings: true, validateEntrySizes: true, strictFileNames: false}.

See open() for the meaning of the options and callback.

dosDateTimeToDate(date, time)

Converts MS-DOS date and time data into a JavaScript Date object. Each parameter is a Number treated as an unsigned 16-bit integer. Note that this format does not support timezones, so the returned object will use the local timezone.

validateFileName(fileName)

Returns null or a String error message depending on the validity of fileName. If fileName starts with "/" or /[A-Za-z]:\// or if it contains ".." path segments or "\\", this function returns an error message appropriate for use like this:

var errorMessage = yauzl.validateFileName(fileName);
if (errorMessage != null) throw new Error(errorMessage);

This function is automatically run for each entry, as long as decodeStrings is true. See open(), strictFileNames, and Event: "entry" for more information.

Class: ZipFile

The constructor for the class is not part of the public API. Use open(), fromFd(), fromBuffer(), or fromRandomAccessReader() instead.

Event: "entry"

Callback gets (entry), which is an Entry. See open() and readEntry() for when this event is emitted.

If decodeStrings is true, entries emitted via this event have already passed file name validation. See validateFileName() and open() for more information.

If validateEntrySizes is true and this entry's compressionMethod is 0 (stored without compression), this entry has already passed entry size validation. See open() for more information.

Event: "end"

Emitted after the last entry event has been emitted. See open() and readEntry() for more info on when this event is emitted.

Event: "close"

Emitted after the fd is actually closed. This is after calling close() (or after the end event when autoClose is true), and after all stream pipelines created from openReadStream() have finished reading data from the fd.

If this ZipFile was acquired from fromRandomAccessReader(), the "fd" in the previous paragraph refers to the RandomAccessReader implemented by the client.

If this ZipFile was acquired from fromBuffer(), this event is never emitted.

Event: "error"

Emitted in the case of errors with reading the zip file. (Note that other errors can be emitted from the streams created from openReadStream() as well.) After this event has been emitted, no further entry, end, or error events will be emitted, but the close event may still be emitted.

readEntry()

Causes this ZipFile to emit an entry or end event (or an error event). This method must only be called when this ZipFile was created with the lazyEntries option set to true (see open()). When this ZipFile was created with the lazyEntries option set to true, entry and end events are only ever emitted in response to this method call.

The event that is emitted in response to this method will not be emitted until after this method has returned, so it is safe to call this method before attaching event listeners.

After calling this method, calling this method again before the response event has been emitted will cause undefined behavior. Calling this method after the end event has been emitted will cause undefined behavior. Calling this method after calling close() will cause undefined behavior.

openReadStream(entry, [options], callback)

entry must be an Entry object from this ZipFile. callback gets (err, readStream), where readStream is a Readable Stream that provides the file data for this entry. If this zipfile is already closed (see close()), the callback will receive an err.

options may be omitted or null, and has the following defaults:

{
  decompress: entry.isCompressed() ? true : null,
  decrypt: null,
  start: 0,                  // actually the default is null, see below
  end: entry.compressedSize, // actually the default is null, see below
}

If the entry is compressed (with a supported compression method), and the decompress option is true (or omitted), the read stream provides the decompressed data. Omitting the decompress option is what most clients should do.

The decompress option must be null (or omitted) when the entry is not compressed (see isCompressed()), and either true (or omitted) or false when the entry is compressed. Specifying decompress: false for a compressed entry causes the read stream to provide the raw compressed file data without going through a zlib inflate transform.

If the entry is encrypted (see isEncrypted()), clients may want to avoid calling openReadStream() on the entry entirely. Alternatively, clients may call openReadStream() for encrypted entries and specify decrypt: false. If the entry is also compressed, clients must also specify decompress: false. Specifying decrypt: false for an encrypted entry causes the read stream to provide the raw, still-encrypted file data. (This data includes the 12-byte header described in the spec.)

The decrypt option must be null (or omitted) for non-encrypted entries, and false for encrypted entries. Omitting the decrypt option (or specifying it as null) for an encrypted entry will result in the callback receiving an err. This default behavior is so that clients not accounting for encrypted files aren't surprised by bogus file data.

The start (inclusive) and end (exclusive) options are byte offsets into this entry's file data, and can be used to obtain part of an entry's file data rather than the whole thing. If either of these options are specified and non-null, then the above options must be used to obain the file's raw data. Specifying {start: 0, end: entry.compressedSize} will result in the complete file, which is effectively the default values for these options, but note that unlike omitting the options, when you specify start or end as any non-null value, the above requirement is still enforced that you must also pass the appropriate options to get the file's raw data.

It's possible for the readStream provided to the callback to emit errors for several reasons. For example, if zlib cannot decompress the data, the zlib error will be emitted from the readStream. Two more error cases (when validateEntrySizes is true) are if the decompressed data has too many or too few actual bytes compared to the reported byte count from the entry's uncompressedSize field. yauzl notices this false information and emits an error from the readStream after some number of bytes have already been piped through the stream.

This check allows clients to trust the uncompressedSize field in Entry objects. Guarding against zip bomb attacks can be accomplished by doing some heuristic checks on the size metadata and then watching out for the above errors. Such heuristics are outside the scope of this library, but enforcing the uncompressedSize is implemented here as a security feature.

It is possible to destroy the readStream before it has piped all of its data. To do this, call readStream.destroy(). You must unpipe() the readStream from any destination before calling readStream.destroy(). If this zipfile was created using fromRandomAccessReader(), the RandomAccessReader implementation must provide readable streams that implement a .destroy() method (see randomAccessReader._readStreamForRange()) in order for calls to readStream.destroy() to work in this context.

close()

Causes all future calls to openReadStream() to fail, and closes the fd, if any, after all streams created by openReadStream() have emitted their end events.

If the autoClose option is set to true (see open()), this function will be called automatically effectively in response to this object's end event.

If the lazyEntries option is set to false (see open()) and this object's end event has not been emitted yet, this function causes undefined behavior. If the lazyEntries option is set to true, you can call this function instead of calling readEntry() to abort reading the entries of a zipfile.

It is safe to call this function multiple times; after the first call, successive calls have no effect. This includes situations where the autoClose option effectively calls this function for you.

If close() is never called, then the zipfile is "kept open". For zipfiles created with fromFd(), this will leave the fd open, which may be desirable. For zipfiles created with open(), this will leave the underlying fd open, thereby "leaking" it, which is probably undesirable. For zipfiles created with fromRandomAccessReader(), the reader's close() method will never be called. For zipfiles created with fromBuffer(), the close() function has no effect whether called or not.

Regardless of how this ZipFile was created, there are no resources other than those listed above that require cleanup from this function. This means it may be desirable to never call close() in some usecases.

isOpen

Boolean. true until close() is called; then it's false.

entryCount

Number. Total number of central directory records.

comment

String. Always decoded with CP437 per the spec.

If decodeStrings is false (see open()), this field is the undecoded Buffer instead of a decoded String.

Class: Entry

Objects of this class represent Central Directory Records. Refer to the zipfile specification for more details about these fields.

These fields are of type Number:

  • versionMadeBy
  • versionNeededToExtract
  • generalPurposeBitFlag
  • compressionMethod
  • lastModFileTime (MS-DOS format, see getLastModDateTime)
  • lastModFileDate (MS-DOS format, see getLastModDateTime)
  • crc32
  • compressedSize
  • uncompressedSize
  • fileNameLength (bytes)
  • extraFieldLength (bytes)
  • fileCommentLength (bytes)
  • internalFileAttributes
  • externalFileAttributes
  • relativeOffsetOfLocalHeader

fileName

String. Following the spec, the bytes for the file name are decoded with UTF-8 if generalPurposeBitFlag & 0x800, otherwise with CP437. Alternatively, this field may be populated from the Info-ZIP Unicode Path Extra Field (see extraFields).

This field is automatically validated by validateFileName() before yauzl emits an "entry" event. If this field would contain unsafe characters, yauzl emits an error instead of an entry.

If decodeStrings is false (see open()), this field is the undecoded Buffer instead of a decoded String. Therefore, generalPurposeBitFlag and any Info-ZIP Unicode Path Extra Field are ignored. Furthermore, no automatic file name validation is performed for this file name.

extraFields

Array with each entry in the form {id: id, data: data}, where id is a Number and data is a Buffer.

This library looks for and reads the ZIP64 Extended Information Extra Field (0x0001) in order to support ZIP64 format zip files.

This library also looks for and reads the Info-ZIP Unicode Path Extra Field (0x7075) in order to support some zipfiles that use it instead of General Purpose Bit 11 to convey UTF-8 file names. When the field is identified and verified to be reliable (see the zipfile spec), the the file name in this field is stored in the fileName property, and the file name in the central directory record for this entry is ignored. Note that when decodeStrings is false, all Info-ZIP Unicode Path Extra Fields are ignored.

None of the other fields are considered significant by this library. Fields that this library reads are left unalterned in the extraFields array.

fileComment

String decoded with the charset indicated by generalPurposeBitFlag & 0x800 as with the fileName. (The Info-ZIP Unicode Path Extra Field has no effect on the charset used for this field.)

If decodeStrings is false (see open()), this field is the undecoded Buffer instead of a decoded String.

Prior to yauzl version 2.7.0, this field was erroneously documented as comment instead of fileComment. For compatibility with any code that uses the field name comment, yauzl creates an alias field named comment which is identical to fileComment.

getLastModDate()

Effectively implemented as:

return dosDateTimeToDate(this.lastModFileDate, this.lastModFileTime);

isEncrypted()

Returns is this entry encrypted with "Traditional Encryption". Effectively implemented as:

return (this.generalPurposeBitFlag & 0x1) !== 0;

See openReadStream() for the implications of this value.

Note that "Strong Encryption" is not supported, and will result in an "error" event emitted from the ZipFile.

isCompressed()

Effectively implemented as:

return this.compressionMethod === 8;

See openReadStream() for the implications of this value.

Class: RandomAccessReader

This class is meant to be subclassed by clients and instantiated for the fromRandomAccessReader() function.

An example implementation can be found in test/test.js.

randomAccessReader._readStreamForRange(start, end)

Subclasses must implement this method.

start and end are Numbers and indicate byte offsets from the start of the file. end is exclusive, so _readStreamForRange(0x1000, 0x2000) would indicate to read 0x1000 bytes. end - start will always be at least 1.

This method should return a readable stream which will be pipe()ed into another stream. It is expected that the readable stream will provide data in several chunks if necessary. If the readable stream provides too many or too few bytes, an error will be emitted. (Note that validateEntrySizes has no effect on this check, because this is a low-level API that should behave correctly regardless of the contents of the file.) Any errors emitted on the readable stream will be handled and re-emitted on the client-visible stream (returned from zipfile.openReadStream()) or provided as the err argument to the appropriate callback (for example, for fromRandomAccessReader()).

The returned stream must implement a method .destroy() if you call readStream.destroy() on streams you get from openReadStream(). If you never call readStream.destroy(), then streams returned from this method do not need to implement a method .destroy(). .destroy() should abort any streaming that is in progress and clean up any associated resources. .destroy() will only be called after the stream has been unpipe()d from its destination.

Note that the stream returned from this method might not be the same object that is provided by openReadStream(). The stream returned from this method might be pipe()d through one or more filter streams (for example, a zlib inflate stream).

randomAccessReader.read(buffer, offset, length, position, callback)

Subclasses may implement this method. The default implementation uses createReadStream() to fill the buffer.

This method should behave like fs.read().

randomAccessReader.close(callback)

Subclasses may implement this method. The default implementation is effectively setImmediate(callback);.

callback takes parameters (err).

This method is called once the all streams returned from _readStreamForRange() have ended, and no more _readStreamForRange() or read() requests will be issued to this object.

How to Avoid Crashing

When a malformed zipfile is encountered, the default behavior is to crash (throw an exception). If you want to handle errors more gracefully than this, be sure to do the following:

  • Provide callback parameters where they are allowed, and check the err parameter.
  • Attach a listener for the error event on any ZipFile object you get from open(), fromFd(), fromBuffer(), or fromRandomAccessReader().
  • Attach a listener for the error event on any stream you get from openReadStream().

Minor version updates to yauzl will not add any additional requirements to this list.

Limitations

No Streaming Unzip API

Due to the design of the .zip file format, it's impossible to interpret a .zip file from start to finish (such as from a readable stream) without sacrificing correctness. The Central Directory, which is the authority on the contents of the .zip file, is at the end of a .zip file, not the beginning. A streaming API would need to either buffer the entire .zip file to get to the Central Directory before interpreting anything (defeating the purpose of a streaming interface), or rely on the Local File Headers which are interspersed through the .zip file. However, the Local File Headers are explicitly denounced in the spec as being unreliable copies of the Central Directory, so trusting them would be a violation of the spec.

Any library that offers a streaming unzip API must make one of the above two compromises, which makes the library either dishonest or nonconformant (usually the latter). This library insists on correctness and adherence to the spec, and so does not offer a streaming API.

Here is a way to create a spec-conformant .zip file using the zip command line program (Info-ZIP) available in most unix-like environments, that is (nearly) impossible to parse correctly with a streaming parser:

$ echo -ne '\x50\x4b\x07\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' > file.txt
$ zip -q0 - file.txt | cat > out.zip

This .zip file contains a single file entry that uses General Purpose Bit 3, which means the Local File Header doesn't know the size of the file. Any streaming parser that encounters this situation will either immediately fail, or attempt to search for the Data Descriptor after the file's contents. The file's contents is a sequence of 16-bytes crafted to exactly mimic a valid Data Descriptor for an empty file, which will fool any parser that gets this far into thinking that the file is empty rather than containing 16-bytes. What follows the file's real contents is the file's real Data Descriptor, which will likely cause some kind of signature mismatch error for a streaming parser (if one hasn't occurred already).

By using General Purpose Bit 3 (and compression method 0), it's possible to create arbitrarily ambiguous .zip files that distract parsers with file contents that contain apparently valid .zip file metadata.

Limitted ZIP64 Support

For ZIP64, only zip files smaller than 8PiB are supported, not the full 16EiB range that a 64-bit integer should be able to index. This is due to the JavaScript Number type being an IEEE 754 double precision float.

The Node.js fs module probably has this same limitation.

ZIP64 Extensible Data Sector Is Ignored

The spec does not allow zip file creators to put arbitrary data here, but rather reserves its use for PKWARE and mentions something about Z390. This doesn't seem useful to expose in this library, so it is ignored.

No Multi-Disk Archive Support

This library does not support multi-disk zip files. The multi-disk fields in the zipfile spec were intended for a zip file to span multiple floppy disks, which probably never happens now. If the "number of this disk" field in the End of Central Directory Record is not 0, the open(), fromFd(), fromBuffer(), or fromRandomAccessReader() callback will receive an err. By extension the following zip file fields are ignored by this library and not provided to clients:

  • Disk where central directory starts
  • Number of central directory records on this disk
  • Disk number where file starts

Limited Encryption Handling

You can detect when a file entry is encrypted with "Traditional Encryption" via isEncrypted(), but yauzl will not help you decrypt it. See openReadStream().

If a zip file contains file entries encrypted with "Strong Encryption", yauzl emits an error.

If the central directory is encrypted or compressed, yauzl emits an error.

Local File Headers Are Ignored

Many unzip libraries mistakenly read the Local File Header data in zip files. This data is officially defined to be redundant with the Central Directory information, and is not to be trusted. Aside from checking the signature, yauzl ignores the content of the Local File Header.

No CRC-32 Checking

This library provides the crc32 field of Entry objects read from the Central Directory. However, this field is not used for anything in this library.

versionNeededToExtract Is Ignored

The field versionNeededToExtract is ignored, because this library doesn't support the complete zip file spec at any version,

No Support For Obscure Compression Methods

Regarding the compressionMethod field of Entry objects, only method 0 (stored with no compression) and method 8 (deflated) are supported. Any of the other 15 official methods will cause the openReadStream() callback to receive an err.

Data Descriptors Are Ignored

There may or may not be Data Descriptor sections in a zip file. This library provides no support for finding or interpreting them.

Archive Extra Data Record Is Ignored

There may or may not be an Archive Extra Data Record section in a zip file. This library provides no support for finding or interpreting it.

No Language Encoding Flag Support

Zip files officially support charset encodings other than CP437 and UTF-8, but the zip file spec does not specify how it works. This library makes no attempt to interpret the Language Encoding Flag.

Change History

  • 2.10.0
    • Added support for non-conformant zipfiles created by Microsoft, and added option strictFileNames to disable the workaround. issue #66, issue #88
  • 2.9.2
  • 2.9.1
    • Removed console.log() accidentally introduced in 2.9.0. issue #64
  • 2.9.0
    • Throw an exception if readEntry() is called without lazyEntries:true. Previously this caused undefined behavior. issue #63
  • 2.8.0
    • Added option validateEntrySizes. issue #53
    • Added examples/promises.js
    • Added ability to read raw file data via decompress and decrypt options. issue #11, issue #38, pull #39
    • Added start and end options to openReadStream(). issue #38
  • 2.7.0
    • Added option decodeStrings. issue #42
    • Fixed documentation for entry.fileComment and added compatibility alias. issue #47
  • 2.6.0
    • Support Info-ZIP Unicode Path Extra Field, used by WinRAR for Chinese file names. issue #33
  • 2.5.0
    • Ignore malformed Extra Field that is common in Android .apk files. issue #31
  • 2.4.3
    • Fix crash when parsing malformed Extra Field buffers. issue #31
  • 2.4.2
    • Remove .npmignore and .travis.yml from npm package.
  • 2.4.1
    • Fix error handling.
  • 2.4.0
  • 2.3.1
    • Documentation updates.
  • 2.3.0
    • Check that uncompressedSize is correct, or else emit an error. issue #13
  • 2.2.1
    • Update dependencies.
  • 2.2.0
    • Update dependencies.
  • 2.1.0
    • Remove dependency on iconv.
  • 2.0.3
    • Fix crash when trying to read a 0-byte file.
  • 2.0.2
    • Fix event behavior after errors.
  • 2.0.1
    • Fix bug with using iconv.
  • 2.0.0
    • Initial release.

Author: Thejoshwolfe
Source Code: https://github.com/thejoshwolfe/yauzl
License: MIT License

#node #library 

Hire Dedicated Node.js Developers - Hire Node.js Developers

If you look at the backend technology used by today’s most popular apps there is one thing you would find common among them and that is the use of NodeJS Framework. Yes, the NodeJS framework is that effective and successful.

If you wish to have a strong backend for efficient app performance then have NodeJS at the backend.

WebClues Infotech offers different levels of experienced and expert professionals for your app development needs. So hire a dedicated NodeJS developer from WebClues Infotech with your experience requirement and expertise.

So what are you waiting for? Get your app developed with strong performance parameters from WebClues Infotech

For inquiry click here: https://www.webcluesinfotech.com/hire-nodejs-developer/

Book Free Interview: https://bit.ly/3dDShFg

#hire dedicated node.js developers #hire node.js developers #hire top dedicated node.js developers #hire node.js developers in usa & india #hire node js development company #hire the best node.js developers & programmers

Oral  Brekke

Oral Brekke

1646117100

Yazl: Yet another Zip Library for Node

yazl

yet another zip library for node. For unzipping, see yauzl.

Design principles:

  • Don't block the JavaScript thread. Use and provide async APIs.
  • Keep memory usage under control. Don't attempt to buffer entire files in RAM at once.
  • Prefer to open input files one at a time than all at once. This is slightly suboptimal for time performance, but avoids OS-imposed limits on the number of simultaneously open file handles.

Usage

var yazl = require("yazl");

var zipfile = new yazl.ZipFile();
zipfile.addFile("file1.txt", "file1.txt");
// (add only files, not directories)
zipfile.addFile("path/to/file.txt", "path/in/zipfile.txt");
// pipe() can be called any time after the constructor
zipfile.outputStream.pipe(fs.createWriteStream("output.zip")).on("close", function() {
  console.log("done");
});
// alternate apis for adding files:
zipfile.addReadStream(process.stdin, "stdin.txt");
zipfile.addBuffer(Buffer.from("hello"), "hello.txt");
// call end() after all the files have been added
zipfile.end();

API

Class: ZipFile

new ZipFile()

No parameters. Nothing can go wrong.

addFile(realPath, metadataPath, [options])

Adds a file from the file system at realPath into the zipfile as metadataPath. Typically metadataPath would be calculated as path.relative(root, realPath). Unzip programs would extract the file from the zipfile as metadataPath. realPath is not stored in the zipfile.

A valid metadataPath must not be blank. If a metadataPath contains "\\" characters, they will be replaced by "/" characters. After this substitution, a valid metadataPath must not start with "/" or /[A-Za-z]:\//, and must not contain ".." path segments. File paths must not end with "/", but see addEmptyDirectory(). After UTF-8 encoding, metadataPath must be at most 0xffff bytes in length.

options may be omitted or null and has the following structure and default values:

{
  mtime: stats.mtime,
  mode: stats.mode,
  compress: true,
  forceZip64Format: false,
  fileComment: "", // or a UTF-8 Buffer
}

Use mtime and/or mode to override the values that would normally be obtained by the fs.Stats for the realPath. The mode is the unix permission bits and file type. The mtime and mode are stored in the zip file in the fields "last mod file time", "last mod file date", and "external file attributes". yazl does not store group and user ids in the zip file.

If compress is true, the file data will be deflated (compression method 8). If compress is false, the file data will be stored (compression method 0).

If forceZip64Format is true, yazl will use ZIP64 format in this entry's Data Descriptor and Central Directory Record regardless of if it's required or not (this may be useful for testing.). Otherwise, yazl will use ZIP64 format where necessary.

If fileComment is a string, it will be encoded with UTF-8. If fileComment is a Buffer, it should be a UTF-8 encoded string. In UTF-8, fileComment must be at most 0xffff bytes in length. This becomes the "file comment" field in this entry's central directory file header.

Internally, fs.stat() is called immediately in the addFile function, and fs.createReadStream() is used later when the file data is actually required. Throughout adding and encoding n files with addFile(), the number of simultaneous open files is O(1), probably just 1 at a time.

addReadStream(readStream, metadataPath, [options])

Adds a file to the zip file whose content is read from readStream. See addFile() for info about the metadataPath parameter. options may be omitted or null and has the following structure and default values:

{
  mtime: new Date(),
  mode: 0o100664,
  compress: true,
  forceZip64Format: false,
  fileComment: "", // or a UTF-8 Buffer
  size: 12345, // example value
}

See addFile() for the meaning of mtime, mode, compress, forceZip64Format, and fileComment. If size is given, it will be checked against the actual number of bytes in the readStream, and an error will be emitted if there is a mismatch.

Note that yazl will .pipe() data from readStream, so be careful using .on('data'). In certain versions of node, .on('data') makes .pipe() behave incorrectly.

addBuffer(buffer, metadataPath, [options])

Adds a file to the zip file whose content is buffer. See below for info on the limitations on the size of buffer. See addFile() for info about the metadataPath parameter. options may be omitted or null and has the following structure and default values:

{
  mtime: new Date(),
  mode: 0o100664,
  compress: true,
  forceZip64Format: false,
  fileComment: "", // or a UTF-8 Buffer
}

See addFile() for the meaning of mtime, mode, compress, forceZip64Format, and fileComment.

This method has the unique property that General Purpose Bit 3 will not be used in the Local File Header. This doesn't matter for unzip implementations that conform to the Zip File Spec. However, 7-Zip 9.20 has a known bug where General Purpose Bit 3 is declared an unsupported compression method (note that it really has nothing to do with the compression method.). See issue #11. If you would like to create zip files that 7-Zip 9.20 can understand, you must use addBuffer() instead of addFile() or addReadStream() for all entries in the zip file (and addEmptyDirectory() is fine too).

Note that even when yazl provides the file sizes in the Local File Header, yazl never uses ZIP64 format for Local File Headers due to the size limit on buffer (see below).

Size limitation on buffer

In order to require the ZIP64 format for a local file header, the provided buffer parameter would need to exceed 0xfffffffe in length. Alternatively, the buffer parameter might not exceed 0xfffffffe in length, but zlib compression fails to compress the buffer and actually inflates the data to more than 0xfffffffe in length. Both of these scenarios are not allowed by yazl, and those are enforced by a size limit on the buffer parameter.

According to this zlib documentation, the worst case compression results in "an expansion of at most 13.5%, plus eleven bytes". Furthermore, some configurations of Node.js impose a size limit of 0x3fffffff on every Buffer object. Running this size through the worst case compression of zlib still produces a size less than 0xfffffffe bytes,

Therefore, yazl enforces that the provided buffer parameter must be at most 0x3fffffff bytes long.

addEmptyDirectory(metadataPath, [options])

Adds an entry to the zip file that indicates a directory should be created, even if no other items in the zip file are contained in the directory. This method is only required if the zip file is intended to contain an empty directory.

See addFile() for info about the metadataPath parameter. If metadataPath does not end with a "/", a "/" will be appended.

options may be omitted or null and has the following structure and default values:

{
  mtime: new Date(),
  mode: 040775,
}

See addFile() for the meaning of mtime and mode.

end([options], [finalSizeCallback])

Indicates that no more files will be added via addFile(), addReadStream(), or addBuffer(), and causes the eventual close of outputStream.

options may be omitted or null and has the following structure and default values:

{
  forceZip64Format: false,
  comment: "", // or a CP437 Buffer
}

If forceZip64Format is true, yazl will include the ZIP64 End of Central Directory Locator and ZIP64 End of Central Directory Record regardless of whether or not they are required (this may be useful for testing.). Otherwise, yazl will include these structures if necessary.

If comment is a string, it will be encoded with CP437. If comment is a Buffer, it should be a CP437 encoded string. comment must be at most 0xffff bytes in length and must not include the byte sequence [0x50,0x4b,0x05,0x06]. This becomes the ".ZIP file comment" field in the end of central directory record. Note that in practice, most zipfile readers interpret this field in UTF-8 instead of CP437. If your string uses only codepoints in the range 0x20...0x7e (printable ASCII, no whitespace except for sinlge space ' '), then UTF-8 and CP437 (and ASCII) encodings are all identical. This restriction is recommended for maxium compatibility. To use UTF-8 encoding at your own risk, pass a Buffer into this function; it will not be validated.

If specified and non-null, finalSizeCallback is given the parameters (finalSize) sometime during or after the call to end(). finalSize is of type Number and can either be -1 or the guaranteed eventual size in bytes of the output data that can be read from outputStream.

Note that finalSizeCallback is usually called well before outputStream has piped all its data; this callback does not mean that the stream is done.

If finalSize is -1, it means means the final size is too hard to guess before processing the input file data. This will happen if and only if the compress option is true on any call to addFile(), addReadStream(), or addBuffer(), or if addReadStream() is called and the optional size option is not given. In other words, clients should know whether they're going to get a -1 or a real value by looking at how they are using this library.

The call to finalSizeCallback might be delayed if yazl is still waiting for fs.Stats for an addFile() entry. If addFile() was never called, finalSizeCallback will be called during the call to end(). It is not required to start piping data from outputStream before finalSizeCallback is called. finalSizeCallback will be called only once, and only if this is the first call to end().

outputStream

A readable stream that will produce the contents of the zip file. It is typical to pipe this stream to a writable stream created from fs.createWriteStream().

Internally, large amounts of file data are piped to outputStream using pipe(), which means throttling happens appropriately when this stream is piped to a slow destination.

Data becomes available in this stream soon after calling one of addFile(), addReadStream(), or addBuffer(). Clients can call pipe() on this stream at any time, such as immediately after getting a new ZipFile instance, or long after calling end().

This stream will remain open while you add entries until you end() the zip file.

As a reminder, be careful using both .on('data') and .pipe() with this stream. In certain versions of node, you cannot use both .on('data') and .pipe() successfully.

dateToDosDateTime(jsDate)

jsDate is a Date instance. Returns {date: date, time: time}, where date and time are unsigned 16-bit integers.

Regarding ZIP64 Support

yazl automatically uses ZIP64 format to support files and archives over 2^32 - 2 bytes (~4GB) in size and to support archives with more than 2^16 - 2 (65534) files. (See the forceZip64Format option in the API above for more control over this behavior.) ZIP64 format is necessary to exceed the limits inherent in the original zip file format.

ZIP64 format is supported by most popular zipfile readers, but not by all of them. Notably, the Mac Archive Utility does not understand ZIP64 format (as of writing this), and will behave very strangely when presented with such an archive.

Output Structure

The Zip File Spec leaves a lot of flexibility up to the zip file creator. This section explains and justifies yazl's interpretation and decisions regarding this flexibility.

This section is probably not useful to yazl clients, but may be interesting to unzip implementors and zip file enthusiasts.

Disk Numbers

All values related to disk numbers are 0, because yazl has no multi-disk archive support. (The exception being the Total Number of Disks field in the ZIP64 End of Central Directory Locator, which is always 1.)

Version Made By

Always 0x033f == (3 << 8) | 63, which means UNIX (3) and made from the spec version 6.3 (63).

Note that the "UNIX" has implications in the External File Attributes.

Version Needed to Extract

Usually 20, meaning 2.0. This allows filenames and file comments to be UTF-8 encoded.

When ZIP64 format is used, some of the Version Needed to Extract values will be 45, meaning 4.5. When this happens, there may be a mix of 20 and 45 values throughout the zipfile.

General Purpose Bit Flag

Bit 11 is always set. Filenames (and file comments) are always encoded in UTF-8, even if the result is indistinguishable from ascii.

Bit 3 is usually set in the Local File Header. To support both a streaming input and streaming output api, it is impossible to know the crc32 before processing the file data. When bit 3 is set, data Descriptors are given after each file data with this information, as per the spec. But remember a complete metadata listing is still always available in the central directory record, so if unzip implementations are relying on that, like they should, none of this paragraph will matter anyway. Even so, some popular unzip implementations do not follow the spec. The Mac Archive Utility requires Data Descriptors to include the optional signature, so yazl includes the optional data descriptor signature. When bit 3 is not used, the Mac Archive Utility requires there to be no data descriptor, so yazl skips it in that case. Additionally, 7-Zip 9.20 does not seem to support bit 3 at all (see issue #11).

All other bits are unset.

Internal File Attributes

Always 0. The "apparently an ASCII or text file" bit is always unset meaning "apparently binary". This kind of determination is outside the scope of yazl, and is probably not significant in any modern unzip implementation.

External File Attributes

Always stats.mode << 16. This is apparently the convention for "version made by" = 0x03xx (UNIX).

Note that for directory entries (see addEmptyDirectory()), it is conventional to use the lower 8 bits for the MS-DOS directory attribute byte. However, the spec says this is only required if the Version Made By is DOS, so this library does not do that.

Directory Entries

When adding a metadataPath such as "parent/file.txt", yazl does not add a directory entry for "parent/", because file entries imply the need for their parent directories. Unzip clients seem to respect this style of pathing, and the zip file spec does not specify what is standard in this regard.

In order to create empty directories, use addEmptyDirectory().

Size of Local File and Central Directory Entry Metadata

The spec recommends that "The combined length of any directory record and [the file name, extra field, and comment fields] should not generally exceed 65,535 bytes". yazl makes no attempt to respect this recommendation. Instead, each of the fields is limited to 65,535 bytes due to the length of each being encoded as an unsigned 16 bit integer.

Change History

  • 2.5.1
    • Fix support for old versions of Node and add official support for Node versions 0.10, 4, 6, 8, 10. pull #49
  • 2.5.0
  • 2.4.3
  • 2.4.2
    • Remove octal literals to make yazl compatible with strict mode. pull #28
  • 2.4.1
    • Fix Mac Archive Utility compatibility issue. issue #24
  • 2.4.0
  • 2.3.1
    • Remove .npmignore from npm package. pull #22
  • 2.3.0
    • metadataPath can have \ characters now; they will be replaced with /. issue #18
  • 2.2.2
  • 2.2.1
    • Fix Mac Archive Utility compatibility issue. issue #14
  • 2.2.0
    • Avoid using general purpose bit 3 for addBuffer() calls. issue #13
  • 2.1.3
    • Fix bug when only addBuffer() and end() are called. issue #12
  • 2.1.2
  • 2.1.1
    • Fixed stack overflow when using addBuffer() in certain ways. issue #9
  • 2.1.0
    • Added addEmptyDirectory().
    • options is now optional for addReadStream() and addBuffer().
  • 2.0.0
    • Initial release.

Author: Thejoshwolfe
Source Code: https://github.com/thejoshwolfe/yazl 
License: MIT License

#node #library 

Aria Barnes

Aria Barnes

1622719015

Why use Node.js for Web Development? Benefits and Examples of Apps

Front-end web development has been overwhelmed by JavaScript highlights for quite a long time. Google, Facebook, Wikipedia, and most of all online pages use JS for customer side activities. As of late, it additionally made a shift to cross-platform mobile development as a main technology in React Native, Nativescript, Apache Cordova, and other crossover devices. 

Throughout the most recent couple of years, Node.js moved to backend development as well. Designers need to utilize a similar tech stack for the whole web project without learning another language for server-side development. Node.js is a device that adjusts JS usefulness and syntax to the backend. 

What is Node.js? 

Node.js isn’t a language, or library, or system. It’s a runtime situation: commonly JavaScript needs a program to work, however Node.js makes appropriate settings for JS to run outside of the program. It’s based on a JavaScript V8 motor that can run in Chrome, different programs, or independently. 

The extent of V8 is to change JS program situated code into machine code — so JS turns into a broadly useful language and can be perceived by servers. This is one of the advantages of utilizing Node.js in web application development: it expands the usefulness of JavaScript, permitting designers to coordinate the language with APIs, different languages, and outside libraries.

What Are the Advantages of Node.js Web Application Development? 

Of late, organizations have been effectively changing from their backend tech stacks to Node.js. LinkedIn picked Node.js over Ruby on Rails since it took care of expanding responsibility better and decreased the quantity of servers by multiple times. PayPal and Netflix did something comparative, just they had a goal to change their design to microservices. We should investigate the motivations to pick Node.JS for web application development and when we are planning to hire node js developers. 

Amazing Tech Stack for Web Development 

The principal thing that makes Node.js a go-to environment for web development is its JavaScript legacy. It’s the most well known language right now with a great many free devices and a functioning local area. Node.js, because of its association with JS, immediately rose in ubiquity — presently it has in excess of 368 million downloads and a great many free tools in the bundle module. 

Alongside prevalence, Node.js additionally acquired the fundamental JS benefits: 

  • quick execution and information preparing; 
  • exceptionally reusable code; 
  • the code is not difficult to learn, compose, read, and keep up; 
  • tremendous asset library, a huge number of free aides, and a functioning local area. 

In addition, it’s a piece of a well known MEAN tech stack (the blend of MongoDB, Express.js, Angular, and Node.js — four tools that handle all vital parts of web application development). 

Designers Can Utilize JavaScript for the Whole Undertaking 

This is perhaps the most clear advantage of Node.js web application development. JavaScript is an unquestionable requirement for web development. Regardless of whether you construct a multi-page or single-page application, you need to know JS well. On the off chance that you are now OK with JavaScript, learning Node.js won’t be an issue. Grammar, fundamental usefulness, primary standards — every one of these things are comparable. 

In the event that you have JS designers in your group, it will be simpler for them to learn JS-based Node than a totally new dialect. What’s more, the front-end and back-end codebase will be basically the same, simple to peruse, and keep up — in light of the fact that they are both JS-based. 

A Quick Environment for Microservice Development 

There’s another motivation behind why Node.js got famous so rapidly. The environment suits well the idea of microservice development (spilling stone monument usefulness into handfuls or many more modest administrations). 

Microservices need to speak with one another rapidly — and Node.js is probably the quickest device in information handling. Among the fundamental Node.js benefits for programming development are its non-obstructing algorithms.

Node.js measures a few demands all at once without trusting that the first will be concluded. Many microservices can send messages to one another, and they will be gotten and addressed all the while. 

Versatile Web Application Development 

Node.js was worked in view of adaptability — its name really says it. The environment permits numerous hubs to run all the while and speak with one another. Here’s the reason Node.js adaptability is better than other web backend development arrangements. 

Node.js has a module that is liable for load adjusting for each running CPU center. This is one of numerous Node.js module benefits: you can run various hubs all at once, and the environment will naturally adjust the responsibility. 

Node.js permits even apportioning: you can part your application into various situations. You show various forms of the application to different clients, in light of their age, interests, area, language, and so on. This builds personalization and diminishes responsibility. Hub accomplishes this with kid measures — tasks that rapidly speak with one another and share a similar root. 

What’s more, Node’s non-hindering solicitation handling framework adds to fast, letting applications measure a great many solicitations. 

Control Stream Highlights

Numerous designers consider nonconcurrent to be one of the two impediments and benefits of Node.js web application development. In Node, at whatever point the capacity is executed, the code consequently sends a callback. As the quantity of capacities develops, so does the number of callbacks — and you end up in a circumstance known as the callback damnation. 

In any case, Node.js offers an exit plan. You can utilize systems that will plan capacities and sort through callbacks. Systems will associate comparable capacities consequently — so you can track down an essential component via search or in an envelope. At that point, there’s no compelling reason to look through callbacks.

 

Final Words

So, these are some of the top benefits of Nodejs in web application development. This is how Nodejs is contributing a lot to the field of web application development. 

I hope now you are totally aware of the whole process of how Nodejs is really important for your web project. If you are looking to hire a node js development company in India then I would suggest that you take a little consultancy too whenever you call. 

Good Luck!

Original Source

#node.js development company in india #node js development company #hire node js developers #hire node.js developers in india #node.js development services #node.js development

The  NineHertz

The NineHertz

1611828639

Node JS Development Company | Hire Node.js Developers

The NineHertz promises to develop a pro-active and easy solution for your enterprise. It has reached the heights in Node js web development and is considered as one of the top-notch Node js development company across the globe.

The NineHertz aims to design a best Node js development solution to improve their branding status and business profit.

Looking to hire the leading Node js development company?

#node js development company #nodejs development company #node.js development company #node.js development companies #node js web development #node development company