ParquetReader<T>

A typed Parquet reader, wrapping the base Parquet reader.

Extends

• ParquetReader

Type Parameters

Type Parameter
T extends ParquetRecordLike

Implements

• AsyncDisposable

Constructors

new ParquetReader()

new ParquetReader<T>(
   metadata, 
   envelopeReader, 
opts?): ParquetReader<T>

Create a new parquet reader from the file metadata and an envelope reader. It is not recommended to call this constructor directly except for advanced and internal use cases. Consider using one of the open methods instead

Parameters

Parameter	Type
metadata	FileMetaDataExt
envelopeReader	ParquetEnvelopeReader
opts?	BufferReaderOptions

Returns

ParquetReader<T>

Inherited from

BaseParquetReader.constructor

Properties

envelopeReader

envelopeReader: null | ParquetEnvelopeReader;

Inherited from

BaseParquetReader.envelopeReader

---

metadata

metadata: null | FileMetaDataExt;

Inherited from

BaseParquetReader.metadata

---

schema

schema: ParquetSchema<T>;

Overrides

BaseParquetReader.schema

Methods

[asyncDispose]()

asyncDispose: Promise<void>

Returns

Promise<void>

Implementation of

AsyncDisposable.[asyncDispose]

---

[asyncIterator]()

asyncIterator: AsyncGenerator<T, void, unknown>

Returns

AsyncGenerator<T, void, unknown>

Overrides

BaseParquetReader.[asyncIterator]

---

close()

close(): Promise<void>

Close this parquet reader. You MUST call this method once you're finished reading rows

Returns

Promise<void>

Inherited from

BaseParquetReader.close

---

decodePages()

decodePages(buffer, opts): Promise<PageData>

Parameters

Parameter	Type
buffer	Buffer
opts	Options

Returns

Promise<PageData>

Inherited from

BaseParquetReader.decodePages

---

dispose()

dispose(): Promise<void>

Returns

Promise<void>

---

exportMetadata()

exportMetadata(indent): Promise<string>

Parameters

Parameter	Type
indent	undefined | string | number

Returns

Promise<string>

Inherited from

BaseParquetReader.exportMetadata

---

getBloomFiltersFor()

getBloomFiltersFor(columnNames): Promise<Record<string, object[]>>

Parameters

Parameter	Type
columnNames	string[]

Returns

Promise<Record<string, object[]>>

Inherited from

BaseParquetReader.getBloomFiltersFor

---

getCursor()

getCursor(columnList?): ParquetCursor

Return a cursor to the file. You may open more than one cursor and use them concurrently. All cursors become invalid once close() is called on the reader object.

The required_columns parameter controls which columns are actually read from disk. An empty array or no value implies all columns. A list of column names means that only those columns should be loaded from disk.

Parameters

Parameter	Type
columnList?	unknown[][]

Returns

ParquetCursor

Inherited from

BaseParquetReader.getCursor

---

getMetadata()

getMetadata(): Record<string, unknown>

Returns the user (key/value) metadata for this file

Returns

Record<string, unknown>

Inherited from

BaseParquetReader.getMetadata

---

getRowCount()

getRowCount(): Int64

Return the number of rows in this file. Note that the number of rows is not necessarily equal to the number of rows in each column.

Returns

Int64

Inherited from

BaseParquetReader.getRowCount

---

getSchema()

getSchema(): ParquetSchema

Returns the ParquetSchema for this file

Returns

ParquetSchema

Inherited from

BaseParquetReader.getSchema

---

openBuffer()

static openBuffer<T>(buffer, options?): Promise<ParquetReader<T>>

Type Parameters

Type Parameter
T extends ParquetRecordLike

Parameters

Parameter	Type
buffer	Buffer
options?	BufferReaderOptions

Returns

Promise<ParquetReader<T>>

Overrides

BaseParquetReader.openBuffer

---

openEnvelopeReader()

static openEnvelopeReader<T>(envelopeReader, opts?): Promise<ParquetReader<T>>

Type Parameters

Type Parameter
T extends ParquetRecordLike

Parameters

Parameter	Type
envelopeReader	ParquetEnvelopeReader
opts?	BufferReaderOptions

Returns

Promise<ParquetReader<T>>

Overrides

BaseParquetReader.openEnvelopeReader

---

openFile()

static openFile<T>(filePath, options?): Promise<ParquetReader<T>>

Open the parquet file pointed to by the specified path and return a new parquet reader

Type Parameters

Type Parameter
T extends ParquetRecordLike

Parameters

Parameter	Type
filePath	PathBuilderLike
options?	BufferReaderOptions

Returns

Promise<ParquetReader<T>>

Overrides

BaseParquetReader.openFile

---

openS3()

static openS3(
   client, 
   params, 
options?): Promise<ParquetReader>

Open the parquet file from S3 using the supplied aws client [, commands] and params The params have to include Bucket and Key to the file requested, If using v3 of the AWS SDK, combine the client and commands into an object wiht keys matching the original module names, and do not instantiate the commands; pass them as classes/modules.

This function returns a new parquet reader [ or throws an Error.]

Parameters

Parameter	Type
client	any
params	ClientParameters
options?	BufferReaderOptions

Returns

Promise<ParquetReader>

Inherited from

BaseParquetReader.openS3

---

openUrl()

static openUrl(params, options?): Promise<ParquetReader>

Open the parquet file from a url using the supplied request module params should either be a string (url) or an object that includes a url property. This function returns a new parquet reader

Parameters

Parameter	Type
params	string | URL | Parameter
options?	BufferReaderOptions

Returns

Promise<ParquetReader>

Inherited from

BaseParquetReader.openUrl