Repository navigation
Cannot parse from a stream in the browser #443
Description
Describe the bug
I want to allow a user to upload a gzipped csv file and parse it in a stream (using pipeThrough / pipeTo). This works in Node, but not in the browser, where I get:
file-upload.html:52 Uncaught (in promise) TypeError: Failed to execute 'pipeTo' on 'ReadableStream': parameter 1 is not of type 'WritableStream'.
at HTMLInputElement.<anonymous> (file-upload.html:52:18)
Line 52 is the pipeTo line from the example below, so it seems to think that the parse object is not a WritableStream.
To Reproduce
Dump the code below into a file, open it in a browser and then try uploading a .tsv.gz file. I tested in Chrome and Firefox. The same code works in Node, although you have to swap pipeThrough and pipeTo for pipe. I'll admit that I find the differences between Node and the browser for streaming a bit confusing so I may have made some obvious error here.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>File Upload</title>
</head>
<body>
<div class="upload-container">
<form class="upload-form">
<input type="file" id="fileInput" name="file">
</form>
</div>
<script type="module">
import { parse } from 'https://cdn.jsdelivr.net/npm/csv-parse@5.5.6/+esm'
document.querySelector('#fileInput').addEventListener('change', (e) => {
e.preventDefault();
const file = document.getElementById('fileInput').files[0];
// turn the file into a stream
const fileStream = file.stream()
// create a gunzip transform stream
const unzipper = new DecompressionStream('gzip')
const records = [];
// create a csv parser writeable stream
const parser = parse({
delimiter: '\t'
});
parser.on('readable', function(){
let record;
while ((record = parser.read()) !== null) {
records.push(record);
}
});
parser.on('error', function(err){
console.error(err.message);
});
parser.on('end', function(){
console.log(records)
})
console.log("piping!")
// pipe the file stream through the gunzip stream and into the csv parse stream
fileStream
.pipeThrough(unzipper)
.pipeTo(parser)
});
</script>
</body>
</html>
** Additional Context **
I wonder if this issue is related, although I'm not trying to parse the complex CSVs listed in that issue.
I also have a similar issue raised for Papaparse.
Activity
wdavidw commentedon Nov 16, 2024
You need to reference the browser ESM or IIFE distributions.
wdavidw commentedon Nov 16, 2024
you can also look at the demo folders.
psaffrey-biomodal commentedon Nov 21, 2024
Thanks very much for the rapid response. Unfortunately, this doesn't seem to fix the error. Here's what I did:
node-csvgit repository and ran the example in thedemo/browserdirectorymyparser.htmlfile with the code I linked above in the same directory so that it's hosted by Expressimport { parse } from '/lib/esm/parse/index.js';But I still get the same error:
Once I've figured out the underlying problem here, I'd also like to use a CDN if possible. Would you be open to a PR on the documentation suggesting how a CDN can be used?
wdavidw commentedon Nov 21, 2024
I noticed your URL referencing
https://cdn.jsdelivr.net/npm/csv-parse@5.5.6/+esm. I have no idea how/by who this was set. Please update the documentation to provide guidance.I just tried the following commands successfully.
psaffrey-biomodal commentedon Nov 21, 2024
The URL is a CDN reference - I put that in there because I want the page I'm creating to be standalone, rather than hosted alongside some node modules.
It's not that I can't import the module at all - the example you've sent works for me too. The problem is that I can't use it in a stream. So if you do the same set of commands, but instead create this code in the
<script>tag:Then it gives the error:
I can also use the
syncversion (by replacing the import withimport { parse } from '/node_modules/csv-parse/dist/esm/sync.js';) and then I get this error: