I’m continuing with my notes on transferring big files from and to AWS S3 with node.js If you are reading a file from a S3 bucket using a stream that you occasionally pause, mind that the read stream will be closed in 60 minutes. If you cannot handle the file in that period of time,… Continue reading AWS Closes S3 Read Stream Unexpectedly
Tag: aws
Node.js Streams and why sometimes they don’t pause()
TL; DR: If you pipe your node.js streams, make sure you pause the last one in the chain. The longer story: We’re building a node.js application that ingests data from multiple data sources for a client of ours. Since they are quite big in size and in user base (we’re going to process data for… Continue reading Node.js Streams and why sometimes they don’t pause()