Skip to content

Possible data loss when importing into DynamoDB! #139

Description

@Dunedan

While trying to use the cli to import data into DynamoDB I wondered why no data appeared in DynamoDB.
After some digging I noticed that aggregator._flush isn't called when piping data into dyno. That causes up to 25 records not to be imported into DynamoDB and happens for put as well as for import!
In my case my test data had less than 25 records, that's why I noticed it.

Tested on Debian/unstable with NodeJS 7.10.0 & 8.1.3.

The internet seems to suggest (nodejs/node#53) that a .push(null) at the end of the stream is necessary, but as I'm not really familiar with NodeJS I don't know if that applies to reading from stdin as well.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions