Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process doesn't close after scraping and DB upload ended #248

Closed
theDavidBarton opened this issue Oct 28, 2019 · 3 comments · Fixed by #279
Closed

process doesn't close after scraping and DB upload ended #248

theDavidBarton opened this issue Oct 28, 2019 · 3 comments · Fixed by #279
Assignees
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@theDavidBarton
Copy link
Owner

since #236 - possibbly - we don't close mongoDB connection in the end of the process scrapeDailyMenu.js

it should be investigated what is the issue.

@theDavidBarton theDavidBarton added the bug Something isn't working label Oct 28, 2019
@theDavidBarton theDavidBarton self-assigned this Oct 28, 2019
@theDavidBarton
Copy link
Owner Author

theDavidBarton commented Oct 31, 2019

once I've got this after connection was not closed, but this happened with an older version of mongoInsertMany plus a new npm version of mongoDB. so it may be a case of not compatible versions. but anyway, it may help in debugging:

events.js:174
      throw er; // Unhandled 'error' event
      ^

Error: read ECONNRESET
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:111:27)
Emitted 'error' event at:
    at TLSSocket.<anonymous> ([...]\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\connection.js:321:10)
    at Object.onceWrapper (events.js:277:13)
    at TLSSocket.emit (events.js:189:13)
    at emitErrorNT (internal/streams/destroy.js:82:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:50:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)

@theDavidBarton
Copy link
Owner Author

this one is getting higher priority since we are running the main script everday on schedule (GitHub actions scrape.yml)

debug results:

checked the processes with why-is-node-running

# TLSWRAP
(unknown stack trace)

# TLSWRAP
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\connect.js:266 - socket = tls.connect(parseSslOptions(family,
options));
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\connect.js:35  - return makeConnection(6, options, (err, ipv6S
ocket) => {
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\pool.js:571    - connect(self.options, (err, connection) => {
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\sdam\server.js:153        - this.s.pool.connect();
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\sdam\topology.js:875      - server.connect();
# TLSWRAP
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\connect.js:266 - socket = tls.connect(parseSslOptions(family,
options));
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\connect.js:37  - makeConnection(4, options, (err, ipv4Socket)
=> {
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\connect.js:261 - _callback(err, ret);
C:\Users\david.barton\workspace_repos\puppeteer-daily-menu-scraper\node_modules\mongodb\lib\core\connection\connect.js:286 - callback(connectionFailureError(eventName, er
r), eventName);

full log: why-is-node-running_20191121.txt

@theDavidBarton theDavidBarton added the help wanted Extra attention is needed label Nov 21, 2019
@theDavidBarton theDavidBarton pinned this issue Nov 21, 2019
@theDavidBarton
Copy link
Owner Author

they say mongodb 3.3.4 solves the issue: https://github.com/mongodb/node-mongodb-native/releases/tag/v3.3.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant