Missing access token parameter

[mferrier]

Randomly getting CRITICAL (missing_token) Missing access token parameter. on subsequent runs with the same config. Creating an issue in case anyone else notices the same. Please let me know if this isn't the right way to handle this issue.

(These runs are via stitchdata.com).

Full log of errored run:

2021-01-11 13:46:32,740Z   main - INFO Running tap-asana version 2.1.1 and target-stitch version 3.2.0
2021-01-11 13:46:32,881Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2021-01-11 13:46:32,885Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2021-01-11 13:46:32,885Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.streamRecordCount ), use_ssl( true )
2021-01-11 13:46:33,893Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [44] at offset None
2021-01-11 13:46:33,896Z   main - INFO Starting tap to discover schemas: tap-env/bin/tap-asana --config /tmp/tap_discover_config.json --discover
2021-01-11 13:48:34,226Z    tap - CRITICAL (missing_token) Missing access token parameter.
2021-01-11 13:48:34,226Z    tap - Traceback (most recent call last):
2021-01-11 13:48:34,226Z    tap -   File "tap-env/bin/tap-asana", line 33, in <module>
2021-01-11 13:48:34,227Z    tap -     sys.exit(load_entry_point('tap-asana==2.1.1', 'console_scripts', 'tap-asana')())
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 229, in wrapped
2021-01-11 13:48:34,227Z    tap -     return fnc(*args, **kwargs)
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_asana/__init__.py", line 172, in main
2021-01-11 13:48:34,227Z    tap -     Context.asana = Asana(**creds)
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_asana/asana.py", line 18, in __init__
2021-01-11 13:48:34,227Z    tap -     self.refresh_access_token()
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_asana/asana.py", line 35, in refresh_access_token
2021-01-11 13:48:34,227Z    tap -     return self._client.session.refresh_token(self._client.session.token_url, client_id=self.client_id, client_secret=self.client_secret, refresh_token=self.refresh_token)
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/requests_oauthlib/oauth2_session.py", line 446, in refresh_token
2021-01-11 13:48:34,227Z    tap -     self.token = self._client.parse_request_body_response(r.text, scope=self.scope)
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/oauthlib/oauth2/rfc6749/clients/base.py", line 421, in parse_request_body_response
2021-01-11 13:48:34,227Z    tap -     self.token = parse_token_response(body, scope=scope)
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 431, in parse_token_response
2021-01-11 13:48:34,227Z    tap -     validate_token_parameters(params)
2021-01-11 13:48:34,227Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/oauthlib/oauth2/rfc6749/parameters.py", line 441, in validate_token_parameters
2021-01-11 13:48:34,227Z    tap -     raise MissingTokenError(description="Missing access token parameter.")
2021-01-11 13:48:34,227Z    tap - oauthlib.oauth2.rfc6749.errors.MissingTokenError: (missing_token) Missing access token parameter.
2021-01-11 13:48:34,266Z   main - INFO Tap exited abnormally with status 1
2021-01-11 13:48:34,907Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [40] at offset None
2021-01-11 13:48:34,908Z   main - INFO No tunnel subprocess to tear down
2021-01-11 13:48:34,908Z   main - INFO Exit status is: Discovery failed with code 1 and error message: "(missing_token) Missing access token parameter.".

Full log of subsequent successful run:

2021-01-11 14:46:32,545Z   main - INFO Running tap-asana version 2.1.1 and target-stitch version 3.2.0
2021-01-11 14:46:32,681Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2021-01-11 14:46:32,685Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2021-01-11 14:46:32,685Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.streamRecordCount ), use_ssl( true )
2021-01-11 14:46:33,694Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [30] at offset None
2021-01-11 14:46:33,696Z   main - INFO Starting tap to discover schemas: tap-env/bin/tap-asana --config /tmp/tap_discover_config.json --discover
2021-01-11 14:46:34,714Z    tap - INFO Starting discover
2021-01-11 14:46:34,716Z    tap - INFO Finished discover
2021-01-11 14:46:34,758Z   main - INFO Tap exited normally.
2021-01-11 14:46:34,760Z   main - INFO Saving list of discovered streams
2021-01-11 14:46:34,779Z   main - INFO Saving structure of stream projects (tap_stream_id: projects)
2021-01-11 14:46:34,823Z   main - INFO Saving structure of stream portfolios (tap_stream_id: portfolios)
2021-01-11 14:46:34,842Z   main - INFO Saving structure of stream tasks (tap_stream_id: tasks)
2021-01-11 14:46:34,900Z   main - INFO Saving structure of stream stories (tap_stream_id: stories)
2021-01-11 14:46:34,934Z   main - INFO Saving structure of stream tags (tap_stream_id: tags)
2021-01-11 14:46:34,949Z   main - INFO Saving structure of stream users (tap_stream_id: users)
2021-01-11 14:46:35,002Z   main - INFO Saving structure of stream workspaces (tap_stream_id: workspaces)
2021-01-11 14:46:35,019Z   main - INFO Saving structure of stream teams (tap_stream_id: teams)
2021-01-11 14:46:35,037Z   main - INFO Saving structure of stream sections (tap_stream_id: sections)
2021-01-11 14:46:35,057Z   main - INFO Writing catalog to file
2021-01-11 14:46:35,083Z   main - INFO Selected fields for stream workspaces: name, resource_type, gid, is_organization
2021-01-11 14:46:35,083Z   main - INFO Selected fields for stream projects: current_status, notes, owner, due_date, name, archived, members, custom_fields, created_at, modified_at, followers, team, color, public, workspace, gid
2021-01-11 14:46:35,083Z   main - INFO Selected fields for stream users: email, name, workspaces, photo, gid
2021-01-11 14:46:35,083Z   main - INFO Selected fields for stream tasks: memberships, completed, num_hearts, start_on, completed_at, assignee, parent, projects, name, notes, assignee_status, custom_fields, created_at, modified_at, followers, hearts, due_on, tags, gid, due_at, hearted, workspace, external
2021-01-11 14:46:35,083Z   main - INFO Selected fields for stream tags: color, followers, name, created_at, workspace, notes, gid
2021-01-11 14:46:35,087Z   main - INFO Current state: {"bookmarks": {"projects": {"modified_at": "2021-01-08T19:31:56.479000"}, "tasks": {"modified_at": "2021-01-11T12:30:31.757000"}, "tags": {"created_at": "2020-12-02T21:44:23.859000"}}}
2021-01-11 14:46:35,087Z   main - INFO Starting tap: tap-env/bin/tap-asana --config /tmp/tap_config.json --properties /tmp/properties.json --catalog /tmp/catalog.json --state /tmp/tap_state.json
2021-01-11 14:46:35,091Z   main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2021-01-11 14:46:35,525Z    tap - INFO Syncing stream: projects
2021-01-11 14:46:35,543Z target - INFO Using batch_size_prefernces of {'batch_size_preference': None, 'user_batch_size_preference': None, 'full_table_streams': []}
2021-01-11 14:47:05,079Z    tap - INFO Skipping stream: portfolios
2021-01-11 14:47:05,079Z    tap - INFO Syncing stream: tasks
2021-01-11 14:53:02,875Z target - INFO Serializing batch with 32 messages for table tasks
2021-01-11 14:53:02,883Z target - INFO Sending batch of 249061 bytes to https://api.stitchdata.com/v2/import/batch
2021-01-11 14:53:03,213Z   main - INFO State update: adding bookmarks.projects.modified_at = "2021-01-11T14:01:10.940000"
2021-01-11 14:53:03,213Z   main - INFO State update: adding bookmarks.tags.created_at = "2020-12-02T21:44:23.859000"
2021-01-11 14:53:03,213Z   main - INFO State update: adding bookmarks.tasks.modified_at = "2021-01-11T12:30:31.757000"
2021-01-11 14:53:03,717Z target - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [1] at offset None
2021-01-11 14:53:03,717Z target - INFO replicated 32 records from "tasks" endpoint
2021-01-11 14:53:03,718Z target - INFO Serializing batch with 3 messages for table projects
2021-01-11 14:53:03,718Z target - INFO Sending batch of 7787 bytes to https://api.stitchdata.com/v2/import/batch
2021-01-11 14:53:03,723Z target - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [1] at offset None
2021-01-11 14:53:03,723Z target - INFO replicated 3 records from "projects" endpoint
2021-01-11 14:55:53,228Z    tap - INFO Skipping stream: stories
2021-01-11 14:55:53,229Z    tap - INFO Syncing stream: tags
2021-01-11 14:56:10,099Z    tap - INFO Syncing stream: users
2021-01-11 14:56:19,001Z    tap - INFO Syncing stream: workspaces
2021-01-11 14:56:20,858Z    tap - INFO Skipping stream: teams
2021-01-11 14:56:20,858Z    tap - INFO Skipping stream: sections
2021-01-11 14:56:20,858Z    tap - INFO ----------------------
2021-01-11 14:56:20,859Z    tap - INFO tasks: 83
2021-01-11 14:56:20,859Z    tap - INFO workspaces: 1
2021-01-11 14:56:20,859Z    tap - INFO tags: 2
2021-01-11 14:56:20,859Z    tap - INFO projects: 3
2021-01-11 14:56:20,859Z    tap - INFO users: 75
2021-01-11 14:56:20,859Z    tap - INFO ----------------------
2021-01-11 14:56:20,894Z target - INFO Serializing batch with 75 messages for table users
2021-01-11 14:56:20,895Z target - INFO Sending batch of 57577 bytes to https://api.stitchdata.com/v2/import/batch
2021-01-11 14:56:20,909Z target - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [8] at offset None
2021-01-11 14:56:20,909Z target - INFO replicated 75 records from "users" endpoint
2021-01-11 14:56:20,909Z target - INFO Serializing batch with 1 messages for table workspaces
2021-01-11 14:56:20,910Z target - INFO Sending batch of 857 bytes to https://api.stitchdata.com/v2/import/batch
2021-01-11 14:56:20,919Z target - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [3] at offset None
2021-01-11 14:56:20,919Z target - INFO replicated 1 records from "workspaces" endpoint
2021-01-11 14:56:20,919Z target - INFO Serializing batch with 51 messages for table tasks
2021-01-11 14:56:20,919Z target - INFO Sending batch of 535520 bytes to https://api.stitchdata.com/v2/import/batch
2021-01-11 14:56:20,925Z target - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [7] at offset None
2021-01-11 14:56:20,925Z target - INFO replicated 51 records from "tasks" endpoint
2021-01-11 14:56:20,925Z target - INFO Serializing batch with 2 messages for table tags
2021-01-11 14:56:20,925Z target - INFO Sending batch of 2167 bytes to https://api.stitchdata.com/v2/import/batch
2021-01-11 14:56:20,931Z target - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [5] at offset None
2021-01-11 14:56:20,931Z target - INFO replicated 2 records from "tags" endpoint
2021-01-11 14:56:21,334Z   main - INFO State update: changing bookmarks.tags.created_at from "2020-12-02T21:44:23.859000" to "2021-01-11T13:43:52.589000"
2021-01-11 14:56:21,334Z   main - INFO State update: changing bookmarks.tasks.modified_at from "2021-01-11T12:30:31.757000" to "2021-01-11T14:51:28.948000"
2021-01-11 14:56:21,336Z target - INFO Requests complete, stopping loop
2021-01-11 14:56:21,405Z   main - INFO Target exited normally with status 0
2021-01-11 14:56:21,766Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [15] at offset None
2021-01-11 14:56:21,767Z   main - INFO No tunnel subprocess to tear down
2021-01-11 14:56:21,767Z   main - INFO Exit status is: Discovery succeeded. Tap succeeded. Target succeeded.

[wesseljt]

@mferrier this is happening for me as well

[wesseljt]

I wonder if this could be related to the token issue present here

[mferrier]

sounds similar. I'll close this issue since it sounds like a stitch issue.

[wesseljt]

I could be wrong @mferrier, but I actually do think the tap has to be fixed to resolve.