Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DSIP-73] Add dolphinscheduler-task-executor module to unify the task execution logic #16790

Merged

Conversation

ruanwenjun
Copy link
Member

Purpose of the pull request

close #16619

Brief change log

  • Add new module dolphinscheduler-task-executor
  • Add LogicTaskEngine, PhysicalTaskEngine impletation.

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

@ruanwenjun ruanwenjun force-pushed the dev_wenjun_implementLogicTaskSupportPasueKill branch from d3942d4 to 9abfb14 Compare November 11, 2024 15:06
@ruanwenjun ruanwenjun marked this pull request as draft November 11, 2024 15:06
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_implementLogicTaskSupportPasueKill branch 4 times, most recently from ac13593 to e3f5a95 Compare November 12, 2024 13:44
@ruanwenjun ruanwenjun added this to the 3.3.0 milestone Nov 12, 2024
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_implementLogicTaskSupportPasueKill branch 14 times, most recently from 2c71278 to 2830a68 Compare November 15, 2024 12:43
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_implementLogicTaskSupportPasueKill branch from a8c7311 to d1fa92b Compare November 20, 2024 14:23
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_implementLogicTaskSupportPasueKill branch 5 times, most recently from 95ea2da to 2f72848 Compare November 21, 2024 14:21
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_implementLogicTaskSupportPasueKill branch 12 times, most recently from c053789 to 0fcf9ab Compare November 27, 2024 12:41
@ruanwenjun ruanwenjun force-pushed the dev_wenjun_implementLogicTaskSupportPasueKill branch from 0fcf9ab to 1e2eaaa Compare November 28, 2024 14:21
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
24.8% Coverage on New Code (required ≥ 60%)
D Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

/**
* The task instance's runtime context changed.
*/
RUNTIME_CONTEXT_CHANGED,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why need to add this event type? The runtime context should be check in running event or finish event.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Import a new event type here can avoid we have multiple event schema for an event type. This event type represent the task runtime context changed, will not affect the state, if we use running event and finish event, we need to add runtime context info into these events, once runtime context changed, we will need to change these events.

And it's hard to explain why some running events have runtime context but some not.
So split the runtime context to a single event type can avoid these, this event type doesn't affect the lifecycle of a task.

Copy link
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall

@SbloodyS SbloodyS merged commit 673687d into apache:dev Nov 29, 2024
68 of 70 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DSIP-73] Add dolphinscheduler-task-executor module to unify the task execution logic
3 participants