Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DSIP-27][Task Plugin] Some improvements of JAVA task plugin #16542

Merged
merged 67 commits into from
Dec 9, 2024

Conversation

ailiujiarui
Copy link
Contributor

@ailiujiarui ailiujiarui commented Aug 29, 2024

Purpose of the pull request

Update the Java task plugin
close:#15819

Brief change log

  • Deprecate write java code in JAVA task

  • Rename the JAR to FATJAR
    Only the displayed name has been modified, the functionality remains unchanged.

  • Add the new type NORMALJAR
    Allow users to upload normal type of jar files, which require external libraries to run properly. Users do not need to package all libraries and runtime files into a fat jar each time, making the submission of jar files more flexible and convenient.

  • Update the tests
    Update tests about JAVA type and add the test of JAR type. Like JavaTaskTest and workflowJavaTaskE2ETest.
    For the JavaE2Etest, the fat.java,normal1.java,normal2.java,have been added e2e-case/resources/docker/java-task. These three Java files will be compiled and packaged into fat.jar, normal1.jar, and normal2.jar. fat.jar is for the FAT_JAR type. The normal1.jar and normal2.jar are for the NORMAL_JAR type.

  • Update the Javadoc comment

type of FATJAR
be51083587282062ceda775cd255c1f

type of NORMALJAR
b06bccb0e0ebeaacd249a8000cf7a10

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

Pull Request Notice

Pull Request Notice

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

@github-actions github-actions bot added UI ui and front end related backend test labels Aug 29, 2024
@SbloodyS SbloodyS changed the title [Feature-15819][Task Plugin]Update the JAVA task [DSIP-15819][Task Plugin] Update the JAVA task Aug 29, 2024
@SbloodyS SbloodyS added first time contributor First-time contributor feature new feature labels Aug 29, 2024
@SbloodyS SbloodyS requested a review from ruanwenjun August 29, 2024 06:21
@SbloodyS SbloodyS changed the title [DSIP-15819][Task Plugin] Update the JAVA task [DSIP-27][Task Plugin] Update the JAVA task Aug 30, 2024
@SbloodyS SbloodyS added the DSIP label Aug 30, 2024
@ailiujiarui ailiujiarui force-pushed the dev branch 2 times, most recently from 9c45ca0 to 6d16b63 Compare September 2, 2024 12:08
@ailiujiarui ailiujiarui force-pushed the dev branch 3 times, most recently from 37b92ed to 151bdfe Compare September 2, 2024 12:44
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please resolve conflicts first.

@fuchanghai
Copy link
Member

image Please check if additional explanation is needed

@SbloodyS
Copy link
Member

There are some CI failed. Please check it. @ailiujiarui

The bug has been fixed

The WorkflowJavaTaskE2ETest still failed.

@ailiujiarui
Copy link
Contributor Author

There are some CI failed. Please check it. @ailiujiarui

The bug has been fixed

The WorkflowJavaTaskE2ETest still failed.

I've encountered a similar issue before as well. What worked for me was rerunning the entire test from the beginning, and that allowed it to pass successfully.

@SbloodyS
Copy link
Member

I've encountered a similar issue before as well. What worked for me was rerunning the entire test from the beginning, and that allowed it to pass successfully.

You should check the error message of java.lang.RuntimeException: No such package: normal1.jar at org.apache.dolphinscheduler.e2e.cases.WorkflowJavaTaskE2ETest.testCreateNormalJarWorkflow(WorkflowJavaTaskE2ETest.java:341)

@ailiujiarui
Copy link
Contributor Author

I've encountered a similar issue before as well. What worked for me was rerunning the entire test from the beginning, and that allowed it to pass successfully.

You should check the error message of java.lang.RuntimeException: No such package: normal1.jar at org.apache.dolphinscheduler.e2e.cases.WorkflowJavaTaskE2ETest.testCreateNormalJarWorkflow(WorkflowJavaTaskE2ETest.java:341)

Through testing, it was found that unstable file uploads—often resulting in failed uploads—and frequently missing resource file selections in the Resources panel are caused by the component's loading process. Specifically, webwait only detects the presence of DOM elements without ensuring that their child elements have fully loaded. In other words, the code continues to execute before the page is completely rendered, leading to these issues.

To address this problem, I implemented thread.sleep to wait for the complete loading. This method is currently stable and effective. I also considered other approaches, but none could effectively and concisely replace it.

SbloodyS
SbloodyS previously approved these changes Nov 27, 2024
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ailiujiarui
Copy link
Contributor Author

Requires Approval @ruanwenjun

Copy link

sonarqubecloud bot commented Dec 8, 2024

Please retry analysis of this Pull-Request directly on SonarQube Cloud

Copy link
Member

@Gallardot Gallardot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@ruanwenjun ruanwenjun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ruanwenjun ruanwenjun merged commit cad8df0 into apache:dev Dec 9, 2024
69 of 70 checks passed
Copy link

boring-cyborg bot commented Dec 9, 2024

Awesome work, congrats on your first merged pull request!

@ruanwenjun
Copy link
Member

Merged into dev, will be released at 3.3.0, thanks for your PR, helps a lot. @ailiujiarui

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend document DSIP e2e e2e test feature new feature first time contributor First-time contributor test UI ui and front end related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants