Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON formatting check to the add-ign function #189

Merged
merged 2 commits into from
Jan 9, 2025

Conversation

skyblaster
Copy link
Contributor

Fixes #187

@baude
Copy link
Member

baude commented Dec 16, 2024

@skyblaster if you can fix the test failure, ill review and merge.

@skyblaster
Copy link
Contributor Author

@baude I'm unfamiliar with these particular tests, and not sure why it's failing, so will have to defer to @ashley-cui

If I download libhvee-0.6.0 or later releases, and run .\winmake.ps1 localtest on my local machine, I see the exact same errors.
If I simply run .\winmake.ps1 binaries, the executables are built without any errors.

@skyblaster
Copy link
Contributor Author

With a little debugging, I can say that the following URL needs to be updated as it results in a 404 which causes the win_test error:

fedoraBaseDirEndpoint = "https://kojipkgs.fedoraproject.org/compose/cloud/latest-Fedora-Cloud-39/compose"

Unfortunately it's not as simple as changing 39 to 40 or 41, as the new VHD images are compressed.

Example from 39: https://kojipkgs.fedoraproject.org/compose/metadata-archive/2024/Fedora-Cloud-39-20241126.0/images.json

                    {
                        "arch": "x86_64",
                        "bootable": false,
                        "checksums": {
                            "sha256": "b0ec83ff08499959ab9590cafc1d71645a48962ca10fbecf2aca2f6f3e0945ab"
                        },
                        "disc_count": 1,
                        "disc_number": 1,
                        "format": "vhd",
                        "implant_md5": null,
                        "mtime": 1732602402,
                        "path": "Cloud/x86_64/images/Fedora-Cloud-Base-Azure-39-20241126.0.x86_64.vhd",
                        "size": 1292184064,
                        "subvariant": "Cloud_Base",
                        "type": "vpc",
                        "volume_id": null
                    },

New compressed format: https://kojipkgs.fedoraproject.org/compose/cloud/latest-Fedora-Cloud-41/compose/metadata/images.json

                    {
                        "arch": "x86_64",
                        "bootable": false,
                        "checksums": {
                            "sha256": "c668dc15cddfae34649cb719c785bcbfdff858b10b2394fb6569bd676befa31a"
                        },
                        "disc_count": 1,
                        "disc_number": 1,
                        "format": "vhd.xz",
                        "implant_md5": null,
                        "mtime": 1734333904,
                        "path": "Cloud/x86_64/images/Fedora-Cloud-Base-Azure-41-20241216.0.x86_64.vhdfixed.xz",
                        "size": 527791932,
                        "subvariant": "Cloud_Base",
                        "type": "vhd-compressed",
                        "volume_id": null
                    },

@skyblaster
Copy link
Contributor Author

@baude I took a stab at fixing the e2e testing and it now passes.
The path to 41 seems to have disappeared, so I had to use 40.

I'm not sure why it appears to copy the decompressed image twice, so please have a look https://cirrus-ci.com/task/6376975813574656

Sorry for the lack of a sign-off. Please let me know if I should rebase, or resubmit the PR.

@baude
Copy link
Member

baude commented Jan 6, 2025

@skyblaster thanks! I was working on this before PTO and same thing as you ... only difference is I added a command line tool to the test image to unxz because the golang library is known to be very slow for decompression. Lets go ahead and merge this and I can switch over to the command line once I'm caught back up

)

const (
fedoraBaseDirEndpoint = "https://kojipkgs.fedoraproject.org/compose/cloud/latest-Fedora-Cloud-39/compose"
fedoraBaseDirEndpoint = "https://kojipkgs.fedoraproject.org/compose/cloud/latest-Fedora-Cloud-40/compose"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mind changing to 41?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed this as well in my image update Pr and tried to do a quick fix #197

Anyway it is complicated as there is no f41 on the server: i.e. 404 https://kojipkgs.fedoraproject.org/compose/cloud/latest-Fedora-Cloud-41/

I asked in the Fedora cloud matrix to see if this is a bug or not on their end. So Sticking to f40 seems to be the right call for now

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works for me

defer outFile.Close()

_, err = io.Copy(outFile, r)
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just return err

if err != nil {
return "", err
}
return vhdPath, nil
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return vhdPath, err wdyt ?

@Luap99
Copy link
Member

Luap99 commented Jan 8, 2025

@skyblaster Can you sign-off your commits, see https://github.com/containers/common/blob/main/CONTRIBUTING.md#sign-your-prs

We cannot accept contributions with unclear copyright status.

We can fixup the minor nits and migrate to the cli tool at a later point. In the interest of working CI I think this is great to merge

@skyblaster
Copy link
Contributor Author

My apologies for the delay and for not creating a branch of my original fork, so as to keep the two commits separate.
I was attempting to work on Brent's suggested changes, but ran out of steam as I'll admit I'm not much of a programmer.
If you're okay with accepting it as is, I'm happy to see it merged.

Cheers!

Copy link
Member

@Luap99 Luap99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

openshift-ci bot commented Jan 9, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Luap99, skyblaster

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved label Jan 9, 2025
@Luap99
Copy link
Member

Luap99 commented Jan 9, 2025

Thanks for the work @skyblaster, no need to apologize

@baude
Copy link
Member

baude commented Jan 9, 2025

@skyblaster thank you very much for your contribution!

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Jan 9, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit 4c29552 into containers:main Jan 9, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kvpctl check for properly formatted JSON
3 participants