Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable CorePack Installing Package Managers from Private Registries #11077

Merged
merged 26 commits into from
Dec 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
30d8dd2
add registry for corepack install package managers
kbukum1 Dec 7, 2024
07c385c
add registry helper into imports
kbukum1 Dec 7, 2024
8da6ed9
remove lockfile to find out registry for corepack install
kbukum1 Dec 7, 2024
b4031df
fix parameters in testing
kbukum1 Dec 9, 2024
e78b1ff
fix specs
kbukum1 Dec 9, 2024
05ed720
fixed spec
kbukum1 Dec 9, 2024
5c596bf
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 9, 2024
234c4db
add dependabot yml credentials to check npm registry
kbukum1 Dec 9, 2024
9874d68
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 9, 2024
e870538
move registry_helper into package manager helper
kbukum1 Dec 10, 2024
8b8b130
remove registry_helper
kbukum1 Dec 10, 2024
2e90495
fix reverse proxy check
kbukum1 Dec 10, 2024
56f8c7c
fix yarnrc issue
kbukum1 Dec 10, 2024
ec013e3
improve the parsing
kbukum1 Dec 10, 2024
def21df
moved registry helper as seperate class
kbukum1 Dec 10, 2024
0789da6
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 10, 2024
7cd79a6
fix linting issue
kbukum1 Dec 10, 2024
16915d5
fix rubocop checks
kbukum1 Dec 10, 2024
6a0c2ec
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 10, 2024
c97ef73
activate installed package manager
kbukum1 Dec 10, 2024
e745cde
add logging when activating the package manager
kbukum1 Dec 10, 2024
2878c4d
fix spec
kbukum1 Dec 10, 2024
02b9f2f
Merge branch 'main' into kamil/set_corepack_registry_for_installing_n…
kbukum1 Dec 10, 2024
1a3805e
fix logging
kbukum1 Dec 10, 2024
d18afbc
add feature for corepack registry
kbukum1 Dec 11, 2024
679cea2
add feature flag for enable registry for corepack
kbukum1 Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion npm_and_yarn/lib/dependabot/npm_and_yarn/file_fetcher.rb
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,9 @@ def package_manager_helper
@package_manager_helper ||= T.let(
PackageManagerHelper.new(
parsed_package_json,
lockfiles: lockfiles
lockfiles,
registry_config_files,
credentials
), T.nilable(PackageManagerHelper)
)
end
Expand All @@ -221,6 +223,17 @@ def lockfiles
}
end

# Returns the .npmrc, and .yarnrc files for the repository.
# @return [Hash{Symbol => Dependabot::DependencyFile}]
sig { returns(T::Hash[Symbol, T.nilable(Dependabot::DependencyFile)]) }
def registry_config_files
{
npmrc: npmrc,
yarnrc: yarnrc,
yarnrc_yml: yarnrc_yml
}
end

sig { returns(DependencyFile) }
def package_json
@package_json ||= T.let(fetch_file_from_host(MANIFEST_FILENAME), T.nilable(DependencyFile))
Expand Down
34 changes: 33 additions & 1 deletion npm_and_yarn/lib/dependabot/npm_and_yarn/file_parser.rb
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,9 @@ def package_manager_helper
@package_manager_helper ||= T.let(
PackageManagerHelper.new(
parsed_package_json,
lockfiles: lockfiles
lockfiles,
registry_config_files,
credentials
), T.nilable(PackageManagerHelper)
)
end
Expand All @@ -112,6 +114,15 @@ def lockfiles
}
end

sig { returns(T::Hash[Symbol, T.nilable(Dependabot::DependencyFile)]) }
def registry_config_files
{
npmrc: npmrc,
yarnrc: yarnrc,
yarnrc_yml: yarnrc_yml
}
end

sig { returns(T.untyped) }
def parsed_package_json
JSON.parse(T.must(package_json.content))
Expand Down Expand Up @@ -156,6 +167,27 @@ def pnpm_lock
end, T.nilable(Dependabot::DependencyFile))
end

sig { returns(T.nilable(Dependabot::DependencyFile)) }
def npmrc
@npmrc ||= T.let(dependency_files.find do |f|
f.name == NpmPackageManager::RC_FILENAME
end, T.nilable(Dependabot::DependencyFile))
end

sig { returns(T.nilable(Dependabot::DependencyFile)) }
def yarnrc
@yarnrc ||= T.let(dependency_files.find do |f|
f.name == YarnPackageManager::RC_FILENAME
end, T.nilable(Dependabot::DependencyFile))
end

sig { returns(T.nilable(DependencyFile)) }
def yarnrc_yml
@yarnrc_yml ||= T.let(dependency_files.find do |f|
f.name == YarnPackageManager::RC_YML_FILENAME
end, T.nilable(Dependabot::DependencyFile))
end

sig { returns(Dependabot::FileParsers::Base::DependencySet) }
def manifest_dependencies
dependency_set = DependencySet.new
Expand Down
31 changes: 25 additions & 6 deletions npm_and_yarn/lib/dependabot/npm_and_yarn/helpers.rb
Original file line number Diff line number Diff line change
Expand Up @@ -388,17 +388,28 @@ def self.run_single_yarn_command(command, fingerprint: nil)
end

# Install the package manager for specified version by using corepack
sig { params(name: String, version: String).returns(String) }
def self.install(name, version)
sig do
params(
name: String,
version: String,
env: T.nilable(T::Hash[String, String])
)
.returns(String)
end
def self.install(name, version, env: {})
Dependabot.logger.info("Installing \"#{name}@#{version}\"")

begin
# Try to install the specified version
output = package_manager_install(name, version)
output = package_manager_install(name, version, env: env)

# Confirm success based on the output
if output.match?(/Adding #{name}@.* to the cache/)
Dependabot.logger.info("#{name}@#{version} successfully installed.")

Dependabot.logger.info("Activating currently installed version of #{name}: #{version}")
package_manager_activate(name, version)

else
Dependabot.logger.error("Corepack installation output unexpected: #{output}")
fallback_to_local_version(name)
Expand Down Expand Up @@ -428,11 +439,19 @@ def self.fallback_to_local_version(name)
end

# Install the package manager for specified version by using corepack
sig { params(name: String, version: String).returns(String) }
def self.package_manager_install(name, version)
sig do
params(
name: String,
version: String,
env: T.nilable(T::Hash[String, String])
)
.returns(String)
end
def self.package_manager_install(name, version, env: {})
Dependabot::SharedHelpers.run_shell_command(
"corepack install #{name}@#{version} --global --cache-only",
fingerprint: "corepack install <name>@<version> --global --cache-only"
fingerprint: "corepack install <name>@<version> --global --cache-only",
env: env
).strip
end

Expand Down
32 changes: 21 additions & 11 deletions npm_and_yarn/lib/dependabot/npm_and_yarn/package_manager.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
require "dependabot/ecosystem"
require "dependabot/npm_and_yarn/requirement"
require "dependabot/npm_and_yarn/version_selector"
require "dependabot/npm_and_yarn/registry_helper"

module Dependabot
module NpmAndYarn
Expand Down Expand Up @@ -311,17 +312,24 @@ class PackageManagerHelper
sig do
params(
package_json: T.nilable(T::Hash[String, T.untyped]),
lockfiles: T::Hash[Symbol, T.nilable(Dependabot::DependencyFile)]
lockfiles: T::Hash[Symbol, T.nilable(Dependabot::DependencyFile)],
registry_config_files: T::Hash[Symbol, T.nilable(Dependabot::DependencyFile)],
credentials: T.nilable(T::Array[Dependabot::Credential])
).void
end
def initialize(package_json, lockfiles:)
def initialize(package_json, lockfiles, registry_config_files, credentials)
@package_json = package_json
@lockfiles = lockfiles
@registry_helper = T.let(
RegistryHelper.new(registry_config_files, credentials),
Dependabot::NpmAndYarn::RegistryHelper
)
@package_manager_detector = T.let(PackageManagerDetector.new(lockfiles, package_json), PackageManagerDetector)
@manifest_package_manager = T.let(package_json&.fetch(MANIFEST_PACKAGE_MANAGER_KEY, nil), T.nilable(String))
@engines = T.let(package_json&.fetch(MANIFEST_ENGINES_KEY, nil), T.nilable(T::Hash[String, T.untyped]))

@installed_versions = T.let({}, T::Hash[String, String])
@registries = T.let({}, T::Hash[String, String])

@language = T.let(nil, T.nilable(Ecosystem::VersionManager))
@language_requirement = T.let(nil, T.nilable(Requirement))
Expand Down Expand Up @@ -379,8 +387,8 @@ def find_engine_constraints_as_requirement(name)
end

# rubocop:disable Metrics/CyclomaticComplexity
# rubocop:disable Metrics/PerceivedComplexity
# rubocop:disable Metrics/AbcSize
# rubocop:disable Metrics/PerceivedComplexity
sig { params(name: String).returns(T.nilable(T.any(Integer, String))) }
def setup(name)
# we prioritize version mentioned in "packageManager" instead of "engines"
Expand Down Expand Up @@ -438,6 +446,9 @@ def setup(name)
end
version
end
# rubocop:enable Metrics/CyclomaticComplexity
# rubocop:enable Metrics/AbcSize
# rubocop:enable Metrics/PerceivedComplexity

sig { params(name: T.nilable(String)).returns(Ecosystem::VersionManager) }
def package_manager_by_name(name)
Expand All @@ -456,21 +467,15 @@ def package_manager_by_name(name)
Dependabot.logger.info("No version requirement found for #{name}")
end

package_manager_instance = package_manager_class.new(
package_manager_class.new(
installed_version,
requirement: package_manager_requirement
)

Dependabot.logger.info("Package manager resolved for #{name}: #{package_manager_instance}")
package_manager_instance
rescue StandardError => e
Dependabot.logger.error("Error resolving package manager for #{name || 'default'}: #{e.message}")
raise
end

# rubocop:enable Metrics/CyclomaticComplexity
# rubocop:enable Metrics/PerceivedComplexity
# rubocop:enable Metrics/AbcSize
# Retrieve the installed version of the package manager by executing
# the "corepack <name> -v" command and using the output.
# If the output does not match the expected version format (PACKAGE_MANAGER_VERSION_REGEX),
Expand Down Expand Up @@ -510,7 +515,12 @@ def raise_if_unsupported!(name, version)
sig { params(name: String, version: T.nilable(String)).void }
def install(name, version)
if Dependabot::Experiments.enabled?(:enable_corepack_for_npm_and_yarn)
return Helpers.install(name, version.to_s)
env = {}
if Dependabot::Experiments.enabled?(:enable_private_registry_for_corepack)
env = @registry_helper.find_corepack_env_variables
end
# Use the Helpers.install method to install the package manager
return Helpers.install(name, version.to_s, env: env)
end

Dependabot.logger.info("Installing \"#{name}@#{version}\"")
Expand Down
Loading
Loading