Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Use osv-scalibr to perform container scanning #1505

Open
wants to merge 55 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
171c3be
Use scalibr fully to do container scanning
another-rex Dec 31, 2024
6941ee3
Fix tests and pipe through
another-rex Jan 2, 2025
11b3f88
Remove image package as it is no longer used
another-rex Jan 2, 2025
14555f8
Pass layer information to output
another-rex Jan 2, 2025
483f7c9
Merge branch 'main' into use-scalibr-container-scanning
another-rex Jan 6, 2025
40cabbd
Run go mod tidy
another-rex Jan 6, 2025
04beb87
Revert "Remove image package as it is no longer used"
another-rex Jan 6, 2025
314d586
Remove just the go files
another-rex Jan 6, 2025
06c40a6
Revert docker hash to build the older version of alpine images
another-rex Jan 6, 2025
11674c5
Updating logging output to be consistent and make sense
another-rex Jan 6, 2025
ef9b826
Update structures
another-rex Jan 6, 2025
2584ed4
go mod replace
another-rex Jan 7, 2025
39f6457
Merge branch 'main' into use-scalibr-container-scanning
another-rex Jan 7, 2025
ec00461
Merge branch 'main' into use-scalibr-container-scanning
another-rex Jan 7, 2025
e50238d
Merge new client matcher
another-rex Jan 7, 2025
d165ceb
update image structs (#14)
hogo6002 Jan 7, 2025
0e621c3
Add package scanning test for golang
another-rex Jan 7, 2025
464b773
Fix stdlib
another-rex Jan 7, 2025
5267f2c
Merge branch 'main' into use-scalibr-container-scanning
another-rex Jan 9, 2025
50fcd89
License matcher
another-rex Jan 8, 2025
18c31f2
Setup user agent as a parameter
another-rex Jan 9, 2025
10a196d
Extract matchers and rename Error
another-rex Jan 8, 2025
17bbc0f
Remove osv package from many places
another-rex Jan 9, 2025
0e07cb1
Use osvdev client in vendoredExtractor
another-rex Jan 9, 2025
a75c674
Fix compilation issue
another-rex Jan 13, 2025
02fc350
Temporarily merge in the symbolic link changes
another-rex Jan 13, 2025
47552bb
enhance --docker flag
hogo6002 Jan 13, 2025
d4f6eec
Hacked together calling deps.dev base image endpoint
another-rex Jan 13, 2025
6f71d15
update usr/lib
hogo6002 Jan 13, 2025
f240e52
add logs
hogo6002 Jan 14, 2025
d37eb73
A weird backfill attempt
another-rex Jan 14, 2025
9941a78
Merge pull request #15 from hogo6002/use-scalibr-container-scanning
hogo6002 Jan 14, 2025
680d70c
Merge branch 'base-image-queries' into use-scalibr-container-scanning
hogo6002 Jan 14, 2025
779d462
Merge pull request #16 from another-rex/use-scalibr-container-scanning
hogo6002 Jan 14, 2025
adf1309
Origin image bug fix
another-rex Jan 14, 2025
e656319
OS fix and index as key
another-rex Jan 15, 2025
b7ae0ab
Better layout of extractor and client definitions.
another-rex Jan 15, 2025
aaba95b
Add artifact extractors
another-rex Jan 15, 2025
216b51c
Merge branch 'models-refactor-2' into base-image-queries
another-rex Jan 15, 2025
f455024
Move where licensematcher is initialized
another-rex Jan 15, 2025
477ad8d
Merge branch 'models-refactor-2' into base-image-queries
another-rex Jan 15, 2025
1d5ba25
Merge branch 'main' into base-image-queries
another-rex Jan 15, 2025
1092859
Properly merge main in
another-rex Jan 15, 2025
4b1a339
Fix bug with backfill
another-rex Jan 15, 2025
bd1066c
Update go.mod to latest scalibr version, apply backfill.
another-rex Jan 15, 2025
2795764
Move docker image initialization out to it's own package.
another-rex Jan 15, 2025
dc16ad0
Copy over a bunch of changes
another-rex Jan 16, 2025
ea75bb0
Update OSPackageName snapshots
another-rex Jan 16, 2025
3d40e79
Update to latest scalibr to fix diffID issue
another-rex Jan 16, 2025
8172875
Add BaseImageMatcher
another-rex Jan 16, 2025
9ba5968
Parallel queries
another-rex Jan 17, 2025
7bd89be
Cleanup and move filling image metadata to imagehelpers
another-rex Jan 17, 2025
8f850dc
Update snapshots because new vuln got found
another-rex Jan 17, 2025
0f286eb
Resolve lints
another-rex Jan 17, 2025
1e78081
Update snapshots with omitempty
another-rex Jan 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
184 changes: 133 additions & 51 deletions cmd/osv-scanner/__snapshots__/main_test.snap

Large diffs are not rendered by default.

6 changes: 1 addition & 5 deletions cmd/osv-scanner/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -762,11 +762,8 @@ func TestRun_Licenses(t *testing.T) {
}
}

// TODO(v2): Image scanning is not temporarily disabled

func TestRun_Docker(t *testing.T) {
t.Parallel()
t.Skip("Skipping until image scanning is reenabled")

testutility.SkipIfNotAcceptanceTesting(t, "Takes a long time to pull down images")

Expand Down Expand Up @@ -795,7 +792,7 @@ func TestRun_Docker(t *testing.T) {
{
name: "Real Alpine image",
args: []string{"", "--docker", "alpine:3.18.9"},
exit: 0,
exit: 1,
},
}
for _, tt := range tests {
Expand All @@ -812,7 +809,6 @@ func TestRun_Docker(t *testing.T) {

func TestRun_OCIImage(t *testing.T) {
t.Parallel()
t.Skip("Skipping until image scanning is reenabled")

testutility.SkipIfNotAcceptanceTesting(t, "Not consistent on MacOS/Windows")

Expand Down
14 changes: 11 additions & 3 deletions cmd/osv-scanner/scan/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"time"

"github.com/google/osv-scanner/internal/spdx"
"github.com/google/osv-scanner/pkg/models"
"github.com/google/osv-scanner/pkg/osvscanner"
"github.com/google/osv-scanner/pkg/reporter"
"golang.org/x/term"
Expand Down Expand Up @@ -282,10 +283,10 @@ func action(context *cli.Context, stdout, stderr io.Writer) (reporter.Reporter,
scanLicensesAllowlist = []string{}
}

vulnResult, err := osvscanner.DoScan(osvscanner.ScannerActions{
scannerAction := osvscanner.ScannerActions{
LockfilePaths: context.StringSlice("lockfile"),
SBOMPaths: context.StringSlice("sbom"),
DockerImageName: context.String("docker"),
Image: context.String("docker"),
Recursive: context.Bool("recursive"),
SkipGit: context.Bool("skip-git"),
NoIgnore: context.Bool("no-ignore"),
Expand All @@ -311,7 +312,14 @@ func action(context *cli.Context, stdout, stderr io.Writer) (reporter.Reporter,
MavenRegistry: context.String("experimental-maven-registry"),
},
},
}, r)
}

var vulnResult models.VulnerabilityResults
if context.String("docker") != "" || context.String("experimental-oci-image") != "" {
vulnResult, err = osvscanner.DoContainerScan(scannerAction, r)
} else {
vulnResult, err = osvscanner.DoScan(scannerAction, r)
}

if err != nil && !errors.Is(err, osvscanner.ErrVulnerabilitiesFound) {
return r, err
Expand Down
48 changes: 44 additions & 4 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@ require (
github.com/go-git/go-billy/v5 v5.6.0
github.com/go-git/go-git/v5 v5.13.0
github.com/google/go-cmp v0.6.0
github.com/google/go-containerregistry v0.20.2
github.com/google/osv-scalibr v0.1.6-0.20250105222824-56e5c3bfb149
github.com/google/osv-scalibr v0.1.6-0.20250116021232-6d1c6b6edc63
github.com/ianlancetaylor/demangle v0.0.0-20240912202439-0a2b6291aafd
github.com/jedib0t/go-pretty/v6 v6.6.5
github.com/muesli/reflow v0.3.0
github.com/opencontainers/go-digest v1.0.0
github.com/ossf/osv-schema/bindings/go v0.0.0-20241210213101-57fd3ddb15aa
github.com/owenrumney/go-sarif/v2 v2.3.3
github.com/package-url/packageurl-go v0.1.3
Expand All @@ -44,7 +44,10 @@ require (

require (
dario.cat/mergo v1.0.0 // indirect
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 // indirect
github.com/AdamKorcz/go-118-fuzz-build v0.0.0-20230306123547-8075edf89bb0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Microsoft/hcsshim v0.11.5 // indirect
github.com/ProtonMail/go-crypto v1.1.3 // indirect
github.com/alecthomas/chroma/v2 v2.14.0 // indirect
github.com/anchore/go-struct-converter v0.0.0-20230627203149-c72ef8859ca9 // indirect
Expand All @@ -54,19 +57,39 @@ require (
github.com/charmbracelet/x/ansi v0.4.5 // indirect
github.com/charmbracelet/x/term v0.2.1 // indirect
github.com/cloudflare/circl v1.3.7 // indirect
github.com/containerd/cgroups v1.1.0 // indirect
github.com/containerd/containerd v1.7.18 // indirect
github.com/containerd/continuity v0.4.2 // indirect
github.com/containerd/errdefs v0.1.0 // indirect
github.com/containerd/fifo v1.1.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.15.1 // indirect
github.com/containerd/ttrpc v1.2.4 // indirect
github.com/containerd/typeurl/v2 v2.1.1 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.5 // indirect
github.com/cyphar/filepath-securejoin v0.2.5 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dlclark/regexp2 v1.11.0 // indirect
github.com/docker/cli v27.1.1+incompatible // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker-credential-helpers v0.8.1 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
github.com/emirpasic/gods v1.18.1 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/erikvarga/go-rpmdb v0.0.0-20240208180226-b97e041ef9af // indirect
github.com/felixge/httpsnoop v1.0.3 // indirect
github.com/gkampitakis/ciinfo v0.3.0 // indirect
github.com/gkampitakis/go-diff v1.3.2 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/google/go-containerregistry v0.20.2 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/groob/plist v0.1.1 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.17.7 // indirect
Expand All @@ -79,34 +102,51 @@ require (
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mattn/go-sqlite3 v1.14.22 // indirect
github.com/microcosm-cc/bluemonday v1.0.27 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/moby/locker v1.0.1 // indirect
github.com/moby/sys/mountinfo v0.6.2 // indirect
github.com/moby/sys/sequential v0.5.0 // indirect
github.com/moby/sys/signal v0.7.0 // indirect
github.com/moby/sys/user v0.1.0 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/opencontainers/runtime-spec v1.1.0 // indirect
github.com/opencontainers/selinux v1.11.0 // indirect
github.com/pjbgf/sha1cd v0.3.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/rogpeppe/go-internal v1.12.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sahilm/fuzzy v0.1.1 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/skeema/knownhosts v1.3.0 // indirect
github.com/spdx/gordf v0.0.0-20221230105357-b735bd5aac89 // indirect
github.com/spdx/tools-golang v0.5.5 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/vbatts/tar-split v0.11.5 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect
github.com/yuin/goldmark v1.7.4 // indirect
github.com/yuin/goldmark-emoji v1.0.3 // indirect
go.etcd.io/bbolt v1.3.10 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 // indirect
go.opentelemetry.io/otel v1.31.0 // indirect
go.opentelemetry.io/otel/metric v1.31.0 // indirect
go.opentelemetry.io/otel/trace v1.31.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/crypto v0.31.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/tools v0.28.0 // indirect
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
google.golang.org/genproto v0.0.0-20240123012728-ef4313101c80 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20241015192408-796eee8c2d53 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20241015192408-796eee8c2d53 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
www.velocidex.com/golang/regparser v0.0.0-20240404115756-2169ac0e3c09 // indirect
)
Loading
Loading