Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dart): add graph support #5374

Merged
merged 11 commits into from
Oct 20, 2023
2 changes: 2 additions & 0 deletions docs/docs/configuration/reporting.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ The following packages/languages are currently supported:
- Composer
- Java
- Maven: pom.xml
- Dart
knqyf263 marked this conversation as resolved.
Show resolved Hide resolved
- pub: pubspec.lock

This tree is the reverse of the npm list command.
However, if you want to resolve a vulnerability in a particular indirect dependency, the reversed tree is useful to know where that dependency comes from and identify which package you actually need to update.
Expand Down
7 changes: 6 additions & 1 deletion docs/docs/coverage/language/dart.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,19 @@ The following table provides an outline of the features Trivy offers.

| Package manager | File | Transitive dependencies | Dev dependencies | [Dependency graph][dependency-graph] | Position |
|-------------------------|--------------|:-----------------------:|:----------------:|:------------------------------------:|:--------:|
| [Dart][dart-repository] | pubspec.lock | ✓ | Included | - | - |
| [Dart][dart-repository] | pubspec.lock | ✓ | Included | | - |

## Dart
In order to detect dependencies, Trivy searches for `pubspec.lock`.

Trivy marks indirect dependencies, but `pubspec.lock` file doesn't have options to separate root and dev transitive dependencies.
So Trivy includes all dependencies in report.

To build `dependency tree` Trivy parses [cache directory][cache-directory]. Currently supported default directories and `PUB_CACHE` environment (absolute path only).
!!! note
Make sure the cache directory contains all the dependencies installed in your application. To download missing dependencies, use `dart pub get` command.

[dart]: https://dart.dev/
[dart-repository]: https://pub.dev/
[dependency-graph]: ../../configuration/reporting.md#show-origins-of-vulnerable-dependencies
[cache-directory]: https://dart.dev/tools/pub/glossary#system-cache
160 changes: 151 additions & 9 deletions pkg/fanal/analyzer/language/dart/pub/pubspec.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,35 +2,177 @@ package pub

import (
"context"
"io"
"io/fs"
"os"
"path/filepath"
"runtime"
"sort"

"github.com/samber/lo"
"golang.org/x/xerrors"
"gopkg.in/yaml.v3"

"github.com/aquasecurity/go-dep-parser/pkg/dart/pub"
godeptypes "github.com/aquasecurity/go-dep-parser/pkg/types"
"github.com/aquasecurity/go-dep-parser/pkg/utils"
"github.com/aquasecurity/trivy/pkg/fanal/analyzer"
"github.com/aquasecurity/trivy/pkg/fanal/analyzer/language"
"github.com/aquasecurity/trivy/pkg/fanal/types"
"github.com/aquasecurity/trivy/pkg/log"
"github.com/aquasecurity/trivy/pkg/utils/fsutils"
)

func init() {
analyzer.RegisterAnalyzer(&pubSpecLockAnalyzer{})
analyzer.RegisterPostAnalyzer(analyzer.TypePubSpecLock, newPubSpecLockAnalyzer)
}

const (
version = 1
version = 2
pubSpecYamlFileName = "pubspec.yaml"
)

// pubSpecLockAnalyzer analyzes pubspec.lock
type pubSpecLockAnalyzer struct{}
// pubSpecLockAnalyzer analyzes `pubspec.lock`
type pubSpecLockAnalyzer struct {
parser godeptypes.Parser
}

func newPubSpecLockAnalyzer(_ analyzer.AnalyzerOptions) (analyzer.PostAnalyzer, error) {
return pubSpecLockAnalyzer{
parser: pub.NewParser(),
}, nil
}

func (a pubSpecLockAnalyzer) PostAnalyze(_ context.Context, input analyzer.PostAnalysisInput) (*analyzer.AnalysisResult, error) {
var apps []types.Application

// get all DependsOn from cache dir
// lib ID -> DependsOn names
allDependsOn, err := findDependsOn()
if err != nil {
log.Logger.Warnf("Unable to parse cache dir: %s", err)
}

required := func(path string, d fs.DirEntry) bool {
return filepath.Base(path) == types.PubSpecLock
}

err = fsutils.WalkDir(input.FS, ".", required, func(path string, _ fs.DirEntry, r io.Reader) error {
app, err := language.Parse(types.Pub, path, r, a.parser)
if err != nil {
return xerrors.Errorf("unable to parse %q: %w", path, err)
}

if app == nil {
return nil
}

if allDependsOn != nil {
// Required to search for library versions for DependsOn.
libs := lo.SliceToMap(app.Libraries, func(lib types.Package) (string, string) {
return lib.Name, lib.ID
})

for i, lib := range app.Libraries {
var dependsOn []string
for _, depName := range allDependsOn[lib.ID] {
if depID, ok := libs[depName]; ok {
dependsOn = append(dependsOn, depID)
}
}
app.Libraries[i].DependsOn = dependsOn
}
}

sort.Sort(app.Libraries)
apps = append(apps, *app)
return nil
})
if err != nil {
return nil, xerrors.Errorf("walk error: %w", err)
}

return &analyzer.AnalysisResult{
Applications: apps,
}, nil
}

func (a pubSpecLockAnalyzer) Analyze(_ context.Context, input analyzer.AnalysisInput) (*analyzer.AnalysisResult, error) {
p := pub.NewParser()
res, err := language.Analyze(types.Pub, input.FilePath, input.Content, p)
func findDependsOn() (map[string][]string, error) {
dir := cacheDir()
if !fsutils.DirExists(dir) {
log.Logger.Debugf("Cache dir (%s) not found. Need 'dart pub get' to fill dependency relationships", dir)
return nil, nil
}

deps := make(map[string][]string)
if err := filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we use fsutils.WalkDir?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is same case as with go.mod licenses -

err := filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {

Some example:
User uses default cache path (~/.pub_cache).
But he scans project dir (e.g. ~/project1).
In this case we don't add pubspec.yaml files from ~/.pub_cache in FS for PostAnalyze.
=> we can't read these files from input.FS.
Correct me, if i missed something.

Perhaps we can use os.DirFS("/") (fsutils.WalkDir(os.DirFS("/"), ".", required, func(path string, d fs.DirEntry, r io.Reader) error ). But i am not sure that it make sense.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't os.DirFS(dir) work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After my suggestion about os.DirFS I didn't think of an obvious solution 😄 .
Thank you!
I used fsutils.WalkDir in 3abdc47

if err != nil {
return err
} else if !d.Type().IsRegular() {
return nil
}
// parse only `pubspec.yaml` files
if filepath.Base(p) != pubSpecYamlFileName {
return nil
}

id, dependsOn, err := parsePubSpecYaml(p)
if err != nil {
log.Logger.Debugf("unable to parse %q: %s", p, err)
return nil
}
if id != "" {
deps[id] = dependsOn
}
return nil

}); err != nil {
return nil, xerrors.Errorf("walk error: %w", err)
}
return deps, nil
}

// https://dart.dev/tools/pub/glossary#system-cache
func cacheDir() string {
if dir := os.Getenv("PUB_CACHE"); dir != "" {
return dir
}

// `%LOCALAPPDATA%\Pub\Cache` for Windows
if runtime.GOOS == "windows" {
return filepath.Join(os.Getenv("LOCALAPPDATA"), "Pub", "Cache")
}

// `~/.pub-cache` for Linux or Mac
return filepath.Join(os.Getenv("HOME"), ".pub_cache")
}

type pubSpecYaml struct {
Name string `yaml:"name"`
Version string `yaml:"version"`
Dependencies map[string]interface{} `yaml:"dependencies,omitempty"`
}

func parsePubSpecYaml(filePath string) (string, []string, error) {
f, err := os.Open(filePath)
if err != nil {
return nil, xerrors.Errorf("%s parse error: %w", input.FilePath, err)
return "", nil, xerrors.Errorf("unable to open %q to get list of direct deps: %w", filePath, err)
}
defer func() { _ = f.Close() }()

var spec pubSpecYaml
if err = yaml.NewDecoder(f).Decode(&spec); err != nil {
return "", nil, xerrors.Errorf("unable to decode %q: %w", filePath, err)
}
if len(spec.Dependencies) > 0 {
// pubspec.yaml uses version ranges
// save only dependencies names
dependsOn := lo.MapToSlice(spec.Dependencies, func(key string, _ interface{}) string {
return key
})
return utils.PackageID(spec.Name, spec.Version), dependsOn, nil
knqyf263 marked this conversation as resolved.
Show resolved Hide resolved
}
return res, nil
return "", nil, nil
}

func (a pubSpecLockAnalyzer) Required(filePath string, _ os.FileInfo) bool {
Expand Down
Loading