Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reconcile chip-sampling functionality in RVPipelines with GeoDatasets (#1496) #2040

Merged
merged 7 commits into from
Feb 5, 2024

Conversation

AdeelH
Copy link
Collaborator

@AdeelH AdeelH commented Jan 30, 2024

Overview

This PR unites the code paths for direct chip sampling and chip sampling during the chip stage. The chip stage now uses GeoDatasets to sample chips, while still retaining the ability to filter out chips based on their content (e.g. based on NODATA %).

The window sampling configs have also been united such that both ChipOptions and GeoDataConfig now accept the same config: WindowSamplingConfig.

See commits for more details.

Checklist

  • Added unit tests, if applicable
  • Updated documentation, if applicable
  • Added needs-backport label if the change should be back-ported to the previous release
  • PR has a name that won't get you publicly shamed for vagueness

Notes

Despite the major changes, backward compatibility with older model bundles has been retained. This required enhancing Registry and upgrade_config() with the ability to handle config renamings.

Testing Instructions

See updated unit and integration tests.

Closes #1496

@AdeelH AdeelH changed the title Fix #1496 Reconcile chip-sampling functionality in RVPipelines with GeoDatasets (#1496) Feb 2, 2024
- Define ChipOptions. Make SemanticSegmentaitonChipOptions and ObjectDetectionChipOptions derive from it.
- Add chip_options field to RVPipelineConfig.
- Move train_chip_sz and chip_nodata_threshold to ChipOptions.
- SemanticSegmentationLabelSource.enough_target_pixles() -->  SemanticSegmentaitonChipOptions.enough_target_pixels().
- pytorch_learner.learner_config.GeoDataWIndowConfig --> core.rv_pipeline.chip_options.WindowSamplingConfig
- pytorch_learner.learner_config.GeoDataWIndowMethod --> core.rv_pipeline.chip_options.WindowSamplingMethod
- Move chipping to Backend and remove RVPipeline.get_train_windows() and RVPipeline.get_train_labels(). RVPipeline.chip() now calls Backend.chip_dataset().
- Remove some legacy object detection random sampling options.
- Update examlpes and integration tests.
- Add and update unit tests.
Copy link

codecov bot commented Feb 5, 2024

Codecov Report

Attention: 39 lines in your changes are missing coverage. Please review.

Comparison is base (2ed63e3) 86.35% compared to head (0095fa6) 89.46%.

Files Patch % Lines
...n/core/rv_pipeline/semantic_segmentation_config.py 71.42% 10 Missing ⚠️
...astervision/core/rv_pipeline/rv_pipeline_config.py 42.85% 8 Missing ⚠️
...ner/rastervision/pytorch_learner/learner_config.py 68.75% 5 Missing ⚠️
...rvision/pytorch_backend/pytorch_learner_backend.py 92.30% 3 Missing ⚠️
...vision/pytorch_backend/pytorch_object_detection.py 90.00% 3 Missing ⚠️
...tervision_pipeline/rastervision/pipeline/config.py 85.71% 2 Missing ⚠️
...ision/pytorch_learner/regression_learner_config.py 77.77% 2 Missing ⚠️
...ion/pytorch_backend/pytorch_chip_classification.py 95.00% 1 Missing ⚠️
...n/pytorch_backend/pytorch_semantic_segmentation.py 95.23% 1 Missing ⚠️
...n/pytorch_learner/classification_learner_config.py 88.88% 1 Missing ⚠️
... and 3 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2040      +/-   ##
==========================================
+ Coverage   86.35%   89.46%   +3.11%     
==========================================
  Files         196      198       +2     
  Lines        9760     9729      -31     
==========================================
+ Hits         8428     8704     +276     
+ Misses       1332     1025     -307     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AdeelH AdeelH marked this pull request as ready for review February 5, 2024 16:17
@AdeelH AdeelH merged commit 6ad3bdd into azavea:master Feb 5, 2024
2 checks passed
@AdeelH AdeelH deleted the chip branch February 5, 2024 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reconcile chip-sampling functionality in RVPipelines with GeoDatasets
1 participant