Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
comments
Browse files Browse the repository at this point in the history
Signed-off-by: Zach Puller <[email protected]>
zpuller committed Jan 14, 2025
1 parent 340c331 commit 1bac407
Showing 1 changed file with 1 addition and 6 deletions.
Original file line number Diff line number Diff line change
@@ -73,7 +73,7 @@ import org.apache.spark.storage.BlockId
* An object is spillable (it will be copied to host or disk during OOM) if:
* - it has a approxSizeInBytes > 0
* - it is not actively being referenced by the user (call to `materialize`, or aliased)
* - it hasn't already spilled
* - it hasn't already spilled, or is not currently being spilled
* - it hasn't been closed
*
* Aliasing:
@@ -140,11 +140,6 @@ import org.apache.spark.storage.BlockId
* `materialize` on a handle, the handle must guarantee that it can satisfy that, even if the caller
* should wait until a spill happens. This is currently implemented using the handle lock.
*
* Note that we hold the handle lock while we are spilling (performing IO). That means that no other
* consumer can access this spillable device handle while it is being spilled, including a second
* thread that is trying to spill and is generating a spill plan, as the handle lock is likely held
* up with IO. We will relax this likely in follow on work.
*
* We never hold a store-wide coarse grain lock in the stores when we do IO.
*/

0 comments on commit 1bac407

Please sign in to comment.