Stabilize target_feature_11

[veluca93]

Stabilization report

This is an updated version of #116114, which is itself a redo of #99767. Most of this commit and report were copied from those PRs. Thanks @LeSeulArtichaut and @calebzulawski!

Summary

Allows for safe functions to be marked with #[target_feature] attributes.

Functions marked with #[target_feature] are generally considered as unsafe functions: they are unsafe to call, cannot generally be assigned to safe function pointers, and don't implement the Fn* traits.

However, calling them from other #[target_feature] functions with a superset of features is safe.

// Demonstration function
#[target_feature(enable = "avx2")]
fn avx2() {}

fn foo() {
    // Calling `avx2` here is unsafe, as we must ensure
    // that AVX is available first.
    unsafe {
        avx2();
    }
}

#[target_feature(enable = "avx2")]
fn bar() {
    // Calling `avx2` here is safe.
    avx2();
}

Moreover, once #135504 is merged, they can be converted to safe function pointers in a context in which calling them is safe:

// Demonstration function
#[target_feature(enable = "avx2")]
fn avx2() {}

fn foo() -> fn() {
    // Converting `avx2` to fn() is a compilation error here.
    avx2
}

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    // `avx2` coerces to fn() here
    avx2
}

See the section "Closures" below for justification of this behaviour.

Test cases

Tests for this feature can be found in tests/ui/target_feature/.

Edge cases

Closures

• target-feature 1.1: should closures inherit target-feature annotations? #73631

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

#[target_feature(enable = "avx2")]
fn qux() {
    let my_closure = || avx2(); // this call to `avx2` is safe
    let f: fn() = my_closure;
}

This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function, any closures defined inside it, as well as any safe function pointers obtained from target-feature functions inside it, execute.

This is usually ensured because target features are assumed to never disappear, and:

• on any unsafe call to a #[target_feature] function, presence of the target feature is guaranteed by the programmer through the safety requirements of the unsafe call.
• on any safe call, this is guaranteed recursively by the caller.

If you work in an environment where target features can be disabled, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again).

Note: this has an effect on existing code, as nowadays closures do not inherit features from the enclosing function, and thus this strengthens a safety requirement. It was originally proposed in #73631 to solve this by adding a new type of UB: “taking a target feature away from your process after having run code that uses that target feature is UB” .
This was motivated by userspace code already assuming in a few places that CPU features never disappear from a program during execution (see i.e. https://github.com/rust-lang/stdarch/blob/2e29bdf90832931ea499755bb4ad7a6b0809295a/crates/std_detect/src/detect/arch/x86.rs); however, concerns were raised in the context of the Linux kernel; thus, we propose to relax that requirement to "causing the set of usable features to be reduced is unsafe; when doing so, the programmer is required to ensure that no closures or safe fn pointers that use removed features are still in scope".

• Fix #[inline(always)] on closures with target feature 1.1 #111836

Closures accept #[inline(always)], even within functions marked with #[target_feature]. Since these attributes conflict, #[inline(always)] wins out to maintain compatibility.

ABI concerns

• The extern "C" ABI of SIMD vector types depends on target features #116558

The ABI of some types can change when compiling a function with different target features. This could have introduced unsoundness with target_feature_11, but recent fixes (#133102, #132173) either make those situations invalid or make the ABI no longer dependent on features. Thus, those issues should no longer occur.

Special functions

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), safe default trait implementations and safe trait methods.

This was not disallowed at the time of the first stabilization PR for target_features_11, and resulted in the following issues/PRs:

• #[target_feature] is allowed on main #108645
• #[target_feature] is allowed on default implementations #108646
• #[target_feature] is allowed on #[panic_handler] with target_feature 1.1 #109411
• Prevent using #[target_feature] on lang item functions #115910

Documentation

• Reference: Document the target_feature_11 feature reference#1181

---

cc tracking issue #69098
cc @workingjubilee
cc @RalfJung
r? @rust-lang/lang

[RalfJung]

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

Oddly, this means this compiles

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    || avx2()
}

but this does not

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    avx2
}

That's not a blocker but might be worth an issue. (But changing this may mean we have to refine the safety condition.)

This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function executes and for as long as any returned closures (or function pointers) that inherit features from that function live.

The definition of when something "live"s is subtle. I would make it about execution: you must guarantee that the target feature is available while the function or any closure defined inside that function executes. This is generally ensured because target features, once available, cannot usually be taken back; if you work in an environment where they can be taken back, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again).

The #[target_feature] attribute is forbidden from a variety of special functions

There is an enumeration saying that the attribute is allowed on main etc, only to then state that it is forbidden. This is confusing.

Have we covered all special functions?

[oli-obk]

I don't think target_feature is special here. You have the same "issue" for unsafe functions:

unsafe fn foo() {}

fn main() {
    let f = if true {
        None
    } else {
        unsafe{Some(|| foo())}
    }.unwrap_or_default();
    f();
}

[veluca93]

The latter does sound like a terrible idea, I would be surprised if that would not easily cause unsoundness in existing code...

The latter being Clone? That's long stable.

No, I meant closures implicitly implementing default if they could :-)

This code would fail to compile, and require unsafe, otherwise -- that would seem surprising to me.

It wouldn't require unsafe, it'd require extra annotations:

#[target_feature(enable="avx")]
fn also_use_avx() {
    println!("Hello from AVX")
}

#[target_feature(enable="avx")]
fn use_avx() {
    let f = #[target_feature(enable="avx")] || also_use_avx();
    f();
}

We allow #[inline] on closures so we might as well allow target_feature as well.

This would indeed fix the requirement for unsafe, but I am not sure it actually does anything for the actual "this is potentially risky" concerns.
Arguably, for the closures to be useful they need to be used wherever one would expect Fn traits, as otherwise i.e. all Iterator methods would not be usable with them. Thus, one could still return a impl Fn, and the concerns here become somewhat orthogonal from whether features are inherited or explicitly annotated.

[RalfJung]

I don't think target_feature is special here. You have the same "issue" for unsafe functions:

Ah right, unsafe fn cannot have Default on their item type. Good point.

[lcnr]

My worry here is not even necessary having the unsafe in dead code and then reconstructing the closure otherwise.

I expect it to be likely that you're able to extract the closure type out of a target feature fn without ever using unsafe using opaque types. I am not confident that the type system does not allow that for RPIT and it definitely allows it with TAITs. So if closures implement Default, you don't need a single call-site of its unsafe parent

[veluca93]

My worry here is not even necessary having the unsafe in dead code and then reconstructing the closure otherwise.

I expect it to be likely that you're able to extract the closure type out of a target feature fn without ever using unsafe using opaque types. I am not confident that the type system does not allow that for RPIT and it definitely allows it with TAITs. So if closures implement Default, you don't need a single call-site of its unsafe parent

Sure, but as Oli was pointing out, that situation would be broken already, right?

In particular it would allow you to synthetize closures generated in unsafe blocks, bypassing checks people might have put there:

fn f() -> impl FnOnce() {
   // do some check
   unsafe { || some_unsafe_fn() } 
}

fn bar() {
  if check() {
    unsafe { Some(|| some_unsafe_fn()) }
  } else {
    None
  }.unwrap_or_default()
}

If closure types ever implemented Default, both of those examples would be unsound.

[lcnr]

If closure types ever implemented Default, both of those examples would be unsound.

No, because the function only returns impl FnOnce(), so even if the underlying type implements Default, you cannot use it without adding a + Default bound to the opaque type (ignoring specialization, which is yet another issue with that feature). With target_feature_11 you can get that behavior without using an unsafe block anywhere. i.e. if you've got

// Unsound without ever actually using `unsafe`
type Tait = impl FnOnce() + Default;
#[target_feature(whatever)]
fn f() -> Tait {
   || target_feature_fn()
}

[veluca93]

Even if target_feature would create a slightly worse situation were Default to be implemented for all closures that could implement Default, I am not convinced that doing so would ever be a good idea, because it would make it significantly harder to write sound unsafe code.

If the eventual decision is to make closures implement Default if possible, it would be fairly easy to have closures that do inherit target_features not implement Default, as Ralf suggested, no?

[RalfJung]

"No unsafe block" and "unsafe block only in dead code" does not make much of a difference IMO.

[bors]

☔ The latest upstream changes (presumably #134299) made this pull request unmergeable. Please resolve the merge conflicts.

[traviscross]

@rustbot labels +S-waiting-on-documentation

Note that we don't want to merge this, even after the FCP completes, until the PR to the Reference (rust-lang/reference#1181) has been approved (it needs some updates for that to happen). So for the moment, this is blocked on documentation.

[veluca93]

@rustbot labels +S-waiting-on-documentation

Note that we don't want to merge this, even after the FCP completes, until the PR to the Reference (rust-lang/reference#1181) has been approved (it needs some updates for that to happen). So for the moment, this is blocked on documentation.

I opened rust-lang/reference#1720 for this.

While writing the documentation, I noticed another small inconsistency in the behaviour of target_feature_11 which we might want to resolve before merging (@RalfJung / @workingjubilee / @lcnr for comments); consider the following snippet:

#[target_feature(enable = "avx")]
fn f() { }


#[target_feature(enable = "avx")]
fn g() -> fn() {
    f
}


#[target_feature(enable = "avx")]
fn h() -> impl Fn() {
    || f()
}

#[target_feature(enable = "avx")]
fn i() -> impl Fn() {
    f
}

#[target_feature(enable = "avx")]
fn j() -> impl Fn() {
    f as fn()
}

Currently, g, h and j compile without any issues, but i gives a compilation error. Intuitively, I would imagine that i should also compile successfully, but I can imagine this running into issues with trait resolution that I am not knowledgeable enough to answer.
(requiring a cast to fn() pointer would IMO be a reasonable workaround, but if we decide on that then the documentation in the reference should definitely point it out)

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[veluca93]

@rustbot labels +S-waiting-on-documentation
Note that we don't want to merge this, even after the FCP completes, until the PR to the Reference (rust-lang/reference#1181) has been approved (it needs some updates for that to happen). So for the moment, this is blocked on documentation.

I opened rust-lang/reference#1720 for this.

While writing the documentation, I noticed another small inconsistency in the behaviour of target_feature_11 which we might want to resolve before merging (@RalfJung / @workingjubilee / @lcnr for comments); consider the following snippet:

#[target_feature(enable = "avx")]
fn f() { }


#[target_feature(enable = "avx")]
fn g() -> fn() {
    f
}


#[target_feature(enable = "avx")]
fn h() -> impl Fn() {
    || f()
}

#[target_feature(enable = "avx")]
fn i() -> impl Fn() {
    f
}

#[target_feature(enable = "avx")]
fn j() -> impl Fn() {
    f as fn()
}

Currently, g, h and j compile without any issues, but i gives a compilation error. Intuitively, I would imagine that i should also compile successfully, but I can imagine this running into issues with trait resolution that I am not knowledgeable enough to answer. (requiring a cast to fn() pointer would IMO be a reasonable workaround, but if we decide on that then the documentation in the reference should definitely point it out)

As discussed on Zulip, having fn items implement Fn traits conditionally would require significant changes to trait resolution, so the best solution is to explicitly mention coercion to fn pointer in the error message (#136064).

I also updated the reference PR accordingly.