-
-
Notifications
You must be signed in to change notification settings - Fork 52
/
Copy path0305-x86-spec-ctrl-Support-for-SRSO_U-S_NO-and-SRSO_MSR_F.patch
350 lines (309 loc) · 15.5 KB
/
0305-x86-spec-ctrl-Support-for-SRSO_U-S_NO-and-SRSO_MSR_F.patch
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
From db40e7b40bb68470684d6bef2c0318c448df34d8 Mon Sep 17 00:00:00 2001
From: Andrew Cooper <[email protected]>
Date: Fri, 20 Dec 2024 19:34:24 +0000
Subject: [PATCH 305/306] x86/spec-ctrl: Support for SRSO_U/S_NO and
SRSO_MSR_FIX
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
AMD have updated the SRSO whitepaper[1] with further information. These
features exist on AMD Zen5 CPUs and are necessary for Xen to use.
The two features are in principle unrelated:
* SRSO_U/S_NO is an enumeration saying that SRSO attacks can't cross the
User(CPL3) / Supervisor(CPL<3) boundary. i.e. Xen don't need to use
IBPB-on-entry for PV64. PV32 guests are explicitly unsupported for
speculative issues, and excluded from consideration for simplicity.
* SRSO_MSR_FIX is an enumeration identifying that the BP_SPEC_REDUCE bit is
available in MSR_BP_CFG. When set, SRSO attacks can't cross the host/guest
boundary. i.e. Xen don't need to use IBPB-on-entry for HVM.
Extend ibpb_calculations() to account for these when calculating
opt_ibpb_entry_{pv,hvm} defaults. Add a `bp-spec-reduce=<bool>` option to
control the use of BP_SPEC_REDUCE, with it active by default.
Because MSR_BP_CFG is core-scoped with a race condition updating it, repurpose
amd_check_erratum_1485() into amd_check_bp_cfg() and calculate all updates at
once.
Xen also needs to to advertise SRSO_U/S_NO to guests to allow the guest kernel
to skip SRSO mitigations too:
* This is trivial for HVM guests. It is also is accurate for PV32 guests
too, but we have already excluded them from consideration, and do so again
here to simplify the policy logic.
* As written, SRSO_U/S_NO does not help for the PV64 user->kernel boundary.
However, after discussing with AMD, an implementation detail of having
BP_SPEC_REDUCE active causes the PV64 user->kernel boundary to have the
property described by SRSO_U/S_NO, so we can advertise SRSO_U/S_NO to
guests when the BP_SPEC_REDUCE precondition is met.
Finally, fix a typo in the SRSO_NO's comment.
[1] https://www.amd.com/content/dam/amd/en/documents/corporate/cr/speculative-return-stack-overflow-whitepaper.pdf
Signed-off-by: Andrew Cooper <[email protected]>
Reviewed-by: Roger Pau Monné <[email protected]>
---
docs/misc/xen-command-line.pandoc | 9 +++-
xen/arch/x86/cpu-policy.c | 21 +++++++++
xen/arch/x86/cpu/amd.c | 29 +++++++++---
xen/arch/x86/include/asm/msr-index.h | 1 +
xen/arch/x86/include/asm/spec_ctrl.h | 1 +
xen/arch/x86/spec_ctrl.c | 51 ++++++++++++++++-----
xen/include/public/arch-x86/cpufeatureset.h | 4 +-
7 files changed, 96 insertions(+), 20 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 2096ae5841de..1944847172d7 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -2392,7 +2392,7 @@ By default SSBD will be mitigated at runtime (i.e `ssbd=runtime`).
> {ibrs,ibpb,ssbd,psfd,
> eager-fpu,l1d-flush,branch-harden,srb-lock,
> unpriv-mmio,gds-mit,div-scrub,lock-harden,
-> bhi-dis-s}=<bool> ]`
+> bhi-dis-s,bp-spec-reduce}=<bool> ]`
Controls for speculative execution sidechannel mitigations. By default, Xen
will pick the most appropriate mitigations based on compiled in support,
@@ -2541,6 +2541,13 @@ boolean can be used to force or prevent Xen from using speculation barriers to
protect lock critical regions. This mitigation won't be engaged by default,
and needs to be explicitly enabled on the command line.
+On hardware supporting SRSO_MSR_FIX, the `bp-spec-reduce=` option can be used
+to force or prevent Xen from using MSR_BP_CFG.BP_SPEC_REDUCE to mitigate the
+SRSO (Speculative Return Stack Overflow) vulnerability. Xen will use
+bp-spec-reduce when available, as it is preferable to using `ibpb-entry=hvm`
+to mitigate SRSO for HVM guests, and because it is a prerequisite to advertise
+SRSO_U/S_NO to PV guests.
+
### sync_console
> `= <boolean>`
diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index 304dc20cfab8..1722f5f90392 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -14,6 +14,7 @@
#include <asm/msr-index.h>
#include <asm/paging.h>
#include <asm/setup.h>
+#include <asm/spec_ctrl.h>
#include <asm/xstate.h>
struct cpu_policy __read_mostly raw_cpu_policy;
@@ -605,6 +606,26 @@ static void __init calculate_pv_max_policy(void)
__clear_bit(X86_FEATURE_IBRS, fs);
}
+ /*
+ * SRSO_U/S_NO means that the CPU is not vulnerable to SRSO attacks across
+ * the User (CPL3)/Supervisor (CPL<3) boundary.
+ *
+ * PV32 guests are unsupported for speculative issues, and excluded from
+ * consideration for simplicity.
+ *
+ * The PV64 user/kernel boundary is CPL3 on both sides, so SRSO_U/S_NO
+ * won't convey the meaning that a PV kernel expects.
+ *
+ * After discussions with AMD, an implementation detail of having
+ * BP_SPEC_REDUCE active causes the PV64 user/kernel boundary to have a
+ * property compatible with the meaning of SRSO_U/S_NO.
+ *
+ * If BP_SPEC_REDUCE isn't active, remove SRSO_U/S_NO from the PV max
+ * policy, which will cause it to filter out of PV default too.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SRSO_MSR_FIX) || !opt_bp_spec_reduce )
+ __clear_bit(X86_FEATURE_SRSO_US_NO, fs);
+
guest_common_max_feature_adjustments(fs);
guest_common_feature_adjustments(fs);
diff --git a/xen/arch/x86/cpu/amd.c b/xen/arch/x86/cpu/amd.c
index ab92333673b9..c448997be551 100644
--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -1009,16 +1009,33 @@ static void cf_check fam17_disable_c6(void *arg)
wrmsrl(MSR_AMD_CSTATE_CFG, val & mask);
}
-static void amd_check_erratum_1485(void)
+static void amd_check_bp_cfg(void)
{
- uint64_t val, chickenbit = (1 << 5);
+ uint64_t val, new = 0;
- if (cpu_has_hypervisor || boot_cpu_data.x86 != 0x19 || !is_zen4_uarch())
+ /*
+ * AMD Erratum #1485. Set bit 5, as instructed.
+ */
+ if (!cpu_has_hypervisor && boot_cpu_data.x86 == 0x19 && is_zen4_uarch())
+ new |= (1 << 5);
+
+ /*
+ * On hardware supporting SRSO_MSR_FIX, activate BP_SPEC_REDUCE by
+ * default. This lets us do two things:
+ *
+ * 1) Avoid IBPB-on-entry to mitigate SRSO attacks from HVM guests.
+ * 2) Advertise SRSO_US_NO to PV guests.
+ */
+ if (boot_cpu_has(X86_FEATURE_SRSO_MSR_FIX) && opt_bp_spec_reduce)
+ new |= BP_CFG_SPEC_REDUCE;
+
+ /* Avoid reading BP_CFG if we don't intend to change anything. */
+ if (!new)
return;
rdmsrl(MSR_AMD64_BP_CFG, val);
- if (val & chickenbit)
+ if ((val & new) == new)
return;
/*
@@ -1027,7 +1044,7 @@ static void amd_check_erratum_1485(void)
* same time before the chickenbit is set. It's benign because the
* value being written is the same on both.
*/
- wrmsrl(MSR_AMD64_BP_CFG, val | chickenbit);
+ wrmsrl(MSR_AMD64_BP_CFG, val | new);
}
static void cf_check init_amd(struct cpuinfo_x86 *c)
@@ -1297,7 +1314,7 @@ static void cf_check init_amd(struct cpuinfo_x86 *c)
disable_c1_ramping();
amd_check_zenbleed();
- amd_check_erratum_1485();
+ amd_check_bp_cfg();
if (fam17_c6_disabled)
fam17_disable_c6(NULL);
diff --git a/xen/arch/x86/include/asm/msr-index.h b/xen/arch/x86/include/asm/msr-index.h
index 17dd857af802..b324356fd550 100644
--- a/xen/arch/x86/include/asm/msr-index.h
+++ b/xen/arch/x86/include/asm/msr-index.h
@@ -412,6 +412,7 @@
#define AMD64_DE_CFG_LFENCE_SERIALISE (_AC(1, ULL) << 1)
#define MSR_AMD64_EX_CFG 0xc001102cU
#define MSR_AMD64_BP_CFG 0xc001102eU
+#define BP_CFG_SPEC_REDUCE (_AC(1, ULL) << 4)
#define MSR_AMD64_DE_CFG2 0xc00110e3U
#define MSR_AMD64_DR0_ADDRESS_MASK 0xc0011027U
diff --git a/xen/arch/x86/include/asm/spec_ctrl.h b/xen/arch/x86/include/asm/spec_ctrl.h
index 72347ef2b959..077225418956 100644
--- a/xen/arch/x86/include/asm/spec_ctrl.h
+++ b/xen/arch/x86/include/asm/spec_ctrl.h
@@ -90,6 +90,7 @@ extern int8_t opt_xpti_hwdom, opt_xpti_domu;
extern bool cpu_has_bug_l1tf;
extern int8_t opt_pv_l1tf_hwdom, opt_pv_l1tf_domu;
+extern bool opt_bp_spec_reduce;
/*
* The L1D address mask, which might be wider than reported in CPUID, and the
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 40f6ae017010..35351044f901 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -83,6 +83,7 @@ static bool __initdata opt_unpriv_mmio;
static bool __ro_after_init opt_verw_mmio;
static int8_t __initdata opt_gds_mit = -1;
static int8_t __initdata opt_div_scrub = -1;
+bool __ro_after_init opt_bp_spec_reduce = true;
static int __init cf_check parse_spec_ctrl(const char *s)
{
@@ -143,6 +144,7 @@ static int __init cf_check parse_spec_ctrl(const char *s)
opt_unpriv_mmio = false;
opt_gds_mit = 0;
opt_div_scrub = 0;
+ opt_bp_spec_reduce = false;
}
else if ( val > 0 )
rc = -EINVAL;
@@ -363,6 +365,8 @@ static int __init cf_check parse_spec_ctrl(const char *s)
opt_gds_mit = val;
else if ( (val = parse_boolean("div-scrub", s, ss)) >= 0 )
opt_div_scrub = val;
+ else if ( (val = parse_boolean("bp-spec-reduce", s, ss)) >= 0 )
+ opt_bp_spec_reduce = val;
else
rc = -EINVAL;
@@ -505,7 +509,7 @@ static void __init print_details(enum ind_thunk thunk)
* Hardware read-only information, stating immunity to certain issues, or
* suggestions of which mitigation to use.
*/
- printk(" Hardware hints:%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s\n",
+ printk(" Hardware hints:%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s\n",
(caps & ARCH_CAPS_RDCL_NO) ? " RDCL_NO" : "",
(caps & ARCH_CAPS_EIBRS) ? " EIBRS" : "",
(caps & ARCH_CAPS_RSBA) ? " RSBA" : "",
@@ -529,10 +533,11 @@ static void __init print_details(enum ind_thunk thunk)
(e8b & cpufeat_mask(X86_FEATURE_BTC_NO)) ? " BTC_NO" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBPB_RET)) ? " IBPB_RET" : "",
(e21a & cpufeat_mask(X86_FEATURE_IBPB_BRTYPE)) ? " IBPB_BRTYPE" : "",
- (e21a & cpufeat_mask(X86_FEATURE_SRSO_NO)) ? " SRSO_NO" : "");
+ (e21a & cpufeat_mask(X86_FEATURE_SRSO_NO)) ? " SRSO_NO" : "",
+ (e21a & cpufeat_mask(X86_FEATURE_SRSO_US_NO)) ? " SRSO_US_NO" : "");
/* Hardware features which need driving to mitigate issues. */
- printk(" Hardware features:%s%s%s%s%s%s%s%s%s%s%s%s%s%s\n",
+ printk(" Hardware features:%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s\n",
(e8b & cpufeat_mask(X86_FEATURE_IBPB)) ||
(_7d0 & cpufeat_mask(X86_FEATURE_IBRSB)) ? " IBPB" : "",
(e8b & cpufeat_mask(X86_FEATURE_IBRS)) ||
@@ -551,7 +556,8 @@ static void __init print_details(enum ind_thunk thunk)
(caps & ARCH_CAPS_FB_CLEAR_CTRL) ? " FB_CLEAR_CTRL" : "",
(caps & ARCH_CAPS_GDS_CTRL) ? " GDS_CTRL" : "",
(caps & ARCH_CAPS_RFDS_CLEAR) ? " RFDS_CLEAR" : "",
- (e21a & cpufeat_mask(X86_FEATURE_SBPB)) ? " SBPB" : "");
+ (e21a & cpufeat_mask(X86_FEATURE_SBPB)) ? " SBPB" : "",
+ (e21a & cpufeat_mask(X86_FEATURE_SRSO_MSR_FIX)) ? " SRSO_MSR_FIX" : "");
/* Compiled-in support which pertains to mitigations. */
if ( IS_ENABLED(CONFIG_INDIRECT_THUNK) || IS_ENABLED(CONFIG_SHADOW_PAGING) ||
@@ -1120,7 +1126,7 @@ static void __init div_calculations(bool hw_smt_enabled)
static void __init ibpb_calculations(void)
{
- bool def_ibpb_entry = false;
+ bool def_ibpb_entry_pv = false, def_ibpb_entry_hvm = false;
/* Check we have hardware IBPB support before using it... */
if ( !boot_cpu_has(X86_FEATURE_IBRSB) && !boot_cpu_has(X86_FEATURE_IBPB) )
@@ -1145,22 +1151,43 @@ static void __init ibpb_calculations(void)
* Confusion. Mitigate with IBPB-on-entry.
*/
if ( !boot_cpu_has(X86_FEATURE_BTC_NO) )
- def_ibpb_entry = true;
+ def_ibpb_entry_pv = def_ibpb_entry_hvm = true;
/*
- * Further to BTC, Zen3/4 CPUs suffer from Speculative Return Stack
- * Overflow in most configurations. Mitigate with IBPB-on-entry if we
- * have the microcode that makes this an effective option.
+ * In addition to BTC, Zen3 and later CPUs suffer from Speculative
+ * Return Stack Overflow in most configurations. If we have microcode
+ * that makes IBPB-on-entry an effective mitigation, see about using
+ * it.
*/
if ( !boot_cpu_has(X86_FEATURE_SRSO_NO) &&
boot_cpu_has(X86_FEATURE_IBPB_BRTYPE) )
- def_ibpb_entry = true;
+ {
+ /*
+ * SRSO_U/S_NO is a subset of SRSO_NO, identifying that SRSO isn't
+ * possible across the User (CPL3) / Supervisor (CPL<3) boundary.
+ *
+ * Ignoring PV32 (not security supported for speculative issues),
+ * this means we only need to use IBPB-on-entry for PV guests on
+ * hardware which doesn't enumerate SRSO_US_NO.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SRSO_US_NO) )
+ def_ibpb_entry_pv = true;
+
+ /*
+ * SRSO_MSR_FIX enumerates that we can use MSR_BP_CFG.SPEC_REDUCE
+ * to mitigate SRSO across the host/guest boundary. We only need
+ * to use IBPB-on-entry for HVM guests if we haven't enabled this
+ * control.
+ */
+ if ( !boot_cpu_has(X86_FEATURE_SRSO_MSR_FIX) || !opt_bp_spec_reduce )
+ def_ibpb_entry_hvm = true;
+ }
}
if ( opt_ibpb_entry_pv == -1 )
- opt_ibpb_entry_pv = IS_ENABLED(CONFIG_PV) && def_ibpb_entry;
+ opt_ibpb_entry_pv = IS_ENABLED(CONFIG_PV) && def_ibpb_entry_pv;
if ( opt_ibpb_entry_hvm == -1 )
- opt_ibpb_entry_hvm = IS_ENABLED(CONFIG_HVM) && def_ibpb_entry;
+ opt_ibpb_entry_hvm = IS_ENABLED(CONFIG_HVM) && def_ibpb_entry_hvm;
if ( opt_ibpb_entry_pv )
{
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index d9eba5e9a714..9c98e4992861 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -312,7 +312,9 @@ XEN_CPUFEATURE(FSRSC, 11*32+19) /*A Fast Short REP SCASB */
XEN_CPUFEATURE(AMD_PREFETCHI, 11*32+20) /*A PREFETCHIT{0,1} Instructions */
XEN_CPUFEATURE(SBPB, 11*32+27) /*A Selective Branch Predictor Barrier */
XEN_CPUFEATURE(IBPB_BRTYPE, 11*32+28) /*A IBPB flushes Branch Type predictions too */
-XEN_CPUFEATURE(SRSO_NO, 11*32+29) /*A Hardware not vulenrable to Speculative Return Stack Overflow */
+XEN_CPUFEATURE(SRSO_NO, 11*32+29) /*A Hardware not vulnerable to Speculative Return Stack Overflow */
+XEN_CPUFEATURE(SRSO_US_NO, 11*32+30) /*A! Hardware not vulnerable to SRSO across the User/Supervisor boundary */
+XEN_CPUFEATURE(SRSO_MSR_FIX, 11*32+31) /* MSR_BP_CFG.BP_SPEC_REDUCE available */
/* Intel-defined CPU features, CPUID level 0x00000007:1.ebx, word 12 */
XEN_CPUFEATURE(INTEL_PPIN, 12*32+ 0) /* Protected Processor Inventory Number */
--
2.46.0