summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* x86/fpu: Generalize 'init_xstate_ctx'Ingo Molnar2015-05-195-16/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | So the handling of init_xstate_ctx has a layering violation: both 'struct xsave_struct' and 'union thread_xstate' have a 'struct i387_fxsave_struct' member: xsave_struct::i387 thread_xstate::fxsave The handling of init_xstate_ctx is generic, it is used on all CPUs, with or without XSAVE instruction. So it's confusing how the generic code passes around and handles an XSAVE specific format. What we really want is for init_xstate_ctx to be a proper fpstate and we use its ::fxsave and ::xsave members, as appropriate. Since the xsave_struct::i387 and thread_xstate::fxsave aliases each other this is not a functional problem. So implement this, and move init_xstate_ctx to the generic FPU code in the process. Also, since init_xstate_ctx is not XSAVE specific anymore, rename it to init_fpstate, and mark it __read_mostly, because it's only modified once during bootup, and used as a reference fpstate later on. There's no change in functionality. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Create 'union thread_xstate' helper for fpstate_init()Ingo Molnar2015-05-194-10/+10
| | | | | | | | | | | | | | | | | fpstate_init() only uses fpu->state, so pass that in to it. This enables the cleanup we will do in the next patch. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Harmonize the names of the fpstate_init() helper functionsIngo Molnar2015-05-194-25/+30
| | | | | | | | | | | | | | | | | | | | Harmonize the inconsistent naming of these related functions: fpstate_init() finit_soft_fpu() => fpstate_init_fsoft() fx_finit() => fpstate_init_fxstate() fx_finit() => fpstate_init_fstate() # split out Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Factor out the exception error code handling codeIngo Molnar2015-05-193-86/+102
| | | | | | | | | | | | | | | | | | | Factor out the FPU error code handling code from traps.c and fpu/internal.h and move them close to each other. Also convert the helper functions to 'struct fpu *', which further simplifies them. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Remove run-once init quirksIngo Molnar2015-05-192-17/+0Star
| | | | | | | | | | | | | | | | | | | | Remove various boot quirks that came from the old code. The new code is cleanly split up into per-system and per-cpu init sequences, and system init functions are only called once. Remove the run-once quirks. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Factor out fpu/regset.h from fpu/internal.hIngo Molnar2015-05-196-14/+26
| | | | | | | | | | | | | | | | | Only a few places use the regset definitions, so factor them out. Also fix related header dependency assumptions. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Split out fpu/signal.h from fpu/internal.h for signal frame ↵Ingo Molnar2015-05-197-24/+37
| | | | | | | | | | | | | | | | | | | | handling functions Most of the FPU does not use them, so split it out and include them in signal.c and ia32_signal.c Also fix header file dependency assumption in fpu/core.c. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Move is_ia32*frame() helpers out of fpu/internal.hIngo Molnar2015-05-192-16/+16
| | | | | | | | | | | | | | | | Move them to their only user. This makes the code easier to read, the header is less cluttered, and it also speeds up the build a bit. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Merge fpu__reset() and fpu__clear()Ingo Molnar2015-05-194-20/+10Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | With recent cleanups and fixes the fpu__reset() and fpu__clear() functions have become almost identical in functionality: the only difference is that fpu__reset() assumed that the fpstate was already active in the eagerfpu case, while fpu__clear() activated it if it was inactive. This distinction almost never matters, the only case where such fpstate activation happens if if the init thread (PID 1) gets exec()-ed for the first time. So keep fpu__clear() and change all fpu__reset() uses to fpu__clear() to simpify the logic. ( In a later patch we'll further simplify fpu__clear() by making sure that all contexts it is called on are already active. ) Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Move the signal frame handling code closer to each otherIngo Molnar2015-05-194-38/+45
| | | | | | | | | | | | | | | | | | | | | Consolidate more signal frame related functions: text data bss dec filename 14108070 2575280 1634304 18317654 vmlinux.before 14107944 2575344 1634304 18317592 vmlinux.after Also, while moving it, rename alloc_mathframe() to fpu__alloc_mathframe(). Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename restore_xstate_sig() to fpu__restore_sig()Ingo Molnar2015-05-194-6/+6
| | | | | | | | | | | | | | | | | | | | restore_xstate_sig() is a misnomer: it's not limited to 'xstate' at all, it is the high level 'restore FPU state from a signal frame' function that works with all legacy FPU formats as well. Rename it (and its helper) accordingly, and also move it to the fpu__*() namespace. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Move fpu__clear() to 'struct fpu *' parameter passingIngo Molnar2015-05-193-8/+5Star
| | | | | | | | | | | | | | | | | | Do it like all other high level FPU state handling functions: they only know about struct fpu, not about the task. (Also remove a dead prototype while at it.) Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Move all the fpu__*() high level methods closer to each otherIngo Molnar2015-05-192-43/+48
| | | | | | | | | | | | | | | | | | | | | | | The fpu__*() methods are closely related, but they are defined in scattered places within the FPU code. Concentrate them, and also uninline fpu__save(), fpu__drop() and fpu__reset() to save about 5K of kernel text on 64-bit kernels: text data bss dec filename 14113063 2575280 1634304 18322647 vmlinux.before 14108070 2575280 1634304 18317654 vmlinux.after Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename restore_fpu_checking() to copy_fpstate_to_fpregs()Ingo Molnar2015-05-193-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | fpu_restore_checking() is a helper function of restore_fpu_checking(), but this is not apparent from the naming. Both copy fpstate contents to fpregs, while the fuller variant does a full copy without leaking information. So rename them to: copy_fpstate_to_fpregs() __copy_fpstate_to_fpregs() Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Synchronize the naming of drop_fpu() and fpu_reset_state()Ingo Molnar2015-05-195-17/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | drop_fpu() and fpu_reset_state() are similar in functionality and in scope, yet this is not apparent from their names. drop_fpu() deactivates FPU contents (both the fpregs and the fpstate), but leaves register contents intact in the eager-FPU case, mostly as an optimization. It disables fpregs in the lazy FPU case. The drop_fpu() method can be used to destroy FPU state in an optimized way, when we know that a new state will be loaded before user-space might see any remains of the old FPU state: - such as in sys_exit()'s exit_thread() where we know this task won't execute any user-space instructions anymore and the next context switch cleans up the FPU. The old FPU state might still be around in the eagerfpu case but won't be saved. - in __restore_xstate_sig(), where we use drop_fpu() before copying a new state into the fpstate and activating that one. No user-pace instructions can execute between those steps. - in sys_execve()'s fpu__clear(): there we use drop_fpu() in the !eagerfpu case, where it's equivalent to a full reinit. fpu_reset_state() is a stronger version of drop_fpu(): both in the eagerfpu and the lazy-FPU case it guarantees that fpregs are reinitialized to init state. This method is used in cases where we need a full reset: - handle_signal() uses fpu_reset_state() to reset the FPU state to init before executing a user-space signal handler. While we have already saved the original FPU state at this point, and always restore the original state, the signal handling code still has to do this reinit, because signals may interrupt any user-space instruction, and the FPU might be in various intermediate states (such as an unbalanced x87 stack) that is not immediately usable for general C signal handler code. - __restore_xstate_sig() uses fpu_reset_state() when the signal frame has no FP context. Since the signal handler may have modified the FPU state, it gets reset back to init state. - in another branch __restore_xstate_sig() uses fpu_reset_state() to handle a restoration error: when restore_user_xstate() fails to restore FPU state and we might have inconsistent FPU data, fpu_reset_state() is used to reset it back to a known good state. - __kernel_fpu_end() uses fpu_reset_state() in an error branch. This is in a 'must not trigger' error branch, so on bug-free kernels this never triggers. - fpu__restore() uses fpu_reset_state() in an error path as well: if the fpstate was set up with invalid FPU state (via ptrace or via a signal handler), then it's reset back to init state. - likewise, the scheduler's switch_fpu_finish() uses it in a restoration error path too. Move both drop_fpu() and fpu_reset_state() to the fpu__*() namespace and harmonize their naming with their function: fpu__drop() fpu__reset() This clearly shows that both methods operate on the full state of the FPU, just like fpu__restore(). Also add comments to explain what each function does. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/alternatives, x86/fpu: Add 'alternatives_patched' debug flag and use it ↵Ingo Molnar2015-05-193-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | in xsave_state() We'd like to use xsave_state() earlier, but its SYSTEM_BOOTING check is too imprecise. The real condition that xsave_state() would like to check is whether alternative XSAVE instructions were patched into the kernel image already. Add such a (read-mostly) debug flag and use it in xsave_state(). Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Better document fpu__clear() state handlingIngo Molnar2015-05-191-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | So prior to this fix: c88d47480d30 ("x86/fpu: Always restore_xinit_state() when use_eager_cpu()") we leaked FPU state across execve() boundaries on eagerfpu systems: $ /host/home/mingo/dump-xmm-regs-exec # XMM state before execve(): XMM0 : 000000000000dede XMM1 : 000000000000dedf XMM2 : 000000000000dee0 XMM3 : 000000000000dee1 XMM4 : 000000000000dee2 XMM5 : 000000000000dee3 XMM6 : 000000000000dee4 XMM7 : 000000000000dee5 XMM8 : 000000000000dee6 XMM9 : 000000000000dee7 XMM10: 000000000000dee8 XMM11: 000000000000dee9 XMM12: 000000000000deea XMM13: 000000000000deeb XMM14: 000000000000deec XMM15: 000000000000deed # XMM state after execve(), in the new task context: XMM0 : 0000000000000000 XMM1 : 2f2f2f2f2f2f2f2f XMM2 : 0000000000000000 XMM3 : 0000000000000000 XMM4 : 00000000000000ff XMM5 : 00000000ff000000 XMM6 : 000000000000dee4 XMM7 : 000000000000dee5 XMM8 : 0000000000000000 XMM9 : 0000000000000000 XMM10: 0000000000000000 XMM11: 0000000000000000 XMM12: 0000000000000000 XMM13: 000000000000deeb XMM14: 000000000000deec XMM15: 000000000000deed Better explain what this function is supposed to do and why. Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Initialize fpregs in fpu__init_cpu_generic()Ingo Molnar2015-05-191-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | FPU fpregs do not get initialized during bootup on secondary CPUs, on non-xsave capable CPUs. For example on one of my systems, the secondary CPU has this FPU state on bootup: x86: Booting SMP configuration: .... node #0, CPUs: #1 x86/fpu ###################### x86/fpu # FPU register dump on CPU#1: x86/fpu # ... CWD: ffff0040 x86/fpu # ... SWD: ffff0000 x86/fpu # ... TWD: ffff555a x86/fpu # ... FIP: 00000000 x86/fpu # ... FCS: 00000000 x86/fpu # ... FOO: 00000000 x86/fpu # ... FOS: ffff0000 x86/fpu # ... FP0: 02 57 00 00 00 00 00 00 ff ff x86/fpu # ... FP1: 1b e2 00 00 00 00 00 00 ff ff x86/fpu # ... FP2: 00 00 00 00 00 00 00 00 00 00 x86/fpu # ... FP3: 00 00 00 00 00 00 00 00 00 00 x86/fpu # ... FP4: 00 00 00 00 00 00 00 00 00 00 x86/fpu # ... FP5: 00 00 00 00 00 00 00 00 00 00 x86/fpu # ... FP6: 00 00 00 00 00 00 00 00 00 00 x86/fpu # ... FP7: 00 00 00 00 00 00 00 00 00 00 x86/fpu # ... SW: dadadada x86/fpu ###################### Note how CWD and TWD are off their usual init state (0x037f and 0xffff), and how FP0 and FP1 has non-zero content. This is normally not a problem, because any user-space FPU state is initalized properly - but it can complicate the use of FPU instructions in kernel code via kernel_fpu_begin()/end(): if the FPU using code does not initialize registers itself, it might generate spurious exceptions depending on which CPU it executes on. Fix this by initializing the x87 state via the FNINIT instruction. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename user_has_fpu() to fpregs_active()Ingo Molnar2015-05-194-7/+7
| | | | | | | | | | | | | | | Rename this function in line with the new FPU nomenclature. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Clarify ancient comments in fpu__restore()Ingo Molnar2015-05-191-7/+7
| | | | | | | | | | | | | | | | | | | | So this function still had ancient language about 'saving current math information' - but we haven't been doing lazy FPU saves for quite some time, we are doing lazy FPU restores. Also remove IRQ13 related comment, which we don't support anymore either. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename save_user_xstate() to copy_fpregs_to_sigframe()Ingo Molnar2015-05-191-2/+2
| | | | | | | | | | | | | | | | | | | | | Move the naming in line with existing names, so that we now have: copy_fpregs_to_fpstate() copy_fpstate_to_sigframe() copy_fpregs_to_sigframe() ... where each function does what its name suggests. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename save_xstate_sig() to copy_fpstate_to_sigframe()Ingo Molnar2015-05-194-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | Standardize the naming of save_xstate_sig() by renaming it to copy_fpstate_to_sigframe(): this tells us at a glance that the function copies an FPU fpstate to a signal frame. This naming also follows the naming of copy_fpregs_to_fpstate(). Don't put 'xstate' into the name: since this is a generic name, it's expected that the function is able to handle xstate frames as well, beyond legacy frames. xstate used to be the odd case in the x86 FPU code - now it's the common case. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Pass 'struct fpu' to fpstate_sanitize_xstate()Ingo Molnar2015-05-193-10/+9Star
| | | | | | | | | | | | | | | | | Currently fpstate_sanitize_xstate() has a task_struct input parameter, but it only uses the fpu structure from it - so pass in a 'struct fpu' pointer only and update all call sites. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Simplify fpstate_sanitize_xstate() callsIngo Molnar2015-05-192-10/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the extra layer of __fpstate_sanitize_xstate(): if (!use_xsaveopt()) return; __fpstate_sanitize_xstate(tsk); and move the check for use_xsaveopt() into fpstate_sanitize_xstate(). In general we optimize for the presence of CPU features, not for the absence of them. Furthermore there's little point in this inlining, as the call sites are not super hot code paths. Doing this uninlining shrinks the code a bit: text data bss dec hex filename 14108751 2573624 1634304 18316679 1177d87 vmlinux.before 14108627 2573624 1634304 18316555 1177d0b vmlinux.after Also remove a pointless '!fx' check from fpstate_sanitize_xstate(). Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename sanitize_i387_state() to fpstate_sanitize_xstate()Ingo Molnar2015-05-193-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | So the sanitize_i387_state() function has the following purpose: on CPUs that support optimized xstate saving instructions, an FPU fpstate might end up having partially uninitialized data. This function initializes that data. Note that the function name is a misnomer and confusing on two levels, not only is it not i387 specific at all, but it is the exact opposite: it only matters on xstate CPUs. So rename sanitize_i387_state() and __sanitize_i387_state() to fpstate_sanitize_xstate() and __fpstate_sanitize_xstate(), to clearly express the purpose and usage of the function. We'll further clean up this function in the next patch. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Move asm/xcr.h to asm/fpu/internal.hIngo Molnar2015-05-195-52/+25Star
| | | | | | | | | | | | | | | | Now that all FPU internals using drivers are converted to public APIs, move xcr.h's definitions into fpu/internal.h and remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/sha1_mb: Remove FPU internal headers from sha1_mb.cIngo Molnar2015-05-191-3/+1Star
| | | | | | | | | | | | | | | | This file only uses the public FPU APIs, so remove the xcr.h, fpu/xstate.h and fpu/internal.h headers and add the fpu/api.h include. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/serpent_avx2: Simplify the init() xfeature checksIngo Molnar2015-05-191-11/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/sha1_ssse3: Simplify the sha1_ssse3_mod_init() xfeature ↵Ingo Molnar2015-05-191-11/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | checks Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/cast6_avx: Simplify the cast6_init() xfeature checksIngo Molnar2015-05-191-11/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/sha512_ssse3: Simplify the sha512_ssse3_mod_init() ↵Ingo Molnar2015-05-191-11/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xfeature checks Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/cast5_avx: Simplify the cast5_init() xfeature checksIngo Molnar2015-05-191-11/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/serpent_avx: Simplify the serpent_init() xfeature checksIngo Molnar2015-05-191-11/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/twofish_avx: Simplify the twofish_init() xfeature checksIngo Molnar2015-05-191-11/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/camellia_aesni_avx2: Simplify the camellia_aesni_init() ↵Ingo Molnar2015-05-191-11/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xfeature checks Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/sha256_ssse3: Simplify the sha256_ssse3_mod_init() ↵Ingo Molnar2015-05-191-11/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xfeature checks Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit. - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu, crypto x86/camellia_aesni_avx: Simplify the camellia_aesni_init() ↵Ingo Molnar2015-05-191-11/+4Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xfeature checks Use the new 'cpu_has_xfeatures()' function to query AVX CPU support. This has the following advantages to the driver: - Decouples the driver from FPU internals: it's now only using <asm/fpu/api.h>. - Removes detection complexity from the driver, no more raw XGETBV instruction - Shrinks the code a bit: text data bss dec hex filename 2128 2896 0 5024 13a0 camellia_aesni_avx_glue.o.before 2067 2896 0 4963 1363 camellia_aesni_avx_glue.o.after - Standardizes feature name error message printouts across drivers There are also advantages to the x86 FPU code: once all drivers are decoupled from internals we can move them out of common headers and we'll also be able to remove xcr.h. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Move xfeature type enumeration to fpu/types.hIngo Molnar2015-05-192-28/+28
| | | | | | | | | | | | | | | | | | | So xsave.h is an internal header that FPU using drivers commonly include, to get access to the xstate feature names, amongst other things. Move these type definitions to fpu/fpu.h to allow simplification of FPU using driver code. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Enumerate xfeature bitsIngo Molnar2015-05-191-12/+26
| | | | | | | | | | | | | | | | | Transform the xstate masks into an enumerated list of xfeature bits. This removes the hard coding of XFEATURES_NR_MAX. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Simplify print_xstate_features()Ingo Molnar2015-05-191-13/+12Star
| | | | | | | | | | | | | | | | | We do a boot time printout of xfeatures in print_xstate_features(), simplify this code to make use of the recently introduced cpu_has_xfeature() method. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Introduce cpu_has_xfeatures(xfeatures_mask, feature_name)Ingo Molnar2015-05-192-1/+61
| | | | | | | | | | | | | | | | | | | | | | | | A lot of FPU using driver code is querying complex CPU features to be able to figure out whether a given set of xstate features is supported by the CPU or not. Introduce a simplified API function that can be used on any CPU type to get this information. Also add an error string return pointer, so that the driver can print a meaningful error message with a standardized feature name. Also mark xfeatures_mask as __read_only. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename fpu/xsave.c to fpu/xstate.cIngo Molnar2015-05-192-1/+1
| | | | | | | | | | | | | Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Rename fpu/xsave.h to fpu/xstate.hIngo Molnar2015-05-1914-13/+13
| | | | | | | | | | | | | | | | | | | | | | 'xsave' is an x86 instruction name to most people - but xsave.h is about a lot more than just the XSAVE instruction: it includes definitions and support, both internal and external, related to xstate and xfeatures support. As a first step in cleaning up the various xstate uses rename this header to 'fpu/xstate.h' to better reflect what this header file is about. Cc: Andy Lutomirski <luto@amacapital.net> Cc: Borislav Petkov <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Optimize fpu_copy() some more on lazy switching systemsIngo Molnar2015-05-191-8/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current fpu_copy() code on lazy switching CPUs always saves into the current fpstate and then copies it over into the child context: preempt_disable(); if (!copy_fpregs_to_fpstate(src_fpu)) fpregs_deactivate(src_fpu); preempt_enable(); memcpy(&dst_fpu->state, &src_fpu->state, xstate_size); That memcpy() can be avoided on all lazy switching setups except really old FNSAVE-only systems: change fpu_copy() to directly save into the child context, for both the lazy and the eager context switching case. Note that we still have to do a memcpy() back into the parent context in the FNSAVE case, but this won't be executed on the majority of x86 systems that got built in the last 10 years or so. Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Optimize fpu_copy()Ingo Molnar2015-05-191-1/+4
| | | | | | | | | | | | | | | | | | | Optimize fpu_copy() a bit by expanding the ->fpstate_active == 1 portion of fpu__save() into it. ( The main purpose of this change is to enable another, larger optimization that will be done in the next patch. ) Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Optimize fpu__save()Ingo Molnar2015-05-191-3/+2Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | So fpu__save() does this currently: copy_fpregs_to_fpstate(fpu); if (!use_eager_fpu()) fpregs_deactivate(fpu); ... which deactivates the FPU on lazy switching systems unconditionally. Both usecases of fpu__save() use this function to save the FPU state into a fpstate: fork()/clone() and math error signal handling. The unconditional disabling of FPU registers in the lazy switching case is probably a mistaken conversion of old FNSAVE code (that had to disable FPU registers). So speed up this code by only disabling FPU registers when absolutely necessary: when indicated by the copy_fpregs_to_fpstate() return code: if (!copy_fpregs_to_fpstate(fpu)) fpregs_deactivate(fpu); Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Simplify fpu__save()Ingo Molnar2015-05-191-5/+2Star
| | | | | | | | | | | | | | | Factor out a common call. Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Eliminate __save_fpu()Ingo Molnar2015-05-191-11/+2Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current implementation of __save_fpu(): if (use_xsave()) { xsave_state(&fpu->state.xsave); } else { fpu_fxsave(fpu); } Is actually a simplified version of copy_fpregs_to_fpstate(), if use_eager_fpu() is true. But all call sites of __save_fpu() call it only it when use_eager_fpu() is true. So we can eliminate __save_fpu() altogether and use the standard copy_fpregs_to_fpstate() function. This cleans up the code by making it use fewer variants of FPU register saving. Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Simplify __save_fpu()Ingo Molnar2015-05-192-4/+3Star
| | | | | | | | | | | | | | | | | | | | | | | | | | | __save_fpu() has this pattern: if (unlikely(system_state == SYSTEM_BOOTING)) xsave_state_booting(&fpu->state.xsave); else xsave_state(&fpu->state.xsave); ... but it does not actually get called during system bootup. So remove the complication and always call xsave_state(). To make sure this assumption is correct, add a WARN_ONCE() debug check to xsave_state(). Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/fpu: Factor out FPU hw activation/deactivationIngo Molnar2015-05-192-12/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have repeat patterns of: if (!use_eager_fpu()) clts(); ... to activate FPU registers, and: if (!use_eager_fpu()) stts(); ... to deactivate them. Encapsulate these in: __fpregs_activate_hw(); __fpregs_activate_hw(); and use them accordingly. Doing this synchronizes the idiom with the fpu->fpregs_active software-flag's handling functions, creating clear patterns of: __fpregs_activate_hw(); __fpregs_activate(fpu); etc., which improves readability. Reviewed-by: Borislav Petkov <bp@alien8.de> Cc: Andy Lutomirski <luto@amacapital.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Fenghua Yu <fenghua.yu@intel.com> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Ingo Molnar <mingo@kernel.org>
generated by cgit v1.2.3-55-g7522 (git 2.39.0) at 2024-10-12 09:30:56 +0200