目錄

• 關于pthread和kthread的說明
• 和kthread/pthread區別對待相關的數據結構
• 調用圖
• fpu load/store 在調度中的處理
• fpu load/store 在KVM中的處理
• 參考

linux 5.10.8，本文以（上下文切換時，用戶態task和內核task對fpu的load/store的區別）為主線

關于pthread和kthread的說明

??pthread時os對外提供的POSIX接口，并不是linux的概念。用戶態程序可以調用pthread的接口創建、運行和銷毀用戶態線程。在下面的內容中，用pthread表示用戶態線程，用kthread表示內核線程。在linux中，用戶態線程和內核線程都是用struct task_struct表示的。

和kthread/pthread區別對待相關的數據結構

??linux中的TCB是task_struct，定義在文件./include/linux/sched.h中，和調度時和kthread和pthread區別對待相關的域如下：

struct task_struct {/* ... */struct thread_info thread_info;/* ... *//* Per task flags (PF_*), defined further below: */unsigned int flags;/* ... */struct mm_struct *mm;struct mm_struct *active_mm;/* ... */ }/** Per process flags*/ #define PF_IDLE 0x00000002 /* I am an IDLE thread */ #define PF_EXITING 0x00000004 /* Getting shut down */ #define PF_VCPU 0x00000010 /* I'm a virtual CPU */ #define PF_WQ_WORKER 0x00000020 /* I'm a workqueue worker */ #define PF_FORKNOEXEC 0x00000040 /* Forked but didn't exec */ #define PF_MCE_PROCESS 0x00000080 /* Process policy on mce errors */ #define PF_SUPERPRIV 0x00000100 /* Used super-user privileges */ #define PF_DUMPCORE 0x00000200 /* Dumped core */ #define PF_SIGNALED 0x00000400 /* Killed by a signal */ #define PF_MEMALLOC 0x00000800 /* Allocating memory */ #define PF_NPROC_EXCEEDED 0x00001000 /* set_user() noticed that RLIMIT_NPROC was exceeded */ #define PF_USED_MATH 0x00002000 /* If unset the fpu must be initialized before use */ #define PF_USED_ASYNC 0x00004000 /* Used async_schedule*(), used by module init */ #define PF_NOFREEZE 0x00008000 /* This thread should not be frozen */ #define PF_FROZEN 0x00010000 /* Frozen for system suspend */ #define PF_KSWAPD 0x00020000 /* I am kswapd */ #define PF_MEMALLOC_NOFS 0x00040000 /* All allocation requests will inherit GFP_NOFS */ #define PF_MEMALLOC_NOIO 0x00080000 /* All allocation requests will inherit GFP_NOIO */ #define PF_LESS_THROTTLE 0x00100000 /* Throttle me less: I clean memory */ #define PF_KTHREAD 0x00200000 /* I am a kernel thread */ #define PF_RANDOMIZE 0x00400000 /* Randomize virtual address space */ #define PF_SWAPWRITE 0x00800000 /* Allowed to write to swap */ #define PF_MEMSTALL 0x01000000 /* Stalled due to lack of memory */ #define PF_UMH 0x02000000 /* I'm an Usermodehelper process */ #define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddle with cpus_mask */ #define PF_MCE_EARLY 0x08000000 /* Early kill for mce process policy */ #define PF_MEMALLOC_NOCMA 0x10000000 /* All allocation request will have _GFP_MOVABLE cleared */ #define PF_IO_WORKER 0x20000000 /* Task is an IO worker */ #define PF_FREEZER_SKIP 0x40000000 /* Freezer should not count it as freezable */ #define PF_SUSPEND_TASK 0x80000000 /* This thread called freeze_processes() and should not be frozen */

??flag字段是task的標志位，標志位所有的信息如上所示。其中PF_KTHREAD標志位置1表示task是一個kthread。

??struct mm_struct是記錄用戶虛擬地址空間的數據結構。對于內核線程，mm為NULL。由于內核線程之前可能是任何用戶層進程在執行，故用戶空間部分的內容本質上是隨機的，內核線程決不能修改其內容，故將mm設置為NULL，同時如果切換出去的是用戶進程，內核將原來進程的mm存放在新內核線程的active_mm中，因為某些時候內核必須知道用戶空間當前包含了什么。在linux的代碼實現中，這個字段和fpu的load/store沒有關系，只是和kthread和pthread的區別對待有關系。

??thread_info 定義在./arch/x86/include/asm/thread_info.h的結構如下。它的flag字段設有32個標志位，如下所示。其中TIF_NEED_FPU_LOAD標志位置1，當task被調度到，在返回用戶態之前，或者kernel需要用到fpu時，fpu會被加載到cpu上。

struct thread_info {unsigned long flags; /* low level flags */u32 status; /* thread synchronous flags */ };/** thread information flags* - these are process state flags that various assembly files* may need to access*/ #define TIF_SYSCALL_TRACE 0 /* syscall trace active */ #define TIF_NOTIFY_RESUME 1 /* callback before returning to user */ #define TIF_SIGPENDING 2 /* signal pending */ #define TIF_NEED_RESCHED 3 /* rescheduling necessary */ #define TIF_SINGLESTEP 4 /* reenable singlestep on user return*/ #define TIF_SSBD 5 /* Speculative store bypass disable */ #define TIF_SYSCALL_EMU 6 /* syscall emulation active */ #define TIF_SYSCALL_AUDIT 7 /* syscall auditing active */ #define TIF_SECCOMP 8 /* secure computing */ #define TIF_SPEC_IB 9 /* Indirect branch speculation mitigation */ #define TIF_SPEC_FORCE_UPDATE 10 /* Force speculation MSR update in context switch */ #define TIF_USER_RETURN_NOTIFY 11 /* notify kernel of userspace return */ #define TIF_UPROBE 12 /* breakpointed or singlestepping */ #define TIF_PATCH_PENDING 13 /* pending live patching update */ #define TIF_NEED_FPU_LOAD 14 /* load FPU on return to userspace */ #define TIF_NOCPUID 15 /* CPUID is not accessible in userland */ #define TIF_NOTSC 16 /* TSC is not accessible in userland */ #define TIF_IA32 17 /* IA32 compatibility process */ #define TIF_NOHZ 19 /* in adaptive nohz mode */ #define TIF_MEMDIE 20 /* is terminating due to OOM killer */ #define TIF_POLLING_NRFLAG 21 /* idle is polling for TIF_NEED_RESCHED */ #define TIF_IO_BITMAP 22 /* uses I/O bitmap */ #define TIF_FORCED_TF 24 /* true if TF in eflags artificially */ #define TIF_BLOCKSTEP 25 /* set when we want DEBUGCTLMSR_BTF */ #define TIF_LAZY_MMU_UPDATES 27 /* task is updating the mmu lazily */ #define TIF_SYSCALL_TRACEPOINT 28 /* syscall tracepoint instrumentation */ #define TIF_ADDR32 29 /* 32-bit address space on 64 bits */ #define TIF_X32 30 /* 32-bit native x86-64 binary */ #define TIF_FSCHECK 31 /* Check FS is USER_DS on return */

調用圖

??具體的linux代碼實現在后文中介紹。這里是調度和返回用戶態的過程中和fpu load/store相關的過程。圖中，箭頭線表示執行流(在前一個空心箭頭的上級函數中），而空心箭頭表示函數調用。

fpu load/store 在調度中的處理

??fpu在調度中的處理流程如下：

• shedule in /kernel/sched/core.c是調度主要的函數。

asmlinkage __visible void __sched schedule(void){/* ... */do {preempt_disable();__schedule(false);sched_preempt_enable_no_resched();} while (need_resched());/* ... */ }

• __schedule in ./kernel/sched/core.c。更新調度隊列的當前task(rq->cur)(但是此時還換棧，下文中的current指的都是棧上的current task，也就是old task)，并調用context_switch進行上下文切換。

static void __sched notrace __schedule(bool preempt){struct task_struct *prev, *next;unsigned long *switch_count;struct rq_flags rf;struct rq *rq;int cpu;cpu = smp_processor_id();rq = cpu_rq(cpu);prev = rq->curr;/* ... */next = pick_next_task(rq, prev, &rf);/* ... */if (likely(prev != next)) {/* ... */RCU_INIT_POINTER(rq->curr, next);/* ... */rq = context_switch(rq, prev, next, &rf);} else {/* ... */}/* ... */ }

• context_switch in ./kernel/sched/core.c。

/** context_switch - switch to the new MM and the new thread's register state.*/ static __always_inline struct rq * context_switch(struct rq *rq, struct task_struct *prev,struct task_struct *next, struct rq_flags *rf){/* ... *//* Here we just switch the register state and the stack. */switch_to(prev, next, prev);/* ... */}

• switch_to in ./arch/x86/include/asm/switch_to.h。

#define switch_to(prev, next, last) \ do { \prepare_switch_to(next); \\((last) = __switch_to_asm((prev), (next))); \ } while (0)

• __switch_to_asm in ./arch/x86/kernel/process_64.c。對fpu的store/load分兩步進行：首先，它調用test_thread_flag(TIF_NEEED_FPU_LOAD) 判斷是否需要store fpu(old thread)，然后調用switch_fpu_prepare store fpu；其次，調用switch_fpu_finish把TIF_NEED_FPU_LOAD位置1，這樣當新的thread返回用戶態，或者需要用到fpu時，就會load fpu。

__visible __notrace_funcgraph struct task_struct * __switch_to(struct task_struct *prev_p, struct task_struct *next_p){struct thread_struct *prev = &prev_p->thread;struct thread_struct *next = &next_p->thread;struct fpu *prev_fpu = &prev->fpu;struct fpu *next_fpu = &next->fpu;int cpu = smp_processor_id();/* ... */if (!test_thread_flag(TIF_NEED_FPU_LOAD))switch_fpu_prepare(prev_fpu, cpu);/* ... */switch_fpu_finish(next_fpu);/* ... */ }

• test_thread_flag in ./include/linux/thread_info.h，檢查對應的位是否置1，這個調用鏈中是檢查TIF_NEED_FPU_LOAD。

#define test_thread_flag(flag) \test_ti_thread_flag(current_thread_info(), flag)static inline int test_ti_thread_flag(struct thread_info *ti, int flag){return test_bit(flag, (unsigned long *)&ti->flags); }#define current_thread_info() ((struct thread_info *)current)/* in ./arch/x86/include/asm/current.h */ DECLARE_PER_CPU(struct task_struct *, current_task); static __always_inline struct task_struct *get_current(void){return this_cpu_read_stable(current_task); } #define current get_current()/* in ./include/asm-generic/bitops/non-atomic.h */ /*** test_bit - Determine whether a bit is set* @nr: bit number to test* @addr: Address to start counting from*/ static inline int test_bit(int nr, const volatile unsigned long *addr) {return 1UL & (addr[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG-1))); }

• switch_fpu_prepare in ./arch/x86/include/asm/fpu/internal.h。調用copy_fpregs_to_fpstate進行fpu store。

/** FPU state switching for scheduling.** This is a two-stage process:** - switch_fpu_prepare() saves the old state.* This is done within the context of the old process.** - switch_fpu_finish() sets TIF_NEED_FPU_LOAD; the floating point state* will get loaded on return to userspace, or when the kernel needs it.** If TIF_NEED_FPU_LOAD is cleared then the CPU's FPU registers* are saved in the current thread's FPU register state.** If TIF_NEED_FPU_LOAD is set then CPU's FPU registers may not* hold current()'s FPU registers. It is required to load the* registers before returning to userland or using the content* otherwise.** The FPU context is only stored/restored for a user task and* PF_KTHREAD is used to distinguish between kernel and user threads.*/ static inline void switch_fpu_prepare(struct fpu *old_fpu, int cpu){if (static_cpu_has(X86_FEATURE_FPU) && !(current->flags & PF_KTHREAD)) {if (!copy_fpregs_to_fpstate(old_fpu))old_fpu->last_cpu = -1;elseold_fpu->last_cpu = cpu;/* But leave fpu_fpregs_owner_ctx! */trace_x86_fpu_regs_deactivated(old_fpu);} }// in ./arch/x86/include/asm/cpufeatures.h /* Intel-defined CPU features, CPUID level 0x00000001 (EDX), word 0 */ #define X86_FEATURE_FPU ( 0*32+ 0) /* Onboard FPU */

• switch_fpu_finish in ./arch/x86/include/asm/fpu/internal.h。設置新 task的TIF_NEED_FPU_LOAD標志位(在返回用戶態之前，會檢查并根據結果load fpu)。

/** Load PKRU from the FPU context if available. Delay loading of the* complete FPU state until the return to userland.*/ static inline void switch_fpu_finish(struct fpu *new_fpu){/* ... */if (!static_cpu_has(X86_FEATURE_FPU))return;set_thread_flag(TIF_NEED_FPU_LOAD);/* ... */ }

• prepare_exit_to_usermode in ./arch/x86/entry/common.c。當調度結束返回用戶態之前，會調用prepare_exit_to_usermode，它會調用switch_fpu_return 完成fpu load的操作。

/* Called with IRQs disabled. */ __visible inline void prepare_exit_to_usermode(struct pt_regs *regs){struct thread_info *ti = current_thread_info();u32 cached_flags;/* ... */cached_flags = READ_ONCE(ti->flags);/* ... */if (unlikely(cached_flags & _TIF_NEED_FPU_LOAD))switch_fpu_return();/* ... */ }

• switch_fpu_return in ./arch/x86/kernel/fpu/core.c。它調用__fregs_load_activate in ./arch/x86/include/asm/fpu/internal.h進行fpu的加載。首先，如果當前線程是kthread，就不會load fpu；然后，load fpu；最后，清除TIF_NEED_FPU_LOAD標志位。清除操作是因為調度時，會根據這個標志位進行store fpu的操作。如果是kthread，這個標志位就不會被清除，所以kthread被調度出去的時候不會保存fpu的狀態；kthread被調度進來的時候也不會恢復fpu的狀態。如果是pthread，這個標志位會被清除，所以pthread被調度出去的時候會保存fpu的狀態；pthread被調度進來的時候會在返回用戶態之前恢復fpu的狀態。

/** Load FPU context before returning to userspace.*/ void switch_fpu_return(void){if (!static_cpu_has(X86_FEATURE_FPU))return;__fpregs_load_activate(); } EXPORT_SYMBOL_GPL(switch_fpu_return);/** Internal helper, do not use directly. Use switch_fpu_return() instead.*/ static inline void __fpregs_load_activate(void){struct fpu *fpu = &current->thread.fpu;int cpu = smp_processor_id();if (WARN_ON_ONCE(current->flags & PF_KTHREAD))return;if (!fpregs_state_valid(fpu, cpu)) {copy_kernel_to_fpregs(&fpu->state);fpregs_activate(fpu);fpu->last_cpu = cpu;}clear_thread_flag(TIF_NEED_FPU_LOAD); }

fpu load/store 在KVM中的處理

??當vm-exit發生時，在內核中會執行當前線程(也就是vcpu對應的線程，是一個普通的線程，可以是kthread，也可以是pthread。但在QEMU/KVM中，這個線程是一個pthread)。然后，vcpu線程和別的線程一樣參與調度，而在調度的過程中對fpu load/store的處理和前文描述的一樣。不同的是，kvm是要vm-entry到客戶機執行，而非返回到用戶態。所以需要和返回用戶態一樣，在返回客戶機執行前執行load fpu的操作。

• vcpu_enter_guest in ./arch/x86/kvm/x86.c

/** Returns 1 to let vcpu_run() continue the guest execution loop without* exiting to the userspace. Otherwise, the value will be returned to the* userspace.*/ static int vcpu_enter_guest(struct kvm_vcpu *vcpu){/* .... */if (test_thread_flag(TIF_NEED_FPU_LOAD))switch_fpu_return();/* ... */kvm_x86_ops->run(vcpu);/* ... */ }

??根據以上的描述。如果vcpu對應的thread是kthread，那么到那個kthread被調度出去時，os調度器不會執行store fpu的操作，這就需要在vm-exit手動的執行fpu store，以防被調度出去時，客戶機需要的fpu state被修改。如果vcpu對應的thread選擇為pthread，那么當pthread被調度出去時，os調度器會執行store fpu的操作；當要返回客戶機之前，需要kvm執行load fpu的操作。

參考

參考內容中的表述并不嚴謹
Linux內核線程kernel thread詳解–Linux進程的管理與調度

總結

以上是生活随笔為你收集整理的linux 上下文切换时对用户task和内核task区别对待——针对fpu的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。