linux中的memory management和page mapping
生活随笔
收集整理的這篇文章主要介紹了
linux中的memory management和page mapping
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
1 首先要說的最簡單的是在一個process在運行的時候，它看到的內存是這個樣子的。3G以后是給kernel使用的運行和動態分配的內存的空間，注意因為是process所看到的，下面全部都是虛擬地址空間。
如下：
?2 然后需要說的是Linux Physical Memory Layout
下面這段話解釋了為什么linux不能占用所有的Ram內存：
several peculiarities that must be taken into account.
For example:
1 Page frame 0 is used by BIOS to store the system hardware configuration detected during the
Power-On Self-Test(POST); the BIOS of many laptops, moreover, writes data on this page frame
even after the system is initialized.
2 Physical addresses ranging from 0x000a0000 to 0x000fffff are usually reserved to BIOS
routines and to map the internal memory of ISA graphics cards. This area is the well-known hole from
640 KB to 1 MB in all IBM-compatible PCs: the physical addresses exist but they are reserved, and
the corresponding page frames cannot be used by the operating system.
3 Additional page frames within the first megabyte may be reserved by specific computer models. For
example, the IBM ThinkPad maps the 0xa0 page frame into the 0x9f one.
所以總之一句話：前1M的內存存儲了BIOS和其他一些硬件信息。所以Linux代碼物理開始地址在1M處。
在不考慮virtual address也就是不考慮使用page table的時候，kernel的物理占用如下圖所示：
圖中各個段的含義都已經很明確了。kernel物理內存 [_text? _end].
具體的值可以不用細扣，因為不同的架構上，不同的內核編譯后可能位置和大小可能有偏差。
比如我的?linux-2.6.38.8版本的內核編譯后產生的System.map文件中_text 和 _end的地址為：
- 0xc0400000 --- _text
- 0xc0cc5000 --- _end
首先說明這是內核使用page table之后的虛擬內存的地址。
圖中_text在虛擬內存中：起始于3G + 偏移量4M。
_end在虛擬內存中：起始于3G + 偏移量超過12M。
這說明我用的內核編譯后比上圖中的內核要大一些。
3 Kernel Page Tables
因為Kernel加載完初始完后，就會進入保護模式，所以在往下走之前需要了解保護模式，并且了解Linux的Page Table的使用，如下可以是Linux的頁表的形式，每個Process和Kernel都有一個Page Table：
然后Process和Kernel的Page Table的關系是怎么樣的呢？請看這句引用：
1 The kernel maintains a set of page tables for its own use, rooted at a so-called master kernel Page Global Directory.2 After system initialization, this set of page tables is never directly used by any process or kernel thread;
3 rather, the highest entries of the master kernel Page Global Directory are the reference model for the corresponding entries of the Page Global Directories of every regular process in the system.
將3這句話復制出來加以強調：
the highest entries of the master kernel Page Global Directory arethe reference model for the corresponding entries of the Page Global Directories of
every regular process in the system.
--------------------------
4 加入頁表后，具體我們分為兩部分來講，
第1：Kernel Page Table中各映射了些什么東西？第2：Kernel是如何完成這些映射的？
第1：Kernel Page Table中各映射了些什么東西？就是Kernel在運行的時候使用的Page Table。
依次介紹下：
- Physical memory mapping ---- 這一塊是最基本的內存映射，
- 先假設內存在0-896M（1G - 128M）之間，那么在初始化的時候，0x0 - 896M(physical address) ----(3G + 0x0) - (3G + 896M)[Linear address]了。Kernel的function variable地址在編譯的時候就確定好了為3G以后的Virtual address.因此Kernel是假設自己有1G的虛擬內存可以使用的，頁不夠就swap【swap比較復雜，先假設自己知道，也可以先假設內存足夠】。
- 如果RAM實際大小大于896M，那么在訪問高地址的時候，動態的remap【section later will discuss it】。
- Fix-mapped linear addresses. ---- 只是知道這一塊可以被映射到任何的內存，【不是太清楚用途，先放一放】
- Persistent kernel mappings ----- Starting from PKMAP_BASE we find the linear addresses used for the persistent kernel mapping of high-memory page frames.
- vmalloc area ----- Linux provides a mechanism via vmalloc() where non-contiguous physically memory can be used that is contiguous in virtual memory.【見下面non-contiguous memory allocation.】
?------------------------------------------------------------------------------------------------
?------------------------------------------------------------------------------------------------
Kernel Mappings of High-Memory page Frames
我想利用這個dynamic kernel-mapping來理解，linear address與physical address 的對應關系的，以及內核是如何keep track of physical page frame including low-memory and high memory.
1 直接用一段話來說明Kernel Mapping存在的必要性。
1 Where to store map page table(其實上圖中有)The linear address that corresponds to the end of the directly mapped physical memory, and thus to the
beginning of the high memory, is stored in the high_memory variable, which is set to 896 MB.
2 Page frames above the 896 MB boundary are not generally mapped in the fourth gigabyte of
the kernel linear address spaces, so the kernel is unable to directly access them.
3 This implies that each page allocator function that returns the linear address of the
assigned page frame doesn't work for high-memory page frames, that is, for page frames in
the ZONE_HIGHMEM memory zone
所以說low-memory本來就被映射了，所以不需要remap。high-memory因為沒有被page table映射，所以需要在用到的時候動態的申請remap。
2 第一種方法：Permanent kernel mappings(如上圖的persistent kernel mappings位置)
用于映射的基本變量和數據結構：
- pkmap_page_table ------- stores the address of this Page Table
- LAST_PKMAP ------? macro yields the number of Page Table entries.
- pkmap_count ------ array in kernel 原型為：int pkmap_count[LAST_PKMAP].
The pkmap_count array includes LAST_PKMAP counters, one for each entry of the pkmap_page_table Page Table
用于記錄counter。
1 The counter is 0
The corresponding Page Table entry does not map any high-memory page frame and is usable.
2 The counter is 1
The corresponding Page Table entry map any high-memory page frame, but it cannot be
used because the corresponding TLB entry has not been flushed since its last usage.
表明這個線性地址被映射過了，可是現在還沒有模塊使用它，它屬于閑置資源，如果暫時資源不夠就對這種資源進行回收。
3 The counter is n (greater than 1)
The corresponding Page Table entry maps a high-memory page frame, which is used by exactly n - 1
kernel components. - page_address_htable ----- This table contains one page_address_map data structure for each page frame in high memory that is currently mapped.
- page_address_map ----- prototype 如下：
struct page_address_map {
struct page *page;
void *virtual;
struct list_head list;
}; - page_address( ) function ----- returns the linear address associated with the page frame, or NULL if the
page frame is in high memory and is not mapped. - struct page ----- State information of a page frame is kept in a page descriptor of type page. All page descriptors are stored in the mem_map array.即是說physical address中的每一個page frame在內核的初始化數據中都有對應的一個struct page數據結構。kernel就是通過對這些struct page類型的page descriptor調度和存儲信息的。就像進程的基本信息都存放在struct task中一樣。還有下面這句話，所以說struct page是物理上的RAM的每一個page在kernel中的數據結構的代表：
The kernel must keep track of the current status of each page frame. For instance, it must
be able to distinguish the page frames that are used to contain pages that belong to
processes from those that contain kernel code or kernel data structures. Similarly, it must
be able to determine whether a page frame in dynamic memory is free. A page frame in
dynamic memory is free if it does not contain any useful data. It is not free when the page
frame contains data of a User Mode process, data of a software cache, dynamically
allocated kernel data structures, buffered data of a device driver, code of a kernel module,
and so on
?首先要說明的是kernel對page的引用是這樣的：
假設Kernel當前正在操作一個struct page，那么當他想得到這個page的線性地址也就是虛擬地址的時候，調用page_address(page)返回它的線性地址。當然如果它是low_memory或者它是high_memory并且已經被映射。
如：_ _va((unsigned long)(page? -? mem_map)? <<? 12) ------ low memory這樣得到線性地址。
下面的偽代碼主要是解釋remap是如何進行的，不解釋，具體參看書本<Understanding the linux kernel>：
void * kmap(struct page * page){
if (!PageHighMem(page))
return page_address(page);
return kmap_high(page);
}
void * kmap_high(struct page * page)
{
unsigned long vaddr;
spin_lock(&kmap_lock);
vaddr = (unsigned long) page_address(page);
if (!vaddr)
vaddr = map_new_virtual(page);
pkmap_count[(vaddr-PKMAP_BASE) >> PAGE_SHIFT]++;
spin_unlock(&kmap_lock);
return (void *) vaddr;
} View Code 1 for (;;) {
2 int count;
3 DECLARE_WAITQUEUE(wait, current);
4 for (count = LAST_PKMAP; count > 0; --count) {
5 last_pkmap_nr = (last_pkmap_nr + 1) & (LAST_PKMAP - 1);
6 if (!last_pkmap_nr) {
7 flush_all_zero_pkmaps( );
8 count = LAST_PKMAP;
9 }
10 if (!pkmap_count[last_pkmap_nr]) {
11 unsigned long vaddr = PKMAP_BASE +
12 (last_pkmap_nr << PAGE_SHIFT);
13 set_pte(&(pkmap_page_table[last_pkmap_nr]),
14 mk_pte(page, _ _pgprot(0x63)));
15 pkmap_count[last_pkmap_nr] = 1;
16 set_page_address(page, (void *) vaddr);
17 return vaddr;
18 }
19 }
20 current->state = TASK_UNINTERRUPTIBLE;
21 add_wait_queue(&pkmap_map_wait, &wait);
22 spin_unlock(&kmap_lock);
23 schedule( );
24 remove_wait_queue(&pkmap_map_wait, &wait);
25 spin_lock(&kmap_lock);
26 if (page_address(page))
27 return (unsigned long) page_address(page);
28 }
3 Temporary Kernel Mappings
Temporary kernel Mappings 和Permanent kernel mappings中有一個比較：
memory is done using the functions kmap(), kunmap(), kmap_atomic() and kunmap_atomic().
2 The function kmap() gives you a persistant mapping, ie. one that will
still be there after you schedule and/or move to another CPU.
However, this kind of mapping is allocated under a global lock, which can be a bottleneck on SMP systems.
The kmap() function is discouraged.
3 Good SMP scalability can be obtained by using kmap_atomic(), which is lockless.
The reason kmap_atomic() can run without any locks is that the page is mapped to a fixed address
which is private to the CPU on which you run. Of course, this means that you can not schedule between setting up
such a mapping and using it, since another process running on the same CPU might also need the same address!
This is the highmem mapping type used most in the 2.6 kernel.
Fix-mapped 的一些數據結構：
- enum fixed-address ----- 主要用于內核編譯的時候確定virtual 地址，它還包括很多其他的用途，但是這里的Temporary kernal mapping只用到了FIX_KMAP_BEGIN和FIX_KMAP_END。以下是它的數據結構定義：。
?Here we define all the compile-time 'special' virtual
?addresses. The point is to have a constant address at
?compile time, but to set the physical address only
?in the boot process. We allocate these special addresses
?from the end of virtual memory (0xfffff000) backwards.
enum fixed_addresses{
....
#ifdef CONFIG_HIGHMEM
FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */
FIX_KMAP_END = FIX_KMAP_BEGIN+(KM_TYPE_NR*NR_CPUS)-1,
#endif
....
} - enum km_type --- 主要用于訪問high_memory的remap。
1 Each CPU has its own set of 13 windows, represented by the enum km_type data structure.
2 The kernel must ensure that the same window is never used by two kernel control paths at the same time.
Thus, each symbol in the km_type structure is dedicated to one kernel component and is named after the
component. The last symbol, KM_TYPE_NR, does not represent a linear address by itself, but yields the
number of different windows usable by every CPU。
以上的意思是：模塊總共可能有13個control path（kernel component）同時運行，于是將這13個control path各分一個window
（即一個page table entry）。這樣就不用加鎖，不會出現沖突了。同時如果是smp, 每個cpu都有13個window。
【雖然暫時不知道為什么會有13個control path？但以后會理解的】
下面這段代碼就是使用fixed_addresses and km_type來進行page的替換，將type轉換成cpu對應的window的linear address, 然后修改page table：
void * kmap_atomic(struct page * page, enum km_type type)
{
enum fixed_addresses idx;
unsigned long vaddr;
current_thread_info( )->preempt_count++;
if (!PageHighMem(page))
return page_address(page);
idx = type + KM_TYPE_NR * smp_processor_id( );
vaddr = fix_to_virt(FIX_KMAP_BEGIN + idx);
set_pte(kmap_pte-idx, mk_pte(page, 0x063));
_ _flush_tlb_single(vaddr);
return (void *) vaddr;
}
------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------
?ps：
1 ZONE_DMAContains page frames of memory below 16 MB
2 ZONE_NORMAL
Contains page frames of memory at and above 16 MB and below 896 MB
3 ZONE_HIGHMEM
Contains page frames of memory at and above 896 MB
-----------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------
Linear Addresses of Noncontiguous Memory Areas?
Linux provides a mechanism via vmalloc() where non-contiguous physically memory can be used that is contiguous in virtual memory.
主要是如果系統中連續的內存不夠的時候，使用vmalloc(),可以在high_memory中分配一些零碎的page，使得這些page在physical memory是離散的，使用page table將其映射成virtual memory是連續的。
get_vm_area() ------ looks for a free range of linear addresses between VMALLOC_START and VMALLOC_END.(就是說分配一塊虛擬地址)，此函數的主要功能就是。
下面是申請物理上的page，并且映射為virtual上連續的page，讀者讀的時候即使有些不理解的地方，大體上就是這個樣子，可以暫時不求甚解。
void * vmalloc(unsigned long size){
struct vm_struct *area;
struct page **pages;
unsigned int array_size, i;
size = (size + PAGE_SIZE - 1) & PAGE_MASK;
area = get_vm_area(size, VM_ALLOC); ------------ 【分配虛擬內存地址】
if (!area)
return NULL;
area->nr_pages = size >> PAGE_SHIFT;
array_size = (area->nr_pages * sizeof(struct page *));
area->pages = pages = kmalloc(array_size, GFP_KERNEL); ---------- 【申請存儲struct page *的指針數組】
if (!area_pages) {
remove_vm_area(area->addr);
kfree(area);
return NULL;
}
memset(area->pages, 0, array_size);
for (i=0; i<area->nr_pages; i++) {
area->pages[i] = alloc_page(GFP_KERNEL|_ _GFP_HIGHMEM); -------- 【在高地址處分配物理上存在的page，其實是返回struct page * 的指針】
if (!area->pages[i]) {
area->nr_pages = i;
fail: vfree(area->addr);
return NULL;
}
}
if (map_vm_area(area, _ _pgprot(0x63), &pages)) ---------- 【在page table做映射，如果存在就修改，不存在就生成page table的各級表項】
goto fail;
return area->addr; ------- 【返回虛擬地址】
}
?
?
?
?
?
?
?
?
?
?
?
?
總結
以上是生活随笔為你收集整理的linux中的memory management和page mapping的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 梦到高僧大德预示什么
- 下一篇： Ubuntu安装BackExec Rem