NetLink Communication Mechanism And Netlink Sourcecode Analysis
catalog
1. Netlink簡(jiǎn)介 2. Netlink Function API Howto 3. Generic Netlink HOWTO kernel API 4. RFC 3549 Linux Netlink as an IP Services Protocol 5. sendmsg、recvmsg In User Space 6. kernel_recvmsg、kernel_sendmsg In Kernel Space 7. NetLink Sockets C++ Library 8. Netlink Protocol Library Suite (libnl)?
1. Netlink簡(jiǎn)介
Netlink is a flexible, robust, wire-format communications channel typically used for kernel to user communication although it can also be used for user to user and kernel to kernel communications. Netlink communication channels are associated with families or "busses", where each bus deals with a specific service; for example
1. 路由daemon(NETLINK_ROUTE) 2. 1-wire子系統(tǒng)(NETLINK_W1) 3. 用戶態(tài)socket協(xié)議(NETLINK_USERSOCK) 4. 防火墻(NETLINK_FIREWALL) 5. socket監(jiān)視(NETLINK_INET_DIAG) 6. netfilter日志(NETLINK_NFLOG) 7. ipsec安全策略(NETLINK_XFRM) 8. SELinux事件通知(NETLINK_SELINUX) 9. iSCSI子系統(tǒng)(NETLINK_ISCSI) 10. 進(jìn)程審計(jì)(NETLINK_AUDIT) 11. 轉(zhuǎn)發(fā)信息表查詢(NETLINK_FIB_LOOKUP) 12. netlink connector(NETLINK_CONNECTOR) 13. netfilter子系統(tǒng)(NETLINK_NETFILTER) 14. IPv6防火墻(NETLINK_IP6_FW) 15. DECnet路由信息(NETLINK_DNRTMSG) 16. 內(nèi)核事件向用戶態(tài)通知(NETLINK_KOBJECT_UEVENT) 17. 通用netlink(NETLINK_GENERIC)Netlink相對(duì)于系統(tǒng)調(diào)用，ioctl以及/proc文件系統(tǒng)而言具有以下優(yōu)點(diǎn)
1. 為了使用netlink，用戶僅需要在include/linux/netlink.h中增加一個(gè)新類型的netlink協(xié)議定義即可，如 #define NETLINK_MYTEST 17 然后，內(nèi)核和用戶態(tài)應(yīng)用就可以立即通過(guò) socket API 使用該 netlink 協(xié)議類型進(jìn)行數(shù)據(jù)交換。但系統(tǒng)調(diào)用需要增加新的系統(tǒng)調(diào)用，ioctl 則需要增加設(shè)備或文件， 那需要不少代碼，proc 文件系統(tǒng)則需要在 /proc 下添加新的文件或目錄，那將使本來(lái)就混亂的 /proc 更加混亂 2. netlink是一種異步通信機(jī)制，在內(nèi)核與用戶態(tài)應(yīng)用之間傳遞的消息保存在socket緩存隊(duì)列中，發(fā)送消息只是把消息保存在接收者的socket的接 收隊(duì)列，而不需要等待接收者收到消息，但系統(tǒng)調(diào)用與 ioctl 則是同步通信機(jī)制，如果傳遞的數(shù)據(jù)太長(zhǎng)，將影響調(diào)度粒度 3．使用 netlink 的內(nèi)核部分可以采用模塊的方式實(shí)現(xiàn)，使用 netlink 的應(yīng)用部分和內(nèi)核部分沒(méi)有編譯時(shí)依賴，但系統(tǒng)調(diào)用就有依賴，而且新的系統(tǒng)調(diào)用的實(shí)現(xiàn)必須靜態(tài)地連接到內(nèi)核中，它無(wú)法在模塊中實(shí)現(xiàn)，使用新系統(tǒng)調(diào)用的應(yīng)用在編譯時(shí)需要依賴內(nèi)核 4．netlink 支持多播，內(nèi)核模塊或應(yīng)用可以把消息多播給一個(gè)netlink組，屬于該neilink 組的任何內(nèi)核模塊或應(yīng)用都能接收到該消息，內(nèi)核事件向用戶態(tài)的通知機(jī)制就使用了這一特性，任何對(duì)內(nèi)核事件感興趣的應(yīng)用都能收到該子系統(tǒng)發(fā)送的內(nèi)核事件 5．內(nèi)核可以使用 netlink 首先發(fā)起會(huì)話(雙向的)，但系統(tǒng)調(diào)用和 ioctl 只能由用戶應(yīng)用發(fā)起調(diào)用 6．netlink 使用標(biāo)準(zhǔn)的 socket API，因此很容易使用，但系統(tǒng)調(diào)用和 ioctl則需要專門的培訓(xùn)才能使用0x2: Netllink通信流程
從用戶態(tài)-內(nèi)核態(tài)交互的角度來(lái)看，Netlink的通信流程如下
1. 應(yīng)用程序?qū)⒋l(fā)送的數(shù)據(jù)通過(guò)sendmsg()傳給Netlink，Netlink進(jìn)行"組包"，這實(shí)際上是一次內(nèi)存拷貝 2. Netlink在buffer滿之后，即組包完成，將消息一次性進(jìn)行"穿透拷貝"，即copy_from_user、copy_to_user，這是一次代價(jià)較高的系統(tǒng)調(diào)用 3. 內(nèi)核模塊從Netlink的buffer逐個(gè)取出數(shù)據(jù)包，即拆包，這個(gè)過(guò)程可以串行的實(shí)現(xiàn)，也可以異步地實(shí)現(xiàn)Relevant Link:
http://www.linuxfoundation.org/collaborate/workgroups/networking/netlink?
2. Netlink Function API Howto
0x1: User Space
用戶態(tài)應(yīng)用使用標(biāo)準(zhǔn)的socket APIs，socket()、bind()、sendmsg()、recvmsg()、close()就能很容易地使用netlink socket
socket(AF_NETLINK, SOCK_RAW, netlink_type) 1. 參數(shù)1: 1) AF_NETLINK2) PF_NETLINK //在 Linux 中，它們倆實(shí)際為一個(gè)東西，它表示要使用netlink2. 參數(shù)2:1) SOCK_RAW2) SOCK_DGRAM3. 參數(shù)3: 指定Netlink協(xié)議類型 #define NETLINK_ROUTE 0 /* Routing/device hook */ #define NETLINK_W1 1 /* 1-wire subsystem */ #define NETLINK_USERSOCK 2 /* Reserved for user mode socket protocols */ #define NETLINK_FIREWALL 3 /* Firewalling hook */ #define NETLINK_INET_DIAG 4 /* INET socket monitoring */ #define NETLINK_NFLOG 5 /* netfilter/iptables ULOG */ #define NETLINK_XFRM 6 /* ipsec */ #define NETLINK_SELINUX 7 /* SELinux event notifications */ #define NETLINK_ISCSI 8 /* Open-iSCSI */ #define NETLINK_AUDIT 9 /* auditing */ #define NETLINK_FIB_LOOKUP 10 #define NETLINK_CONNECTOR 11 #define NETLINK_NETFILTER 12 /* netfilter subsystem */ #define NETLINK_IP6_FW 13 #define NETLINK_DNRTMSG 14 /* DECnet routing messages */ #define NETLINK_KOBJECT_UEVENT 15 /* Kernel messages to userspace */ #define NETLINK_GENERIC 16 //NETLINK_GENERIC是一個(gè)通用的協(xié)議類型，它是專門為用戶使用的，因此，用戶可以直接使用它，而不必再添加新的協(xié)議類型對(duì)于每一個(gè)netlink協(xié)議類型，可以有多達(dá) 32多播組，每一個(gè)多播組用一個(gè)位表示，netlink 的多播特性使得發(fā)送消息給同一個(gè)組僅需要一次系統(tǒng)調(diào)用，因而對(duì)于需要多撥消息的應(yīng)用而言，大大地降低了系統(tǒng)調(diào)用的次數(shù)
bind(fd, (struct sockaddr*)&nladdr, sizeof(struct sockaddr_nl)); 函數(shù)bind()用于把一個(gè)打開(kāi)的netlink socket與netlink源socket地址綁定在一起。netlink socket的地址結(jié)構(gòu)如下struct sockaddr_nl {//字段 nl_family 必須設(shè)置為 AF_NETLINK 或著 PF_NETLINK sa_family_t nl_family;//字段 nl_pad 當(dāng)前沒(méi)有使用，因此要總是設(shè)置為 0unsigned short nl_pad;/*字段 nl_pid 為接收或發(fā)送消息的進(jìn)程的 ID1. nl_pid = 0: 消息接收者為內(nèi)核或多播組2. nl_pid != 0: nl_pid 實(shí)際上未必是進(jìn)程 ID，它只是用于區(qū)分不同的接收者或發(fā)送者的一個(gè)標(biāo)識(shí)，用戶可以根據(jù)自己需要設(shè)置該字段 */__u32 nl_pid;/*nl_groups 用于指定多播組，bind 函數(shù)用于把調(diào)用進(jìn)程加入到該字段指定的多播組1. nl_groups = 0: 該消息為單播消息，調(diào)用者不加入任何多播組2. nl_groups != 0: 多播消息*/__u32 nl_groups; };值得注意的是，傳遞給 bind 函數(shù)的地址的 nl_pid 字段應(yīng)當(dāng)設(shè)置為本進(jìn)程的進(jìn)程 ID，這相當(dāng)于 netlink socket 的本地地址。但是，對(duì)于一個(gè)進(jìn)程的多個(gè)線程使用 netlink socket 的情況，字段 nl_pid 則可以設(shè)置為其它的值，如
pthread_self() << 16 | getpid();字段 nl_pid 實(shí)際上未必是進(jìn)程 ID,它只是用于區(qū)分不同的接收者或發(fā)送者的一個(gè)標(biāo)識(shí)，用戶可以根據(jù)自己需要設(shè)置該字段
關(guān)于使用netlink api及其相關(guān)參數(shù)，請(qǐng)參閱另一篇文章 http://www.cnblogs.com/LittleHann/p/3867214.html //搜索：user_client.c(用戶態(tài)程序)從netlink發(fā)送消息相關(guān)的數(shù)據(jù)結(jié)構(gòu)中我們可以看出netlink發(fā)送消息的邏輯
1. 對(duì)于程序員來(lái)說(shuō)，發(fā)送消息的系統(tǒng)調(diào)用接口只有sendmsg，每次調(diào)用sendmsg只需要傳入struct msghdr結(jié)構(gòu)體的實(shí)例即可 2. 對(duì)于每個(gè)struct msghdr結(jié)構(gòu)的實(shí)例來(lái)說(shuō)，都必須指定struct iovec成員，即所有單個(gè)的消息都會(huì)被"掛入"一個(gè)"隊(duì)列"中，用于緩存集中發(fā)送 3. 每個(gè)代表"消息隊(duì)列"的struct iovec結(jié)構(gòu)體實(shí)例，都必須指定struct nlmsghdr成員，即消息頭，用于實(shí)現(xiàn)"多路復(fù)用"和"多路分解"0x2: Kernel Space
內(nèi)核使用netlink需要專門的API，這完全不同于用戶態(tài)應(yīng)用對(duì)netlink的使用。如果用戶需要增加新的netlink協(xié) 議類型，必須通過(guò)修改linux/netlink.h來(lái)實(shí)現(xiàn)，當(dāng)然，目前的netlink實(shí)現(xiàn)已經(jīng)包含了一個(gè)通用的協(xié)議類型 NETLINK_GENERIC以方便用戶使用，用戶可以直接使用它而不必增加新的協(xié)議類型
在內(nèi)核中，為了創(chuàng)建一個(gè)netlink socket用戶需要調(diào)用如下函數(shù) struct sock *netlink_kernel_create(int unit, void (*input)(struct sock *sk, int len));當(dāng)內(nèi)核中發(fā)送netlink消息時(shí)，也需要設(shè)置目標(biāo)地址與源地址，linux/netlink.h中定義了一個(gè)宏
struct netlink_skb_parms {/*Skb credentials struct scm_creds {//pid表示消息發(fā)送者進(jìn)程ID，也即源地址，對(duì)于內(nèi)核，它為 0u32 pid;kuid_t uid;kgid_t gid;};struct scm_creds creds; /*字段portid表示消息接收者進(jìn)程 ID，也即目標(biāo)地址，如果目標(biāo)為組或內(nèi)核，它設(shè)置為 0，否則 dst_group 表示目標(biāo)組地址，如果它目標(biāo)為某一進(jìn)程或內(nèi)核，dst_group 應(yīng)當(dāng)設(shè)置為 0 */__u32 portid;__u32 dst_group;__u32 flags;struct sock *sk; }; #define NETLINK_CB(skb) (*(struct netlink_skb_parms*)&((skb)->cb))在內(nèi)核中，模塊調(diào)用函數(shù) netlink_unicast 來(lái)發(fā)送單播消息
int netlink_unicast(struct sock *sk, struct sk_buff *skb, u32 pid, int nonblock);Relevant Link:
http://www.cnblogs.com/iceocean/articles/1594195.html http://blog.csdn.net/zcabcd123/article/details/8272423?
3. Generic Netlink HOWTO kernel API
Relevant Link:
http://www.linuxfoundation.org/collaborate/workgroups/networking/generic_netlink_howto?
4. RFC 3549 Linux Netlink as an IP Services Protocol
A Control Plane (CP) is an execution environment that may have several sub-components, which we refer to as CPCs.? Each CPC provides control for a different IP service being executed by a Forwarding Engine (FE) component.? This relationship means that there might be several CPCs on a physical CP, if it is controlling several IP services. ?
In essence, the cohesion between a CP component and an FE component is the service abstraction.
0x1: Control Plane Components (CPCs)
Control Plane Components encompass signalling protocols, with diversity ranging from dynamic routing protocols, such as OSPF to tag distribution protocols, such as CR-LDP. Classical management protocols and activities also fall under this category.
These include SNMP、COPS、and proprietary CLI/GUI configuration mechanisms.? The purpose of the control plane is to provide an execution environment for the above-mentioned activities with the ultimate goal being to configure and manage the second Network Element (NE) component: the FE.? The result of the configuration defines the way that packets traversing the FE are treated.
0x2: Forwarding Engine Components (FECs)
The FE is the entity of the NE that incoming packets (from the network into the NE) first encounter.
The FE's service-specific component massages the packet to provide it with a treatment to achieve an IP service, as defined by the Control Plane Components for that IP service.? Different services will utilize different FECs.? Service modules may be chained to achieve a more complex service ?
When built for providing a specific service, the FE service component will adhere to a forwarding model.
1. Linux IP Forwarding Engine Model
?
____ +---------------++->-| FW |---> | TCP, UDP, ... || +----+ +---------------+| |^ v| _|_+----<----+ | FW || +----+^ || YTo host From hoststack stack^ ||_____ | Ingress ^ Y device ____ +-------+ +|---|--+ ____ +--------+ Egress ->----->| FW |-->|Ingress|-->---->| Forw- |->| FW |->| Egress | device+----+ | TC | | ard | +----+ | TC |-->+-------+ +-------+ +--------+The figure above shows the Linux FE model per device.? The only mandatory part of the datapath is the Forwarding module, which is RFC 1812 conformant.? The different Firewall (FW), Ingress Traffic Control, and Egress Traffic Control building blocks are not mandatory in the datapath and may even be used to bypass the RFC 1812 module.
These modules are shown as simple blocks in the datapath but, in fact, could be multiple cascaded, independent submodules within the indicated blocks.
2. IP Services
In the diagram below, we show a simple FE<->CP setup to provide an example of the classical IPv4 service with an extension to do some basic QoS egress scheduling and illustrate how the setup fits in this described model.
Control Plane (CP).------------------------------------| /^^^^^^\ /^^^^^^\ || | | | COPS |-\ || | ospfd | | PEP | \ || \ / \_____/ | |/------\_____/ | / || | | | / || |_________\__________|____|_________|| | | |******************************************Forwarding ************* Netlink layer ************Engine (FE) *****************************************.-------------|-----------|----------|---|-------------| IPv4 forwarding | | || FE Service / / || Component / / || ---------------/---------------/--------- || | | / | |packet | | --------|-- ----|----- | packetin | | | IPv4 | | Egress | | out-->--->|------>|---->|Forwarding|----->| QoS |--->| ---->|->| | | | | Scheduler| | || | ----------- ---------- | || | | || --------------------------------------- || |-------------------------------------------------------0x3: Netlink Logical Model
In the diagram below we show a simple FEC<->CPC logical relationship. We use the IPv4 forwarding FEC (NETLINK_ROUTE, which is discussed further below) as an example.
Control Plane (CP).------------------------------------| /^^^^^\ /^^^^^\ || | | / CPC-2 \ || | CPC-1 | | COPS | || | ospfd | | PEP | || | / \____ _/ || \____/ | || | | |****************************************|************* BROADCAST WIRE ************FE---------- *****************************************.| IPv4 forwarding | | | || FEC | | | || --------------/ ----|-----------|-------- || | / | | | || | .-------. .-------. .------. | || | |Ingress| | IPv4 | |Egress| | || | |police | |Forward| | QoS | | || | |_______| |_______| |Sched | | || | ------ | || --------------------------------------- || |-----------------------------------------------------Netlink logically models FECs and CPCs in the form of nodes interconnected to each other via a broadcast wire.
The wire is specific to a service.? The example above shows the broadcast wire belonging to the extended IPv4 forwarding service.
Nodes (CPCs or FECs as illustrated above) connect to the wire and register to receive specific messages.? CPCs may connect to multiple wires if it helps them to control the service better.? All nodes(CPCs and FECs) dump packets on the broadcast wire.? Packets can be discarded by the wire if they are malformed or not specifically formatted for the wire.? Dropped packets are not seen by any of the nodes.? The Netlink service may signal an error to the sender if it detects a malformatted Netlink packet.
0x4: Message Format
There are three levels to a Netlink message: The general Netlink message header, the IP service specific template, and the IP service specific data.
從網(wǎng)絡(luò)的角度來(lái)看，Netlink是一種傳輸層通信協(xié)議
The Netlink message is used to communicate between the FEC and CPC for parameterization of the FECs, asynchronous event notification of FEC events to the CPCs, and statistics querying/gathering (typically by a CPC).
0x5: Protocol Model
1. Service Addressing
Access is provided by first connecting to the service on the FE.? The connection is achieved by making a socket() system call to the PF_NETLINK domain.? Each FEC is identified by a protocol number.? One may open either SOCK_RAW or SOCK_DGRAM type sockets, although Netlink does not distinguish between the two.? The socket connection provides the basis for the FE<->CP addressing.
Connecting to a service is followed (at any point during the life of the connection) by either issuing a service-specific command (from the CPC to the FEC, mostly for configuration purposes), issuing a statistics-collection command, or subscribing/unsubscribing to service events.? Closing the socket terminates the transaction.
2. Netlink Message Header
Netlink messages consist of a byte stream with one or multiple Netlink headers and an associated payload.? If the payload is too big to fit into a single message it, can be split over multiple Netlink messages, collectively called a multipart message.? For multipart messages, the first and all following headers have the NLM_F_MULTI Netlink header flag set, except for the last header which has the Netlink header type NLMSG_DONE.
0 1 2 30 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Length |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Type | Flags |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Sequence Number |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Process ID (PID) |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+3. The ACK Netlink Message
This message is actually used to denote both an ACK and a NACK. Typically, the direction is from FEC to CPC (in response to an ACK request message).? However, the CPC should be able to send ACKs back to FEC when requested.? The semantics for this are IP service specific.
0 1 2 30 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Netlink message header || type = NLMSG_ERROR |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Error code |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| OLD Netlink message header |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Relevant Link:
https://tools.ietf.org/html/rfc3549?
5. sendmsg、recvmsg In User Space
0x1: sendmsg
/source/net/socket.c
/** BSD sendmsg interface*/ SYSCALL_DEFINE3(sendmsg, int, fd, struct msghdr __user *, msg, unsigned, flags) {struct compat_msghdr __user *msg_compat = (struct compat_msghdr __user *)msg;struct socket *sock;struct sockaddr_storage address;struct iovec iovstack[UIO_FASTIOV], *iov = iovstack;unsigned char ctl[sizeof(struct cmsghdr) + 20] __attribute__ ((aligned(sizeof(__kernel_size_t))));/* 20 is size of ipv6_pktinfo */unsigned char *ctl_buf = ctl;struct msghdr msg_sys;int err, ctl_len, iov_size, total_len;int fput_needed;err = -EFAULT;if (MSG_CMSG_COMPAT & flags) {if (get_compat_msghdr(&msg_sys, msg_compat))return -EFAULT;}else {err = copy_msghdr_from_user(&msg_sys, msg);if (err)return err;}sock = sockfd_lookup_light(fd, &err, &fput_needed);if (!sock)goto out;/* do not move before msg_sys is valid */err = -EMSGSIZE;if (msg_sys.msg_iovlen > UIO_MAXIOV)goto out_put;/* Check whether to allocate the iovec area */err = -ENOMEM;iov_size = msg_sys.msg_iovlen * sizeof(struct iovec);if (msg_sys.msg_iovlen > UIO_FASTIOV) {iov = sock_kmalloc(sock->sk, iov_size, GFP_KERNEL);if (!iov)goto out_put;}/* This will also move the address data into kernel space */if (MSG_CMSG_COMPAT & flags) {err = verify_compat_iovec(&msg_sys, iov, (struct sockaddr *)&address, VERIFY_READ);} elseerr = verify_iovec(&msg_sys, iov, (struct sockaddr *)&address, VERIFY_READ);if (err < 0)goto out_freeiov;total_len = err;err = -ENOBUFS;if (msg_sys.msg_controllen > INT_MAX)goto out_freeiov;ctl_len = msg_sys.msg_controllen;if ((MSG_CMSG_COMPAT & flags) && ctl_len) {err = cmsghdr_from_user_compat_to_kern(&msg_sys, sock->sk, ctl, sizeof(ctl));if (err)goto out_freeiov;ctl_buf = msg_sys.msg_control;ctl_len = msg_sys.msg_controllen;} else if (ctl_len) {if (ctl_len > sizeof(ctl)) {ctl_buf = sock_kmalloc(sock->sk, ctl_len, GFP_KERNEL);if (ctl_buf == NULL)goto out_freeiov;}err = -EFAULT;/** Careful! Before this, msg_sys.msg_control contains a user pointer.* Afterwards, it will be a kernel pointer. Thus the compiler-assisted* checking falls down on this.*/if (copy_from_user(ctl_buf, (void __user *)msg_sys.msg_control, ctl_len))goto out_freectl;msg_sys.msg_control = ctl_buf;}msg_sys.msg_flags = flags;if (sock->file->f_flags & O_NONBLOCK)msg_sys.msg_flags |= MSG_DONTWAIT;err = sock_sendmsg(sock, &msg_sys, total_len);out_freectl:if (ctl_buf != ctl)sock_kfree_s(sock->sk, ctl_buf, ctl_len); out_freeiov:if (iov != iovstack)sock_kfree_s(sock->sk, iov, iov_size); out_put:fput_light(sock->file, fput_needed); out:return err; }/source/net/socket.c
int sock_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) {struct kiocb iocb;struct sock_iocb siocb;int ret;init_sync_kiocb(&iocb, NULL);iocb.private = &siocb;/*調(diào)用__sock_sendmsg進(jìn)行UDP數(shù)據(jù)報(bào)的發(fā)送*/ret = __sock_sendmsg(&iocb, sock, msg, size);if (-EIOCBQUEUED == ret)ret = wait_on_sync_kiocb(&iocb);return ret; }static inline int __sock_sendmsg(struct kiocb *iocb, struct socket *sock, struct msghdr *msg, size_t size) {struct sock_iocb *si = kiocb_to_siocb(iocb);int err;si->sock = sock;si->scm = NULL;si->msg = msg;si->size = size;err = security_socket_sendmsg(sock, msg, size);if (err)return err;/*const struct proto_ops inet_dgram_ops = {.family = PF_INET,.owner = THIS_MODULE,.release = inet_release,.bind = inet_bind,.connect = inet_dgram_connect,.socketpair = sock_no_socketpair,.accept = sock_no_accept,.getname = inet_getname,.poll = udp_poll,.ioctl = inet_ioctl,.listen = sock_no_listen,.shutdown = inet_shutdown,.setsockopt = sock_common_setsockopt,.getsockopt = sock_common_getsockopt,.sendmsg = inet_sendmsg,.recvmsg = sock_common_recvmsg,.mmap = sock_no_mmap,.sendpage = inet_sendpage,#ifdef CONFIG_COMPAT.compat_setsockopt = compat_sock_common_setsockopt,.compat_getsockopt = compat_sock_common_getsockopt,#endif};EXPORT_SYMBOL(inet_dgram_ops);從結(jié)構(gòu)體中可以看出，sendmsg()對(duì)應(yīng)的系統(tǒng)調(diào)用是inet_sendmsg()我們繼續(xù)跟進(jìn)分析inet_sendmsg()\linux-2.6.32.63\net\ipv4\af_inet.c*/return sock->ops->sendmsg(iocb, sock, msg, size); }\linux-2.6.32.63\net\ipv4\af_inet.c
int inet_sendmsg(struct kiocb *iocb, struct socket *sock, struct msghdr *msg, size_t size) {struct sock *sk = sock->sk;/* We may need to bind the socket. */if (!inet_sk(sk)->num && inet_autobind(sk))return -EAGAIN; /*INET SOCKET調(diào)用協(xié)議特有sendmsg操作符 對(duì)于INET socket中的udp發(fā)送，協(xié)議特有操作符集為udp_protlinux-2.6.32.63\net\ipv4\udp.cstruct proto udp_prot = {.name = "UDP",.owner = THIS_MODULE,.close = udp_lib_close,.connect = ip4_datagram_connect,.disconnect = udp_disconnect,.ioctl = udp_ioctl,.destroy = udp_destroy_sock,.setsockopt = udp_setsockopt,.getsockopt = udp_getsockopt,.sendmsg = udp_sendmsg,.recvmsg = udp_recvmsg,.sendpage = udp_sendpage,.backlog_rcv = __udp_queue_rcv_skb,.hash = udp_lib_hash,.unhash = udp_lib_unhash,.get_port = udp_v4_get_port,.memory_allocated = &udp_memory_allocated,.sysctl_mem = sysctl_udp_mem,.sysctl_wmem = &sysctl_udp_wmem_min,.sysctl_rmem = &sysctl_udp_rmem_min,.obj_size = sizeof(struct udp_sock),.slab_flags = SLAB_DESTROY_BY_RCU,.h.udp_table = &udp_table,#ifdef CONFIG_COMPAT.compat_setsockopt = compat_udp_setsockopt,.compat_getsockopt = compat_udp_getsockopt,#endif};EXPORT_SYMBOL(udp_prot);可以看出，對(duì)于UDP，流程進(jìn)入udp_sendmsg函數(shù)(.sendmsg對(duì)應(yīng)的是udp_sendmsg()函數(shù))，我們繼續(xù)跟進(jìn)udp_sendmsg()\linux-2.6.32.63\net\ipv4\udp.c*/return sk->sk_prot->sendmsg(iocb, sk, msg, size); } EXPORT_SYMBOL(inet_sendmsg);0x2: recvmsg
/source/net/socket.c
/** BSD recvmsg interface*/ SYSCALL_DEFINE3(recvmsg, int, fd, struct msghdr __user *, msg, unsigned int, flags) {struct compat_msghdr __user *msg_compat = (struct compat_msghdr __user *)msg;struct socket *sock;struct iovec iovstack[UIO_FASTIOV];struct iovec *iov = iovstack;struct msghdr msg_sys;unsigned long cmsg_ptr;int err, iov_size, total_len, len;int fput_needed;/* kernel mode address */struct sockaddr_storage addr;/* user mode address pointers */struct sockaddr __user *uaddr;int __user *uaddr_len;if (MSG_CMSG_COMPAT & flags) {if (get_compat_msghdr(&msg_sys, msg_compat))return -EFAULT;}else {err = copy_msghdr_from_user(&msg_sys, msg);if (err)return err;}sock = sockfd_lookup_light(fd, &err, &fput_needed);if (!sock)goto out;err = -EMSGSIZE;if (msg_sys.msg_iovlen > UIO_MAXIOV)goto out_put;/* Check whether to allocate the iovec area */err = -ENOMEM;iov_size = msg_sys.msg_iovlen * sizeof(struct iovec);if (msg_sys.msg_iovlen > UIO_FASTIOV) {iov = sock_kmalloc(sock->sk, iov_size, GFP_KERNEL);if (!iov)goto out_put;}/* Save the user-mode address (verify_iovec will change the* kernel msghdr to use the kernel address space)*/uaddr = (__force void __user *)msg_sys.msg_name;uaddr_len = COMPAT_NAMELEN(msg);if (MSG_CMSG_COMPAT & flags)err = verify_compat_iovec(&msg_sys, iov, (struct sockaddr *)&addr, VERIFY_WRITE);elseerr = verify_iovec(&msg_sys, iov, (struct sockaddr *)&addr, VERIFY_WRITE);if (err < 0)goto out_freeiov;total_len = err;cmsg_ptr = (unsigned long)msg_sys.msg_control;msg_sys.msg_flags = flags & (MSG_CMSG_CLOEXEC|MSG_CMSG_COMPAT);/* We assume all kernel code knows the size of sockaddr_storage */msg_sys.msg_namelen = 0;if (sock->file->f_flags & O_NONBLOCK)flags |= MSG_DONTWAIT;err = sock_recvmsg(sock, &msg_sys, total_len, flags);if (err < 0)goto out_freeiov;len = err;if (uaddr != NULL) {err = move_addr_to_user((struct sockaddr *)&addr, msg_sys.msg_namelen, uaddr, uaddr_len);if (err < 0)goto out_freeiov;}err = __put_user((msg_sys.msg_flags & ~MSG_CMSG_COMPAT), COMPAT_FLAGS(msg));if (err)goto out_freeiov;if (MSG_CMSG_COMPAT & flags)err = __put_user((unsigned long)msg_sys.msg_control - cmsg_ptr, &msg_compat->msg_controllen);elseerr = __put_user((unsigned long)msg_sys.msg_control - cmsg_ptr, &msg->msg_controllen);if (err)goto out_freeiov;err = len;out_freeiov:if (iov != iovstack)sock_kfree_s(sock->sk, iov, iov_size); out_put:fput_light(sock->file, fput_needed); out:return err; }?
6. kernel_recvmsg、kernel_sendmsg In Kernel Space
0x1: kernel_recvmsg
/source/net/socket.c
int kernel_recvmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec, size_t num, size_t size, int flags) {mm_segment_t oldfs = get_fs();int result;set_fs(KERNEL_DS);/** the following is safe, since for compiler definitions of kvec and* iovec are identical, yielding the same in-core layout and alignment*/msg->msg_iov = (struct iovec *)vec, msg->msg_iovlen = num;result = sock_recvmsg(sock, msg, size, flags);set_fs(oldfs);return result; }對(duì)于內(nèi)核態(tài)來(lái)說(shuō)，數(shù)據(jù)包此時(shí)已經(jīng)copy到了Netlink的KERNEL態(tài)緩存了
0x2: kernel_sendmsg
/source/net/socket.c
int kernel_sendmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec, size_t num, size_t size) {mm_segment_t oldfs = get_fs();int result;set_fs(KERNEL_DS);/** the following is safe, since for compiler definitions of kvec and* iovec are identical, yielding the same in-core layout and alignment*/msg->msg_iov = (struct iovec *)vec;msg->msg_iovlen = num;result = sock_sendmsg(sock, msg, size);set_fs(oldfs);return result; }Relevant Link:
http://www.opensource.apple.com/source/Heimdal/Heimdal-247.9/lib/roken/sendmsg.c https://fossies.org/dox/glibc-2.21/sysdeps_2mach_2hurd_2sendmsg_8c_source.html http://lxr.free-electrons.com/source/net/socket.c?
7. NetLink Sockets C++ Library
0x1: Features
1. Cross Platform Library 2. Easy to use 3. Powerful and Reliable 4. Supports both Ip4 and Ip6 5. SocketGroup class to manage the connections 6. OnAcceptReady, OnReadReady, OnDisconnect callback model 7. Fully documented library API 8. Enables to Develop socket functionality extremely Fast 9. Fits single threaded and multi-threaded designsRelevant Link:
http://sourceforge.net/projects/netlinksockets/?
8. Netlink Protocol Library Suite (libnl)
The libnl suite is a collection of libraries providing APIs to netlink protocol based Linux kernel interfaces.
Netlink is a IPC mechanism primarly between the kernel and user space processes. It was designed to be a more flexible successor to ioctl to provide mainly networking related kernel configuration and monitoring interfaces.
?
The interfaces are split into several small libraries to not force applications to link against a single, bloated library.
0x1: libnl
Core library implementing the fundamentals required to use the netlink protocol such as socket handling, message construction and parsing, and sending and receiving of data. This library is kept small and minimalistic. Other libraries of the suite depend on this library.
0x2:?libnl-route
API to the configuration interfaces of the NETLINK_ROUTE family including network interfaces, routes, addresses, neighbours, and traffic control.
0x3: libnl-genl
API to the generic netlink protocol, an extended version of the netlink protocol.
0x4: libnl-nf
API to netlink based netfilter configuration and monitoring interfaces (conntrack, log, queue)
Relevant Link:
http://www.carisma.slowglass.com/~tgr/libnl/ http://www.carisma.slowglass.com/~tgr/libnl/doc/core.html?
Copyright (c) 2015 LittleHann All rights reserved
?
轉(zhuǎn)載于:https://www.cnblogs.com/LittleHann/p/4418754.html
總結(jié)
以上是生活随笔為你收集整理的NetLink Communication Mechanism And Netlink Sourcecode Analysis的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。
- 上一篇： GitHub Top 100 Objec
- 下一篇： 关于图片转换