上篇《白話tornado源碼之一個腳本引發的血案》用上帝視角多整個框架做了一個概述，同時也看清了web框架的的本質，下面我們從tornado程序的起始來分析其源碼。

概述

上圖是tornado程序啟動以及接收到客戶端請求后的整個過程，對于整個過程可以分為兩大部分：

• 啟動程序階段，又稱為待請求階段（上圖1、2所有系列和3.0）
• 接收并處理客戶端請求階段（上圖3系列）
  

簡而言之：

1、在啟動程序階段，第一步，獲取配置文件然后生成url映射（即：一個url對應一個XXRequestHandler，從而讓XXRequestHandler來處理指定url發送的請求）；第二步，創建服務器socket對象并添加到epoll中；第三步，創建無線循環去監聽epoll。

2、在接收并處理請求階段，第一步，接收客戶端socket發送的請求（socket.accept）；第二步，從請求中獲取請求頭信息，再然后根據請求頭中的請求url去匹配某個XXRequestHandler；第三步，匹配成功的XXRequestHandler處理請求；第四步，將處理后的請求發送給客戶端；第五步，關閉客戶端socket。

本篇的內容主要剖析【啟動程序階段】，下面我們就來一步一步的剖析整個過程，在此階段主要是有下面重點標注的三個方法來實現。

import tornado.ioloop import tornado.webclass MainHandler(tornado.web.RequestHandler):def get(self):self.write("Hello, world")application = tornado.web.Application([(r"/index", MainHandler), ])if __name__ == "__main__": application.listen(8888)tornado.ioloop.IOLoop.instance().start()

一、application = tornado.web.Application([(xxx,xxx)])

　　執行Application類的構造函數，并傳入一個列表類型的參數，這個列表里保存的是url規則和對應的處理類，即：當客戶端的請求url可以配置這個規則時，那么該請求就交由對應的Handler去執行。

注意：Handler泛指繼承自RequestHandler的所有類
? ? ? ? Handlers泛指繼承自RequestHandler的所有類的集合

class Application(object):def __init__(self, handlers=None, default_host="", transforms=None,wsgi=False, **settings):#設置響應的編碼和返回方式，對應的http相應頭：Content-Encoding和Transfer-Encoding#Content-Encoding:gzip 表示對數據進行壓縮，然后再返回給用戶，從而減少流量的傳輸。#Transfer-Encoding:chunck 表示數據的傳送方式通過一塊一塊的傳輸。if transforms is None:self.transforms = []if settings.get("gzip"):self.transforms.append(GZipContentEncoding)self.transforms.append(ChunkedTransferEncoding)else:self.transforms = transforms#將參數賦值為類的變量self.handlers = []self.named_handlers = {}self.default_host = default_hostself.settings = settings#ui_modules和ui_methods用于在模版語言中擴展自定義輸出#這里將tornado內置的ui_modules和ui_methods添加到類的成員變量self.ui_modules和self.ui_methods中self.ui_modules = {'linkify': _linkify,'xsrf_form_html': _xsrf_form_html,'Template': TemplateModule,}self.ui_methods = {}self._wsgi = wsgi#獲取獲取用戶自定義的ui_modules和ui_methods，并將他們添加到之前創建的成員變量self.ui_modules和self.ui_methods中self._load_ui_modules(settings.get("ui_modules", {}))self._load_ui_methods(settings.get("ui_methods", {}))#設置靜態文件路徑，設置方式則是通過正則表達式匹配url，讓StaticFileHandler來處理匹配的urlif self.settings.get("static_path"):#從settings中讀取key為static_path的值，用于設置靜態文件路徑path = self.settings["static_path"]#獲取參數中傳入的handlers，如果空則設置為空列表handlers = list(handlers or [])#靜態文件前綴，默認是/static/static_url_prefix = settings.get("static_url_prefix","/static/")#在參數中傳入的handlers前再添加三個映射：#【/static/.*】 --> StaticFileHandler#【/(favicon\.ico)】 --> StaticFileHandler#【/(robots\.txt)】 --> StaticFileHandlerhandlers = [(re.escape(static_url_prefix) + r"(.*)", StaticFileHandler,dict(path=path)),(r"/(favicon\.ico)", StaticFileHandler, dict(path=path)),(r"/(robots\.txt)", StaticFileHandler, dict(path=path)),] + handlers#執行本類的Application的add_handlers方法#此時，handlers是一個列表，其中的每個元素都是一個對應關系，即：url正則表達式和處理匹配該正則的url的Handlerif handlers: self.add_handlers(".*$", handlers)# Automatically reload modified modules#如果settings中設置了 debug 模式，那么就使用自動加載重啟if self.settings.get("debug") and not wsgi:import autoreloadautoreload.start() Application.__init__ class Application(object):def add_handlers(self, host_pattern, host_handlers):#如果主機模型最后沒有結尾符，那么就為他添加一個結尾符。if not host_pattern.endswith("$"):host_pattern += "$"handlers = []#對主機名先做一層路由映射，例如：http://www.wupeiqi.com 和 http://safe.wupeiqi.com#即：safe對應一組url映射，www對應一組url映射，那么當請求到來時，先根據它做第一層匹配，之后再繼續進入內部匹配。#對于第一層url映射來說，由于.*會匹配所有的url，所將 .* 的永遠放在handlers列表的最后，不然 .* 就會截和了...#re.complie是編譯正則表達式，以后請求來的時候只需要執行編譯結果的match方法就可以去匹配了if self.handlers and self.handlers[-1][0].pattern == '.*$':self.handlers.insert(-1, (re.compile(host_pattern), handlers))else:self.handlers.append((re.compile(host_pattern), handlers))#遍歷我們設置的和構造函數中添加的【url->Handler】映射，將url和對應的Handler封裝到URLSpec類中(構造函數中會對url進行編譯)#并將所有的URLSpec對象添加到handlers列表中，而handlers列表和主機名模型組成一個元祖，添加到self.Handlers列表中。for spec in host_handlers:if type(spec) is type(()):assert len(spec) in (2, 3)pattern = spec[0]handler = spec[1]if len(spec) == 3:kwargs = spec[2]else:kwargs = {}spec = URLSpec(pattern, handler, kwargs)handlers.append(spec)if spec.name:#未使用該功能，默認spec.name = Noneif spec.name in self.named_handlers:logging.warning("Multiple handlers named %s; replacing previous value",spec.name)self.named_handlers[spec.name] = spec Application.add_handlers class URLSpec(object):def __init__(self, pattern, handler_class, kwargs={}, name=None):if not pattern.endswith('$'):pattern += '$'self.regex = re.compile(pattern)self.handler_class = handler_classself.kwargs = kwargsself.name = nameself._path, self._group_count = self._find_groups() URLSpec

上述代碼主要完成了以下功能：加載配置信息和生成url映射，并且把所有的信息封裝在一個application對象中。

加載的配置信息包括：

• 編碼和返回方式信息
  
• 靜態文件路徑
• ui_modules（模版語言中使用，暫時忽略）
• ui_methods（模版語言中使用，暫時忽略）
• 是否debug模式運行

　　以上的所有配置信息，都可以在settings中配置，然后在創建Application對象時候，傳入參數即可。如：application = tornado.web.Application([(r"/index", MainHandler),],**settings)

生成url映射：

• 將url和對應的Handler添加到對應的主機前綴中，如：safe.index.com、www.auto.com

?封裝數據：

　　將配置信息和url映射關系封裝到Application對象中，信息分別保存在Application對象的以下字段中：

• self.transforms，保存著編碼和返回方式信息
• self.settings，保存著配置信息
• self.ui_modules，保存著ui_modules信息
• self.ui_methods，保存這ui_methods信息
• self.handlers，保存著所有的主機名對應的Handlers，每個handlers則是url正則對應的Handler
  

二、application.listen(xxx)

　　第一步操作將配置和url映射等信息封裝到了application對象中，而這第二步執行application對象的listen方法，該方法內部又把之前包含各種信息的application對象封裝到了一個HttpServer對象中，然后繼續調用HttpServer對象的liseten方法。

class Application(object):#創建服務端socket，并綁定IP和端口并添加相應設置，注：未開始通過while監聽accept，等待客戶端連接 def listen(self, port, address="", **kwargs):from tornado.httpserver import HTTPServerserver = HTTPServer(self, **kwargs)server.listen(port, address)

詳細代碼：

class HTTPServer(object):def __init__(self, request_callback, no_keep_alive=False, io_loop=None,xheaders=False, ssl_options=None):#Application對象self.request_callback = request_callback#是否長連接self.no_keep_alive = no_keep_alive#IO循環self.io_loop = io_loopself.xheaders = xheaders#Http和Httpself.ssl_options = ssl_optionsself._socket = Noneself._started = Falsedef listen(self, port, address=""):self.bind(port, address)self.start(1)def bind(self, port, address=None, family=socket.AF_UNSPEC):assert not self._socket#創建服務端socket對象，IPV4和TCP連接self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)flags = fcntl.fcntl(self._socket.fileno(), fcntl.F_GETFD)flags |= fcntl.FD_CLOEXECfcntl.fcntl(self._socket.fileno(), fcntl.F_SETFD, flags)#配置socket對象self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)self._socket.setblocking(0)#綁定IP和端口 self._socket.bind((address, port))#最大阻塞數量self._socket.listen(128)def start(self, num_processes=1):assert not self._startedself._started = Trueif num_processes is None or num_processes <= 0:num_processes = _cpu_count()if num_processes > 1 and ioloop.IOLoop.initialized():logging.error("Cannot run in multiple processes: IOLoop instance ""has already been initialized. You cannot call ""IOLoop.instance() before calling start()")num_processes = 1#如果進程數大于1if num_processes > 1:logging.info("Pre-forking %d server processes", num_processes)for i in range(num_processes):if os.fork() == 0:import randomfrom binascii import hexlifytry:# If available, use the same method as# random.pyseed = long(hexlify(os.urandom(16)), 16)except NotImplementedError:# Include the pid to avoid initializing two# processes to the same valueseed(int(time.time() * 1000) ^ os.getpid())random.seed(seed)self.io_loop = ioloop.IOLoop.instance()self.io_loop.add_handler(self._socket.fileno(), self._handle_events,ioloop.IOLoop.READ)returnos.waitpid(-1, 0)#進程數等于1，默認else:if not self.io_loop:#設置成員變量self.io_loop為IOLoop的實例，注：IOLoop使用methodclass完成了一個單例模式self.io_loop = ioloop.IOLoop.instance()#執行IOLoop的add_handler方法，將socket句柄、self._handle_events方法和IOLoop.READ當參數傳入 self.io_loop.add_handler(self._socket.fileno(),self._handle_events,ioloop.IOLoop.READ)def _handle_events(self, fd, events):while True:try:#====important=====#connection, address = self._socket.accept()except socket.error, e:if e.args[0] in (errno.EWOULDBLOCK, errno.EAGAIN):returnraiseif self.ssl_options is not None:assert ssl, "Python 2.6+ and OpenSSL required for SSL"try:#====important=====#connection = ssl.wrap_socket(connection,server_side=True,do_handshake_on_connect=False,**self.ssl_options)except ssl.SSLError, err:if err.args[0] == ssl.SSL_ERROR_EOF:return connection.close()else:raiseexcept socket.error, err:if err.args[0] == errno.ECONNABORTED:return connection.close()else:raisetry:if self.ssl_options is not None:stream = iostream.SSLIOStream(connection, io_loop=self.io_loop)else:stream = iostream.IOStream(connection, io_loop=self.io_loop)#====important=====# HTTPConnection(stream, address, self.request_callback,self.no_keep_alive, self.xheaders) except:logging.error("Error in connection callback", exc_info=True) HTTPServer class IOLoop(object):# Constants from the epoll module_EPOLLIN = 0x001_EPOLLPRI = 0x002_EPOLLOUT = 0x004_EPOLLERR = 0x008_EPOLLHUP = 0x010_EPOLLRDHUP = 0x2000_EPOLLONESHOT = (1 << 30)_EPOLLET = (1 << 31)# Our events map exactly to the epoll eventsNONE = 0READ = _EPOLLINWRITE = _EPOLLOUTERROR = _EPOLLERR | _EPOLLHUP | _EPOLLRDHUPdef __init__(self, impl=None):self._impl = impl or _poll()if hasattr(self._impl, 'fileno'):self._set_close_exec(self._impl.fileno())self._handlers = {}self._events = {}self._callbacks = []self._timeouts = []self._running = Falseself._stopped = Falseself._blocking_signal_threshold = None# Create a pipe that we send bogus data to when we want to wake# the I/O loop when it is idleif os.name != 'nt':r, w = os.pipe()self._set_nonblocking(r)self._set_nonblocking(w)self._set_close_exec(r)self._set_close_exec(w)self._waker_reader = os.fdopen(r, "rb", 0)self._waker_writer = os.fdopen(w, "wb", 0)else:self._waker_reader = self._waker_writer = win32_support.Pipe()r = self._waker_writer.reader_fdself.add_handler(r, self._read_waker, self.READ)@classmethoddef instance(cls):if not hasattr(cls, "_instance"):cls._instance = cls()return cls._instancedef add_handler(self, fd, handler, events):"""Registers the given handler to receive the given events for fd."""self._handlers[fd] = stack_context.wrap(handler)self._impl.register(fd, events | self.ERROR) IOLoop def wrap(fn):'''Returns a callable object that will resore the current StackContextwhen executed.Use this whenever saving a callback to be executed later in adifferent execution context (either in a different thread orasynchronously in the same thread).'''if fn is None:return None# functools.wraps doesn't appear to work on functools.partial objects#@functools.wraps(fn)def wrapped(callback, contexts, *args, **kwargs):# If we're moving down the stack, _state.contexts is a prefix# of contexts. For each element of contexts not in that prefix,# create a new StackContext object.# If we're moving up the stack (or to an entirely different stack),# _state.contexts will have elements not in contexts. Use# NullContext to clear the state and then recreate from contexts.if (len(_state.contexts) > len(contexts) orany(a[1] is not b[1]for a, b in itertools.izip(_state.contexts, contexts))):# contexts have been removed or changed, so start overnew_contexts = ([NullContext()] +[cls(arg) for (cls,arg) in contexts])else:new_contexts = [cls(arg)for (cls, arg) in contexts[len(_state.contexts):]]if len(new_contexts) > 1:with contextlib.nested(*new_contexts):callback(*args, **kwargs)elif new_contexts:with new_contexts[0]:callback(*args, **kwargs)else:callback(*args, **kwargs)if getattr(fn, 'stack_context_wrapped', False):return fncontexts = _state.contextsresult = functools.partial(wrapped, fn, contexts)result.stack_context_wrapped = Truereturn result stack_context.wrap

備注：stack_context.wrap其實就是對函數進行一下封裝，即：函數在不同情況下上下文信息可能不同。

上述代碼本質上就干了以下這么四件事：

把包含了各種配置信息的application對象封裝到了HttpServer對象的request_callback字段中

創建了服務端socket對象

單例模式創建IOLoop對象，然后將socket對象句柄作為key，被封裝了的函數_handle_events作為value，添加到IOLoop對象的_handlers字段中

向epoll中注冊監聽服務端socket對象的讀可用事件

目前，我們只是看到上述代碼大致干了這四件事，而其目的有什么？他們之間的聯系又是什么呢？

答：現在不妨先來做一個猜想，待之后再在源碼中確認驗證是否正確！猜想：通過epoll監聽服務端socket事件，一旦請求到達時，則執行3中被封裝了的_handle_events函數，該函數又利用application中封裝了的各種配置信息對客戶端url來指定判定，然后指定對應的Handler處理該請求。

注意：使用epoll創建服務端socket

import socket, select EOL1 = b'/n/n' EOL2 = b'/n/r/n' response = b'HTTP/1.0 200 OK/r/nDate: Mon, 1 Jan 1996 01:01:01 GMT/r/n' response += b'Content-Type: text/plain/r/nContent-Length: 13/r/n/r/n' response += b'Hello, world!' serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) serversocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) serversocket.bind(('0.0.0.0', 8080)) serversocket.listen(1) serversocket.setblocking(0) epoll = select.epoll() epoll.register(serversocket.fileno(), select.EPOLLIN) try: connections = {}; requests = {}; responses = {} while True: events = epoll.poll(1) for fileno, event in events: if fileno == serversocket.fileno(): connection, address = serversocket.accept() connection.setblocking(0) epoll.register(connection.fileno(), select.EPOLLIN) connections[connection.fileno()] = connection requests[connection.fileno()] = b'' responses[connection.fileno()] = response elif event & select.EPOLLIN: requests[fileno] += connections[fileno].recv(1024) if EOL1 in requests[fileno] or EOL2 in requests[fileno]: epoll.modify(fileno, select.EPOLLOUT) print('-'*40 + '/n' + requests[fileno].decode()[:-2]) elif event & select.EPOLLOUT: byteswritten = connections[fileno].send(responses[fileno]) responses[fileno] = responses[fileno][byteswritten:] if len(responses[fileno]) == 0: epoll.modify(fileno, 0) connections[fileno].shutdown(socket.SHUT_RDWR) elif event & select.EPOLLHUP: epoll.unregister(fileno) connections[fileno].close() del connections[fileno] finally: epoll.unregister(serversocket.fileno()) epoll.close() serversocket.close() Code

?上述，其實就是利用epoll對象的poll(timeout)方法去輪詢已經注冊在epoll中的socket句柄，當有讀可用的信息時候，則返回包含當前句柄和Event Code的序列，然后在通過句柄對客戶端的請求進行處理

三、tornado.ioloop.IOLoop.instance().start()

上一步中創建了socket對象并使得socket對象和epoll建立了關系，該步驟則就來執行epoll的epoll方法去輪詢已經注冊在epoll對象中的socket句柄，當有讀可用信息時，則觸發一些操作什么的....

class IOLoop(object):def add_handler(self, fd, handler, events):#HttpServer的Start方法中會調用該方法self._handlers[fd] = stack_context.wrap(handler)self._impl.register(fd, events | self.ERROR)def start(self):while True:poll_timeout = 0.2try:#epoll中輪詢event_pairs = self._impl.poll(poll_timeout)except Exception, e:#省略其他#如果有讀可用信息，則把該socket對象句柄和Event Code序列添加到self._events中self._events.update(event_pairs)#遍歷self._events，處理每個請求while self._events:fd, events = self._events.popitem()try:#以socket為句柄為key，取出self._handlers中的stack_context.wrap(handler)，并執行#stack_context.wrap(handler)包裝了HTTPServer類的_handle_events函數的一個函數#是在上一步中執行add_handler方法時候，添加到self._handlers中的數據。self._handlers[fd](fd, events)except:#省略其他 class IOLoop(object):def start(self):"""Starts the I/O loop.The loop will run until one of the I/O handlers calls stop(), whichwill make the loop stop after the current event iteration completes."""if self._stopped:self._stopped = Falsereturnself._running = Truewhile True:# Never use an infinite timeout here - it can stall epollpoll_timeout = 0.2# Prevent IO event starvation by delaying new callbacks# to the next iteration of the event loop.callbacks = self._callbacksself._callbacks = []for callback in callbacks:self._run_callback(callback)if self._callbacks:poll_timeout = 0.0if self._timeouts:now = time.time()while self._timeouts and self._timeouts[0].deadline <= now:timeout = self._timeouts.pop(0)self._run_callback(timeout.callback)if self._timeouts:milliseconds = self._timeouts[0].deadline - nowpoll_timeout = min(milliseconds, poll_timeout)if not self._running:breakif self._blocking_signal_threshold is not None:# clear alarm so it doesn't fire while poll is waiting for# events. signal.setitimer(signal.ITIMER_REAL, 0, 0)try:event_pairs = self._impl.poll(poll_timeout)except Exception, e:# Depending on python version and IOLoop implementation,# different exception types may be thrown and there are# two ways EINTR might be signaled:# * e.errno == errno.EINTR# * e.args is like (errno.EINTR, 'Interrupted system call')if (getattr(e, 'errno', None) == errno.EINTR or(isinstance(getattr(e, 'args', None), tuple) andlen(e.args) == 2 and e.args[0] == errno.EINTR)):continueelse:raiseif self._blocking_signal_threshold is not None:signal.setitimer(signal.ITIMER_REAL,self._blocking_signal_threshold, 0)# Pop one fd at a time from the set of pending fds and run# its handler. Since that handler may perform actions on# other file descriptors, there may be reentrant calls to# this IOLoop that update self._events self._events.update(event_pairs)while self._events:fd, events = self._events.popitem()try:self._handlers[fd](fd, events)except (KeyboardInterrupt, SystemExit):raiseexcept (OSError, IOError), e:if e.args[0] == errno.EPIPE:# Happens when the client closes the connectionpasselse:logging.error("Exception in I/O handler for fd %d",fd, exc_info=True)except:logging.error("Exception in I/O handler for fd %d",fd, exc_info=True)# reset the stopped flag so another start/stop pair can be issuedself._stopped = Falseif self._blocking_signal_threshold is not None:signal.setitimer(signal.ITIMER_REAL, 0, 0) View Code

對于上述代碼，執行start方法后，程序就進入“死循環”，也就是會一直不停的輪詢的去檢查是否有請求到來，如果有請求到達，則執行封裝了HttpServer類的_handle_events方法和相關上下文的stack_context.wrap(handler)（其實就是執行HttpServer類的_handle_events方法），詳細見下篇博文，簡要代碼如下：

class HTTPServer(object):def _handle_events(self, fd, events):while True:try:connection, address = self._socket.accept()except socket.error, e:if e.args[0] in (errno.EWOULDBLOCK, errno.EAGAIN):returnraiseif self.ssl_options is not None:assert ssl, "Python 2.6+ and OpenSSL required for SSL"try:connection = ssl.wrap_socket(connection,server_side=True,do_handshake_on_connect=False,**self.ssl_options)except ssl.SSLError, err:if err.args[0] == ssl.SSL_ERROR_EOF:return connection.close()else:raiseexcept socket.error, err:if err.args[0] == errno.ECONNABORTED:return connection.close()else:raisetry:if self.ssl_options is not None:stream = iostream.SSLIOStream(connection, io_loop=self.io_loop)else:stream = iostream.IOStream(connection, io_loop=self.io_loop)HTTPConnection(stream, address, self.request_callback,self.no_keep_alive, self.xheaders)except:logging.error("Error in connection callback", exc_info=True)?

結束

本篇博文介紹了“待請求階段”的所作所為，簡要來說其實就是三件事：其一、把setting中的各種配置以及url和Handler之間的映射關系封裝到來application對象中（application對象又被封裝到了HttpServer對象的request_callback字段中）；其二、結合epoll創建服務端socket；其三、當請求到達時交由HttpServer類的_handle_events方法處理請求，即：處理請求的入口。對于處理請求的詳細，請參見下篇博客（客官莫急，加班編寫中...）

?

轉載于:https://www.cnblogs.com/wupeiqi/p/4375610.html

總結

以上是生活随笔為你收集整理的第二篇：白话tornado源码之待请求阶段的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。