冷遷移與Resize

1、遷移是指將虛擬機從一個計算節點遷移到另外一個節點上。冷遷移過程中虛擬機是關機或是處于不可用的狀態，而熱遷移則需要保證虛擬機時刻運行。

2、Resize則是指根據需求調整虛擬機的計算能力和資源。Resize和冷遷移的工作流程相同，區別只是Resize時必須保持新的Flavor配置大于舊的配置，而冷遷移則要求兩者相同。

3、resize和冷遷移的本質區別：

同一個接口，根據傳參不同，有flavor就執行升配，無flavor就走的是冷遷移。

遷移一般都是從一個節點遷移到另一個節點上，升配的時候一般默認也是如此，但可以配置resize_to_same_host，升配的時候，不遷移到別的節點。

1、?入口API:nova/api/openstack/compute/migrate_server.py

class MigrateServerController(wsgi.Controller):................def _migrate(self, req, id, body):"""允許管理員將服務器遷移到新主機。"""context = req.environ['nova.context']instance = common.get_instance(self.compute_api, context, id,expected_attrs=['flavor', 'services'])context.can(ms_policies.POLICY_ROOT % 'migrate',target={'project_id': instance.project_id})host_name = Noneif (api_version_request.is_supported(req, min_version='2.56') andbody['migrate'] is not None):host_name = body['migrate'].get('host')try:self.compute_api.resize(req.environ['nova.context'], instance,host_name=host_name)................

instance = common.get_instance(self.compute_api, context, id,expected_attrs=['flavor', 'services'])依據instance id 獲取instance id 并對instance的存在性做了校驗

實際調用函數self.compute_api.resize(req.environ['nova.context'], instance,host_name=host_name)。compute_api.resize定義在 nova/compute/api.py

2、nova/compute/api.py

class API:"""用于與計算管理器交互的API."""@check_instance_lock @check_instance_state(vm_state=[vm_states.ACTIVE, vm_states.STOPPED]) @check_instance_host(check_is_up=True) def resize(self, context, instance, flavor_id=None, clean_shutdown=True,host_name=None, auto_disk_config=None):"""調整(即，遷移)一個正在運行的實例的大小。如果flavor_id為None，則認為該進程是遷移，保持原flavor_id。如果flavor_id不是None，實例應該遷移到新的主機并調整大小為新的flavor_id。在調整大小的情況下，host_name總是None。Host_name僅在冷遷移情況下可以設置。"""allow_cross_cell_resize = self._allow_cross_cell_resize(context, instance) if host_name is not None:node = self._validate_host_for_cold_migrate(context, instance, host_name, allow_cross_cell_resize)#?校驗磁盤信息 self._check_auto_disk_config(instance, auto_disk_config=auto_disk_config) # 獲取當前flavor 并校驗flavor current_flavor = instance.get_flavor() ’’’注意(aarents):確保image_base_image_ref存在，因為在finish_resize/cross_cell_resize期間需要它。實例升級從一個較老的新星釋放可能沒有這個屬性，因為a rebuild bug bug /1893618。‘’’ instance.system_metadata.update({'image_base_image_ref': instance.image_ref} ) volume_backed = None# 如果沒有提供flavor_id，則只遷移實例。 if not flavor_id:LOG.debug("flavor_id is None. Assuming migration.",instance=instance)# 保證遷移前后虛擬機 Flavor 不會發生改變new_flavor = current_flavor else:................ filter_properties = {'ignore_hosts': []}“””通過配置項 allow_resize_to_same_host 來決定是否會 resize 到同一個計算節點實際上，當 Migrate 到同一個計算節點時，nova-compute 會觸發 UnableToMigrateToSelf 異常，再繼續 Retry Scheduler，直至調度到合適的計算節點或異常退出，前提是 nova-scheduler 啟用了 RetryFilter“”” if not self._allow_resize_to_same_host(same_flavor, instance):filter_properties['ignore_hosts'].append(instance.host) # 獲取request_spec，該參數為新的調度參數,將ignore host加入調度 request_spec = objects.RequestSpec.get_by_instance_uuid(context, instance.uuid) request_spec.ignore_hosts = filter_properties['ignore_hosts']................# 更新虛擬機狀態為RESIZE_PREP, 虛擬機設置為該狀態開始，將會鎖定，不允許進行其他任務instance.task_state = task_states.RESIZE_PREP instance.progress = 0 instance.auto_disk_config = auto_disk_config or False instance.save(expected_task_state=[None])# 發送遷移狀態if not flavor_id:self._record_action_start(context, instance, instance_actions.MIGRATE) else:self._record_action_start(context, instance,instance_actions.RESIZE) scheduler_hint = {'filter_properties': filter_properties}................# resize_instance 任務遷移 self.compute_task_api.resize_instance(context, instance,scheduler_hint=scheduler_hint,flavor=new_flavor,clean_shutdown=clean_shutdown,request_spec=request_spec,do_cast=True)

該步驟主要是進行參數校驗，并獲取request_spec 調度參數。 調用nova/conductor/api的resize_instance函數

3、nova/conductor/api

def resize_instance(self, context, instance, scheduler_hint, flavor,reservations=None, clean_shutdown=True,request_spec=None, host_list=None, do_cast=False):self.conductor_compute_rpcapi.migrate_server(context, instance, scheduler_hint, live=False, rebuild=False,flavor=flavor, block_migration=None, disk_over_commit=None,reservations=reservations, clean_shutdown=clean_shutdown,request_spec=request_spec, host_list=host_list,do_cast=do_cast)

調用的conductor 層的 migrate_server, 通過rpc遠程調用方式進行。

4、nova/conductor/rcpapi.py

def migrate_server(self, context, instance, scheduler_hint, live, rebuild,flavor, block_migration, disk_over_commit,reservations=None, clean_shutdown=True, request_spec=None,host_list=None, do_cast=False):cctxt = self.client.prepare(version=version,call_monitor_timeout=CONF.rpc_response_timeout,timeout=CONF.long_rpc_timeout) if do_cast:return cctxt.cast(context, 'migrate_server', **kw) return cctxt.call(context, 'migrate_server', **kw)

實際調用的conductor的migrate_server該過程實際通過Manager進行處理。

5、nova/conductor/manager.py

@targets_cell @wrap_instance_event(prefix='conductor') def migrate_server(self, context, instance, scheduler_hint, live, rebuild,flavor, block_migration, disk_over_commit, reservations=None,clean_shutdown=True, request_spec=None, host_list=None):if instance and not isinstance(instance, nova_object.NovaObject):# NOTE(danms): Until v2 of the RPC API, we need to tolerate# old-world instance objects hereattrs = ['metadata', 'system_metadata', 'info_cache','security_groups']instance = objects.Instance._from_db_object(context, objects.Instance(), instance,expected_attrs=attrs)# NOTE: Remove this when we drop support for v1 of the RPC APIif flavor and not isinstance(flavor, objects.Flavor):# Code downstream may expect extra_specs to be populated since it# is receiving an object, so lookup the flavor to ensure this.flavor = objects.Flavor.get_by_id(context, flavor['id'])if live and not rebuild and not flavor:self._live_migrate(context, instance, scheduler_hint,block_migration, disk_over_commit, request_spec)elif not live and not rebuild and flavor:instance_uuid = instance.uuidwith compute_utils.EventReporter(context, 'cold_migrate',self.host, instance_uuid):self._cold_migrate(context, instance, flavor,scheduler_hint['filter_properties'],clean_shutdown, request_spec,host_list)else:raise NotImplementedError()

migrate_server 是一個公用的函數， 冷遷移， resiez， 以及熱遷移都經過該函數。這里冷遷移實際使用_cold_migrate?進行處理。

def _cold_migrate(self, context, instance, flavor, filter_properties,clean_shutdown, request_spec, host_list):# ?更新request_specrequest_spec = self._get_request_spec_for_cold_migrate(context, instance, flavor, filter_properties, request_spec)# 創建冷遷移任務 并執行 task = self._build_cold_migrate_task(context, instance, flavor,request_spec, clean_shutdown, host_list) try:task.execute() except exception.NoValidHost as ex:..................# 保存request_spec 調度參數if request_spec.obj_what_changed():request_spec.save()

該過程實際就是做異常處理，并保存請求參數。task.execute() 實際處理由_execute進行。定義在nova/conductor/tasks/migrate.py

6、nova/conductor/task/migrate.py

def _execute(self):# 將forced host 重置掉，以免影響調度self.request_spec.reset_forced_destinations()legacy_props = self.request_spec.to_legacy_filter_properties_dict() scheduler_utils.setup_instance_group(self.context, self.request_spec)if not ('requested_destination' in self.request_spec and?self.request_spec.requested_destination and?'host' in?self.request_spec.requested_destination): scheduler_utils.populate_retry(legacy_props,self.instance.uuid) port_res_req, req_lvl_params = (self.network_api.get_requested_resource_for_instance( self.context, self.instance.uuid))# 重新調度生成request_specself.request_spec.requested_resources = port_res_req self.request_spec.request_level_params = req_lvl_params self._set_requested_destination_cell(legacy_props).................if self.host_list is None:# 調度， 獲取可用宿主機，并選用一臺selection = self._schedule().................else:# 這是一個重新調度，將使用提供的備用主機，在host_list中作為目的主機。selection = self._reschedule()scheduler_utils.populate_filter_properties(legacy_props, selection) (host, node) = (selection.service_host, selection.nodename)# rpc 調用目的節點prep_resizeself.compute_rpcapi.prep_resize(self.context, self.instance, self.request_spec.image,self.flavor, host, migration,request_spec=self.request_spec, filter_properties=legacy_props,node=node, clean_shutdown=self.clean_shutdown,host_list=self.host_list)

該步驟主要進行調度， 選取目標宿主機， 發送任務， 進行遷移。Nova-compute發起RPC請求后，Nova-compute的manager響應對應的方法

7、nova/compute/manager.py

def prep_resize(self, context, image, instance, flavor,request_spec, filter_properties, node,clean_shutdown, migration, host_list):# 校驗參數if node is None:node = self._get_nodename(instance, refresh=True)instance_state = instance.vm_state with self._error_out_instance_on_exception(context, instance, instance_state=instance_state),\errors_out_migration_ctxt(migration):# 通知狀態self._send_prep_resize_notifications(context, instance, fields.NotificationPhase.START,flavor)try:scheduler_hints = self._get_scheduler_hints(filter_properties,request_spec)try:self._validate_instance_group_policy(context, instance,scheduler_hints) except exception.RescheduledException as e:raise exception.InstanceFaultRollback(inner_exception=e) self._prep_resize(context, image, instance,flavor, filter_properties,node, migration, request_spec,clean_shutdown)..............finally:self._send_prep_resize_notifications(context, instance, fields.NotificationPhase.END,flavor)

由_prep_resize 進行處理

def _prep_resize(self, context, image, instance, flavor, filter_properties, node,migration, request_spec, clean_shutdown=True, ):if not filter_properties:filter_properties = {} if not instance.host:self._set_instance_obj_error_state(instance)msg = _('Instance has no source host')raise exception.MigrationError(reason=msg)“””判斷是否為同一臺主機，并檢查 supports_migrate_to_same_host 是否支持遷移到同一臺主機上“”” same_host = instance.host == self.host# 如果flavor?id匹配，它就是遷移;否則調整 if same_host and flavor.id == instance['instance_type_id']:if not self.driver.capabilities.get('supports_migrate_to_same_host', False):raise exception.InstanceFaultRollback(inner_exception=exception.UnableToMigrateToSelf(instance_id=instance.uuid, host=self.host))# 更新instance 信息 并保存instance.new_flavor = flavor vm_state = instance.vm_state LOG.debug('Stashing vm_state: %s', vm_state, instance=instance) instance.system_metadata['old_vm_state'] = vm_state instance.save()............limits = filter_properties.get('limits', {}) allocs = self.reportclient.get_allocations_for_consumer(context, instance.uuid)# 開始遷移 with self.rt.resize_claim(context, instance, flavor, node, migration, allocs,image_meta=image, limits=limits, ) as claim:LOG.info('Migrating', instance=instance)# RPC轉換到源主機以啟動實際的大小調整/遷移。self.compute_rpcapi.resize_instance(context, instance, claim.migration, image,flavor, request_spec, clean_shutdown)

通過RPC回到源主機上由源主機的nova-compute服務完成遷移。

8、nova/compure/rpcapi.py

def resize_instance(self, ctxt, instance, migration, image, flavor,request_spec, clean_shutdown=True):version = '6.0'msg_args = {'instance': instance, 'migration': migration,'image': image,'flavor': flavor,'clean_shutdown': clean_shutdown,'request_spec': request_spec,}client = self.router.client(ctxt)if not client.can_send_version(version):version = self._ver(ctxt, '5.2')del msg_args['flavor']msg_args['instance_type'] = flavorif not client.can_send_version(version):msg_args.pop('request_spec')version = '5.0'cctxt = client.prepare(server=_compute_host(None, instance),version=version)cctxt.cast(ctxt, 'resize_instance', **msg_args)

通過源碼分析，得到 compute_rpcapi.resize_instance 其實是調用instance原來主機上的過程，
進行正式的遷移。

9、nova/compute/manager.py

def resize_instance(self, context, instance, image,migration, flavor, clean_shutdown,request_spec):"""啟動一個正在運行的實例遷移到另一個主機。這是從目標主機的' ' prep_resize ' '例程啟動的并在源主機上運行。"""try:self._resize_instance(context, instance, image, migration,flavor, clean_shutdown, request_spec)except Exception:with excutils.save_and_reraise_exception():self._revert_allocation(context, instance, migration)

主要調用_resize_instance函數。

def _resize_instance(self, context, instance, image, migration, flavor,clean_shutdown, request_spec, ):instance_state = instance.vm_state with self._error_out_instance_on_exception(context, instance, instance_state=instance_state), \errors_out_migration_ctxt(migration):# 獲取instance 網絡信息 該部分現在由NEUTRON 處理network_info = self.network_api.get_instance_nw_info(context, instance)# 更新狀態migration.status = 'migrating'migration.save()instance.task_state = task_states.RESIZE_MIGRATINGinstance.save(expected_task_state=task_states.RESIZE_PREP)bdms = objects.BlockDeviceMappingList.get_by_instance_uuid(context, instance.uuid)# 通知instance狀態self._send_resize_instance_notifications(context, instance, bdms, network_info,fields.NotificationPhase.START)# 獲取虛擬機的塊設備信息block_device_info = self._get_instance_block_device_info(context, instance, bdms=bdms)timeout, retry_interval = self._get_power_off_values(instance, clean_shutdown)# 關閉并進行C盤遷移disk_info = self.driver.migrate_disk_and_power_off(context, instance, migration.dest_host,flavor, network_info,block_device_info,timeout, retry_interval)# 中斷磁盤映射關系self._terminate_volume_connections(context, instance, bdms) # 遷移虛擬機網絡self.network_api.migrate_instance_start(context,instance,migration)# 保存虛擬機狀態migration.status = 'post-migrating'migration.save()# 遷移完成instance.host = migration.dest_computeinstance.node = migration.dest_nodeinstance.old_flavor = instance.flavorinstance.task_state = task_states.RESIZE_MIGRATEDinstance.save(expected_task_state=task_states.RESIZE_MIGRATING)# 完成遷移， 完成遷移的收尾工作， 實例創建， 啟動等self.compute_rpcapi.finish_resize(context, instance,migration, image, disk_info, migration.dest_compute,request_spec) self._send_resize_instance_notifications(context, instance, bdms, network_info,fields.NotificationPhase.END) self.instance_events.clear_events_for_instance(instance)

關鍵步驟：

1、關閉并進行磁盤遷移

2、網絡遷移

3、遷移收尾工作， 創建實例， 啟動實例等

核心步驟：

磁盤遷移：

disk_info = self.driver.migrate_disk_and_power_off(context, instance, migration.dest_host,flavor, network_info,block_device_info,timeout, retry_interval)

driver.migrate_disk_and_power_off過程·定義在nova/virt/libvirt/driver.py

def migrate_disk_and_power_off(self, context, instance, dest,flavor, network_info,block_device_info=None,timeout=0, retry_interval=0):LOG.debug("Starting migrate_disk_and_power_off",instance=instance) ephemerals = driver.block_device_info_get_ephemerals(block_device_info)eph_size = (block_device.get_bdm_ephemeral_disk_size(ephemerals) orinstance.flavor.ephemeral_gb)# 磁盤參數校驗root_down = flavor.root_gb < instance.flavor.root_gb ephemeral_down = flavor.ephemeral_gb < eph_size booted_from_volume = self._is_booted_from_volume(block_device_info) if (root_down and not booted_from_volume) or ephemeral_down:reason = _("Unable to resize disk down.")raise exception.InstanceFaultRollback(exception.ResizeError(reason=reason)).................# 是否為共享存儲， 如果不是則在目標宿主機上創建對應的目錄if not shared_instance_path:try:self._remotefs.create_dir(dest, inst_base)except processutils.ProcessExecutionError as e:reason = _("not able to execute ssh command: %s") % eraise exception.InstanceFaultRollback(exception.ResizeError(reason=reason))# 關閉虛擬機 self.power_off(instance, timeout, retry_interval) self.unplug_vifs(instance, network_info)# 獲取磁盤映射關系 block_device_mapping = driver.block_device_info_get_mapping(block_device_info) for vol in block_device_mapping:connection_info = vol['connection_info']self._disconnect_volume(context, connection_info, instance) disk_info = self._get_instance_disk_info(instance, block_device_info)# 進行目錄操作， 磁盤設置等 try:self._cleanup_failed_instance_base(inst_base_resize) os.rename(inst_base, inst_base_resize)if shared_instance_path:dest = Nonefileutils.ensure_tree(inst_base) on_execute = lambda process: \self.job_tracker.add_job(instance, process.pid) on_completion = lambda process: \self.job_tracker.remove_job(instance, process.pid)...............# 將磁盤復制到目標節點 compression = info['type'] not in NO_COMPRESSION_TYPES libvirt_utils.copy_image(from_path, img_path, host=dest,on_execute=on_execute,on_completion=on_completion,compression=compression)# 遷移disk.info 磁盤信息， 例如注入信息等src_disk_info_path = os.path.join(inst_base_resize, 'disk.info') if os.path.exists(src_disk_info_path):dst_disk_info_path = os.path.join(inst_base, 'disk.info')libvirt_utils.copy_image(src_disk_info_path,dst_disk_info_path,host=dest, on_execute=on_execute,on_completion=on_completion)libvirt_utils.save_and_migrate_vtpm_dir(instance.uuid, inst_base_resize, inst_base, dest,on_execute, on_completion) except Exception:with excutils.save_and_reraise_exception():self._cleanup_remote_migration(dest, inst_base,inst_base_resize,shared_instance_path) return jsonutils.dumps(disk_info)

磁盤遷移完成后， 進行網絡遷移。

網絡遷移完成后，finish_resize。通過RPC到目標主機上完成虛擬機的Resize

10、nova/compute/rcpapi.py

def finish_resize(self, ctxt, instance, migration, image, disk_info, host,request_spec):.............cctxt = client.prepare(server=host, version=version) cctxt.cast(ctxt, 'finish_resize', **msg_args)

nova/compute/manager.py

def finish_resize(self, context, disk_info, image, instance,migration, request_spec):“””完成遷移過程。設置新傳輸的磁盤，并在其新的主機上打開實例。“””try:self._finish_resize_helper(context, disk_info, image, instance,migration, request_spec)except Exception:with excutils.save_and_reraise_exception():LOG.info('Deleting allocations for old flavor on source node ''%s after finish_resize failure. You may be able to ''recover the instance by hard rebooting it.',migration.source_compute, instance=instance) self._delete_allocation_after_move(context, instance, migration)

通過調用._finish_resize_helper函數調用_finish_resize?。

def _finish_resize(self, context, instance, migration, disk_info,image_meta, bdms, request_spec):resize_instance = Falseold_instance_type_id = migration.old_instance_type_idnew_instance_type_id = migration.new_instance_type_idold_flavor = instance.flavorold_vm_state = instance.system_metadata.get('old_vm_state',vm_states.ACTIVE)if old_instance_type_id != new_instance_type_id:new_flavor = instance.new_flavor ?# this is set in _prep_resizeself._set_instance_info(instance, new_flavor)for key in ('root_gb', 'swap', 'ephemeral_gb'):if old_flavor[key] != new_flavor[key]:resize_instance = Truebreakinstance.apply_migration_context()# 創建設置網絡， 并完成網絡遷移self.network_api.setup_networks_on_host(context, instance,migration.dest_compute) provider_mappings = self._get_request_group_mapping(request_spec)self.network_api.migrate_instance_finish(context, instance, migration, provider_mappings)network_info = self.network_api.get_instance_nw_info(context, instance)instance.task_state = task_states.RESIZE_FINISHinstance.save(expected_task_state=task_states.RESIZE_MIGRATED)self._send_finish_resize_notifications(context, instance, bdms, network_info,fields.NotificationPhase.START)self._update_volume_attachments(context, instance, bdms)# ?獲取磁盤信息block_device_info = self._get_instance_block_device_info(context, instance, refresh_conn_info=True, bdms=bdms)power_on = old_vm_state != vm_states.STOPPEDallocations = self.reportclient.get_allocs_for_consumer(context, instance.uuid)['allocations']try:# 完成遷移self.driver.finish_migration(context, migration, instance,disk_info, ?network_info,image_meta, resize_instance,allocations, block_device_info, power_on)except Exception:with excutils.save_and_reraise_exception():if old_instance_type_id != new_instance_type_id:self._set_instance_info(instance, old_flavor)self._complete_volume_attachments(context, bdms)migration.status = 'finished'migration.save()# 更新狀態instance.vm_state = vm_states.RESIZEDinstance.task_state = Noneinstance.launched_at = timeutils.utcnow()instance.save(expected_task_state=task_states.RESIZE_FINISH)return network_info

finish_migration 定義在 virt/libvirt/driver.py

12、nova/virt/libvirt/driver.py

def finish_migration(self,context: nova_context.RequestContext,migration: 'objects.Migration',instance: 'objects.Instance',disk_info: str,network_info: network_model.NetworkInfo,image_meta: 'objects.ImageMeta',resize_instance: bool,allocations: ty.Dict[str, ty.Any],block_device_info: ty.Optional[ty.Dict[str, ty.Any]] = None,power_on: bool = True, ) -> None:# 在目標主機上完成遷移過程# 獲取磁盤信息，并創建磁盤鏡像block_disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,instance,image_meta,block_device_info)self._create_image(context, instance, block_disk_info['mapping'],block_device_info=block_device_info,ignore_bdi_for_swap=True,fallback_from_host=migration.source_compute)self._ensure_console_log_for_instance(instance) gen_confdrive = functools.partial(self._create_configdrive, context, instance,InjectionInfo(admin_pass=None, network_info=network_info,files=None))for info in jsonutils.loads(disk_info):path = info['path']disk_name = os.path.basename(path)if (disk_name != 'disk.config' andinfo['type'] == 'raw' and CONF.use_cow_images):self._disk_raw_to_qcow2(info['path'])mdevs = self._allocate_mdevs(allocations)self._finish_migration_vtpm(context, instance)# 依據instance 信息生成 xml 文件 xml = self._get_guest_xml(context, instance, network_info,block_disk_info, image_meta,block_device_info=block_device_info,mdevs=mdevs)# 創建虛擬機guest = self._create_guest_with_network(context, xml, instance, network_info, block_device_info,power_on=power_on, vifs_already_plugged=True,post_xml_callback=gen_confdrive) if power_on:timer = loopingcall.FixedIntervalLoopingCall(self._wait_for_running,instance)timer.start(interval=0.5).wait()guest.sync_guest_time() LOG.debug("finish_migration finished successfully.", instance=instance)

13、_create_guest_with_network創建虛擬機

_create_guest_and_network?在目標宿主機創建虛擬機， 如果原始狀態為active則啟動
該步驟在虛擬機創建中分析。

總結

以上是生活随笔為你收集整理的Nova组件源码分析之冷迁移与Resize的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。