簡介

敏捷開發迭代周期短發布快，每周都可能面臨版本發版上線，為最大可能的降低對用戶的影響提高服務可用率，大部分團隊都需要等到半夜做發布和支持。本文就如何基于spring cloud體系做灰度發版改造提供了方案，讓我們終于白天也能偷偷摸摸的無感知發版，驗證，上線等動作，從此再也不用因為發版要熬夜了。

本文闡述的方案是灰度發版方案的一種實現(各種部署方案可參考文檔最后的附錄)，屬于一種比較節約資源的部署方案，通過精準的導流和開關控制實現用戶無感知的熱部署，比較適合中小企業采納應用。整體技術架構基于nepxion discovery插件結合項目中各個實踐場景做了方案說明和源代碼展示，如需要做權重，分組等策略可自行擴展。

術語與配置

名稱說明

灰度節點	被標記為灰度的節點
灰度入口	前端部署的節點被標記為灰度的節點
灰度用戶	賬號被標記位灰度的用戶
灰度流量	路由是需要優先選擇灰度節點的請求鏈
灰度開關	是否開啟灰度路由，值為：開啟/關閉
灰度流量開關	是否所有流量都是灰度流量，值為開啟/關閉

開關與流量關系

灰度流量開關\灰度總開關適用場景正常用戶
正常入口灰度用戶
灰度入口

(灰度總開關)開 (灰度流量開關)關	灰度節點發版，新版本驗證階段	舊版本體驗	新版本體驗
(灰度總開關)開 (灰度流量開關)開	正常節點發版，新版本批量部署階段	新版本體驗	新版本體驗
(灰度總開關)關 (灰度流量開關)開/關	新版本完成上線	新版本體驗	新版本體驗

灰度配置 Gray Properties


用戶白名單：

• 節點清單加載可以從eureka獲取

public?ResultMessage?getServices()?{
????????//本地配置的服務map
????????Map?servicesLocalMap?=?getServicesLocalMap();//要返回的服務清單
????????List?services?=?new?ArrayList();
????????discoveryClient.getServices().forEach(service?->?{final?List?instances?=?new?ArrayList();
????????????discoveryClient.getInstances(service).forEach(instanceInfo?->?{
????????????????instances.add(toInstance(instanceInfo));
????????????});//優先使用本地實例if?(null?!=?servicesLocalMap.get(service))?{final?List?serviceLocalInstances?=?servicesLocalMap.get(service).getData();//更新狀態
????????????????List?serviceLocalInstancesHasLatestStatus?=?serviceLocalInstances.stream()
????????????????????????.map(instanceLocal?->?instances.stream()
????????????????????????????????????????.filter(instance?->?StringUtils.join(instance.getHost(),?instance.getPort().toString()).equals(StringUtils.join(instanceLocal.getHost(),?instanceLocal.getPort().toString())))
????????????????????????????????????????.findFirst().map(m?->?{
????????????????????????????????????????????instanceLocal.setStatus(m.getStatus());return?instanceLocal;
????????????????????????????????????????})//????????????????????????????????????????.orElse(null)
????????????????????????????????????????.orElseGet(()?->?{
????????????????????????????????????????????instanceLocal.setStatus("OFFLINE");return?instanceLocal;
????????????????????????????????????????})
????????????????????????).filter(Objects::nonNull).collect(Collectors.toList());//去除eureka中本地已配置的實例
????????????????List?instancesRemoveLocal?=?instances.stream().filter(instance?->?!serviceLocalInstancesHasLatestStatus.stream()
????????????????????????.anyMatch(instanceLocal?->?StringUtils.join(instance.getHost(),?instance.getPort().toString()).equals(StringUtils.join(instanceLocal.getHost(),?instanceLocal.getPort().toString()))))
????????????????????????.collect(Collectors.toList());//清空并重新添加處理過的本地和遠程實例
????????????????instances.clear();
????????????????instances.addAll(instancesRemoveLocal);
????????????????instances.addAll(serviceLocalInstancesHasLatestStatus);//本地服務實例排除已經添加的服務
????????????????servicesLocalMap.remove(service);
????????????}//排序
????????????Collections.sort(instances);//單個服務和節點添加
????????????Service?serviceResp?=?new?Service();
????????????serviceResp.setService(service);
????????????serviceResp.setData(instances);
????????????services.add(serviceResp);
????????});//添加eureka中不存在，本地存在的服務
????????servicesLocalMap.values().forEach(service->{
????????????Collections.sort(service.getData());
????????????services.add(service);
????????});
????????Collections.sort(services);return?ResultCode.SUCCESS.withData(services);
????}

依賴 Gray Dependency

<dependency>
????<groupId>com.sfgroupId>
????<artifactId>cloud-discovery-service-starterartifactId>
????<version>0.0.1-SNAPSHOTversion>
dependency>


<dependency>
????<groupId>com.sfgroupId>
????<artifactId>cloud-discovery-gateway-starterartifactId>
????<version>0.0.1-SNAPSHOTversion>
dependency>



<plugin>
????<groupId>com.sfgroupId>
????<artifactId>maven-pluginartifactId>
????<version>0.0.1-SNAPSHOTversion>
????<executions>
????????<execution>
????????????<phase>compilephase>
????????????<goals>
????????????????<goal>gray-plugingoal>
????????????goals>
????????????<configuration>
????????????????<grayBuildLocationExclude>grayBuildLocationExclude>
????????????configuration>
????????execution>
????executions>
plugin>

灰度頭部 GrayHeader

灰度頭部信息信息，編碼支持的類有 < GrayHeader，GrayHeaderConstant，GrayUtil, ServiceGrayUtil >

灰度對象存放的上下文有：自定義實現類 < GrayHeaderHolder >，Request內置實現類 < RequestContextHolder >

參數key參數value說明

h-gray-is	true/false	是否為灰度流量
h-gray-domain	域名	用戶請求的域名
h-gray-userid	xxx	用戶請求的賬戶

技術改造點

灰度改造分三大類：網關改造，服務改造，場景改造。主要目的是實現灰度頭部計算、復用、續傳，負載均衡的改造。

網關改造 cloud-discovery-gateway-starter

修改pom.xml依賴

<dependency>
????<groupId>com.nepxiongroupId>
????<artifactId>discovery-plugin-starter-eurekaartifactId>
????<version>0.0.2-SNAPSHOTversion>
dependency>
<dependency>
????<groupId>com.nepxiongroupId>
????<artifactId>discovery-plugin-strategy-starter-gatewayartifactId>
????<version>0.0.2-SNAPSHOTversion>
dependency>

configure加載項

@Configuration
public?class?DiscoveryGatewayAutoConfiguration?{
????//負載均衡改造注入
????@Bean
????public?DiscoveryEnabledAdapter?discoveryEnabledAdapter()?{
????????return?new?GatewayGrayDiscoveryEnabledAdapter();
????}
????//灰度路由計算注入
????@Bean
????public?GrayRouteFlagFilter?grayRouteFilter()?{
????????return?new?GrayRouteFlagFilter();
????}
????//灰度配置獲取注入
????@Bean
????public?GrayPropertiesLoader?grayPropertiesLoader()?{
????????return?new?GrayPropertiesLoader(gatewayRedisson);
????}
}

[路由場景]增加header

在網關新增Filter，將request上下文和灰度配置匹配，算出灰度路由標記

public?class?GrayRouteFlagFilter?implements?GlobalFilter,?Ordered?{
????@Autowired
????private?GrayPropertiesLoader?grayPropertiesLoader;

????@Override
????public?Mono?filter(ServerWebExchange?exchange,?GatewayFilterChain?chain)?{
????????try?{
????????????ServerHttpRequest?request?=?exchange.getRequest();
????????????Boolean?isGray?=?grayPropertiesLoader.calculateGrayFlag(request.getHeaders().getFirst(GrayHeaderConstant.GRAY_IS),
????????????????????request.getHeaders().getFirst(GrayHeaderConstant.GRAY_NGINX_IP),
????????????????????request.getHeaders().getFirst(GrayHeaderConstant.GRAY_USER_ACCOUNT));
????????????request?=?request.mutate().header(GrayHeaderConstant.GRAY_IS,?isGray.toString()).build();
????????????exchange?=?exchange.mutate().request(request).build();
????????}catch?(Throwable?e){
????????????log.error("未知錯誤",e);
????????}finally?{
????????????return?chain.filter(exchange);
????????}
????}
}

[路由場景]改造feign負載均衡實現節點篩選

改造負載均衡，計算灰度路由標記，灰度開關和節點清單的匹配性，篩選出符合條件的節點

public?class?GatewayGrayDiscoveryEnabledAdapter?extends?DefaultDiscoveryEnabledAdapter?{

????????@Autowired
????????private?GrayPropertiesLoader?grayPropertiesLoader;

????????public?boolean?apply(Server?server)?{
????????????if?(!this.applyVersion(server))?{
????????????????return?false;
????????????}?
????????}

????????private?boolean?applyVersion(Server?server)?{
????????????//判斷灰度總開關
????????????if(!Boolean.valueOf(grayPropertiesLoader.get().getEnable())){
????????????????return?true;
????????????}
????????????Boolean?isGray?=?Boolean.valueOf(strategyContextHolder.getHeader(GrayHeaderConstant.GRAY_IS));
????????????try{
????????????????return?GrayLoadBalanceUtil.isServerMatch(this.pluginAdapter.getServerServiceId(server),?isGray,?server.getHost(),?server.getPort());
????????????}catch(Exception?e){
????????????????return?true;
????????????}
????????}

????}

服務改造 cloud-discovery-service-starter

修改pom.xml依賴

<dependency>
????<groupId>com.nepxiongroupId>
????<artifactId>discovery-plugin-starter-eurekaartifactId>
????<version>0.0.2-SNAPSHOTversion>
dependency>
<dependency>
????<groupId>com.nepxiongroupId>
????<artifactId>discovery-plugin-strategy-starter-serviceartifactId>
????<version>0.0.2-SNAPSHOTversion>
dependency>

configure加載項

@Configuration
public?class?DiscoveryServiceAutoConfiguration?{

????@Autowired
????Redisson?getRedisson;
?//灰度負載均衡注入
????@Bean
????public?DiscoveryEnabledAdapter?discoveryEnabledAdapter()?{
????????return?new?ServiceGrayDiscoveryEnabledAdapter();
????}????
????//Feign灰度頭部續傳改造注入
????@Bean
????public?FeignStrategyInterceptor?feignStrategyInterceptor()?{
????????return?new?ServiceFeignStrategyInterceptor(GrayHeaderConstant.STRATEGY_REQUEST_HEADERS);
????}
?//灰度配置獲取注入
????@Bean
????public?GrayPropertiesLoader?grayPropertiesLoader()?{
????????return?new?GrayPropertiesLoader(getRedisson);
????}
?//restTemplate灰度頭部續傳改造注入
????@Bean
????public?RestTemplateStrategyInterceptor?restTemplateStrategyInterceptor()?{
????????return?new?ServiceRestTemplateStrategyInterceptor(GrayHeaderConstant.STRATEGY_REQUEST_HEADERS);
????}
????//線程池灰度頭部續傳改造注入
????@Bean
????public?RequestContextDecorator?requestContextDecorator()?{
????????return?new?RequestContextDecorator();
????}

}

• [feign同步調用場景]改造feign負載均衡實現節點篩選 < ServiceGrayDiscoveryEnabledAdapter >

public?class?ServiceGrayDiscoveryEnabledAdapter?extends?DefaultDiscoveryEnabledAdapter?{

????????@Autowired
????????private?GrayPropertiesLoader?grayPropertiesLoader;

????????public?boolean?apply(Server?server)?{
????????????if?(!this.applyVersion(server))?{
????????????????return?false;
????????????}
????????????return?true;
????????}

????????private?boolean?applyVersion(Server?server)?{
????????????//灰度開關是否開啟
????????????if(!Boolean.valueOf(grayPropertiesLoader.get().getEnable())){
????????????????return?true;
????????????}
????????????Boolean?isGray?=?Boolean.valueOf(ServiceGrayUtil.getGrayHeaderFromContext().getIsGray());
????????????try{
????????????????return?GrayLoadBalanceUtil.isServerMatch(this.pluginAdapter.getServerServiceId(server),?isGray,?server.getHost(),?server.getPort());
????????????}catch(Exception?e){
????????????????return?true;
????????????}
????????}

}

• [restTemplate調用場景]支持header續傳 < ServiceRestTemplateStrategyInterceptor >

public?class?ServiceRestTemplateStrategyInterceptor?extends?RestTemplateStrategyInterceptor?{

????public?ClientHttpResponse?intercept(HttpRequest?request,?byte[]?body,?ClientHttpRequestExecution?execution)?throws?IOException?{
????????this.applyOuterHeader(request);
????????return?execution.execute(request,?body);
????}

????private?void?applyOuterHeader(HttpRequest?request)?{
????????HttpHeaders?headers?=?request.getHeaders();
????????GrayUtil.toMap(ServiceGrayUtil.getGrayHeaderFromContext()).forEach((key,?value)->{
????????????if(null==headers.get(key)){
????????????????headers.add(key,?value);
????????????}
????????});
????}

}

場景改造

線程池灰度改造 ThreadPoolTaskExecutor

對于線程池的使用請使用封裝過的taskExecutor，這樣就自動實現了跨線程的頭部續傳，裝飾類為： < RequestContextDecorator >

public class RequestContextDecorator implements TaskDecorator {
@Override
public Runnable decorate(Runnable runnable) {

try{
RequestAttributes context = RequestContextHolder.currentRequestAttributes();
GrayHeader grayHeader = ServiceGrayUtil.getGrayHeaderFromContext();
log.debug(" --- gray debug: THREAD parent:{}", Thread.currentThread().getName());
return () -> {
try {
try {
RequestContextHolder.setRequestAttributes(context);
GrayUtil.setGrayHeader2Local(grayHeader);
log.debug(" --- gray debug: THREAD child:{}", Thread.currentThread().getName());
}catch(Exception e){
log.error("跨線程傳遞變量出錯：", e);
}
runnable.run();
}catch(Exception e){
log.error("定時任務執行出錯：", e);
}finally{
RequestContextHolder.resetRequestAttributes();
GrayUtil.removeGrayHeader();
}
};
}catch(Exception e){
log.debug("線程裝飾出錯：", e);
}
return () -> {
runnable.run();
};
}
}

• 使用場景

#引入bean依賴(此線程池添加了灰度header續傳，已把RequestContextHolder從主線程遷移過來)：
@Resource
ThreadPoolTaskExecutor?taskExecutor;

//場景1，異步注解方式
*?@Async("taskExcutor")

//場景2，異步調用
taskExcutor.submit(task);

//場景3，異步回調方式
*?CompletableFuture?userAccountCompletableFuture?=?CompletableFuture.completedFuture(null)
????????????????.thenApplyAsync(dummy?->?{...},?taskExcutor);
自定義executor注意添加setTaskDecorator(new?RequestContextDecorator())private?ThreadPoolTaskExecutor?executorService;@PostConstructpublic?void?init()?{
????????executorService?=?new?ThreadPoolTaskExecutor();
????????...//此處添加了灰度header續傳
????????executorService.setTaskDecorator(new?RequestContextDecorator());
????????executorService.initialize();
????}

定時任務改造 Job

• 需要考慮支持的定時任務調度模式模式

全隨機：從所有的節點選一個執行任務 灰度隨機：從灰度節點中選一個執行任務 并行隨機：從非灰度節點中選一個執行任務，同時從灰度節點中選一個執行任務 非灰度隨機：從非灰度節點中選一個執行任務

• 以Quartz Job Scheduling分布式任務為例，其他xxl-job可參考此方案改造

public class QuartzJobRunner implements Job {

@Override ? ?public final void execute(JobExecutionContext context) throws JobExecutionException { ? ? //對不同的調度模式執行不同的負載均衡算法選擇節點，可參考附件工具類 ? ? GrayLoadBalanceUtil.getServerUrl(String serviceName, Boolean isGray, String grayDomain, String grayUserid) ? ?} }

消息隊列改造 Queue

統一標準的消息隊列結構封裝，帶入GrayHeader信息。

統一改造封裝消費方，如果GrayHeader信息與消費方機器匹配才做消費。

遺留問題

第三方組件未做隔離帶來的問題

因此方案從節約成本的考慮仍然共用redis,db等第三方組件，如開發過程中應盡量避免第三方存儲的數據結構發生變更，需要評估新舊版本是否會存在差異導致服務不可用來決定是否停服維護。

跨平臺的灰度服務改造探討

• 對于spring cloud體系發起調用時，統一按灰度規則改造Http Client實現灰度負載均衡，實現GrayHeader傳遞
• 對spring cloud體系接收調用時，統一規范第三方填入GrayHeader信息
• 對于通過消息隊列交互的第三方，可定制A,B topic輪換策略來實現灰度發版

附錄：

灰度工具類代碼

• GrayUtil，ServiceGrayUtil提供了一系列工具類，如獲取灰度上下文，構造灰度上下文，設置灰度上下文到下一個請求等

/**?灰度頭部上下文獲取方式?*/
GrayHeader?ServiceGrayUtil.getGrayHeaderFromContext()??
/**?跨線程的灰度頭部續傳范例?**/
GrayUtil.snapshotGrayHeaderProperties();
...
GrayUtil.recoverGrayHeaderProperties(new?HttpHeaders(),?grayHeaderMap));

GrayUtil

public?class?GrayUtil1?{

????public?static?List?requestHeaderList?=?StringUtil.splitToList(GrayHeaderConstant.STRATEGY_REQUEST_HEADERS.toLowerCase(),?";");/**
?????*?此方法只適用于網關入口處獲取灰度頭部，不能在續傳過程中使用，如續傳使用請參見ServiceGrayUtil.getGrayHeaderFromContext()
?????*?從頭部獲取灰度灰度參數對象，指定grayUserid覆蓋頭部中的參數
?????*?此方法不會重用header:is-gray標記，只會重新計算。
?????*?@param?headers
?????*?@param?grayUserid
?????*/public?static?GrayHeader?getGrayHeaderFromHeader(HttpHeaders?headers,?String?grayUserid){
????????GrayHeader?grayHeader?=?new?GrayHeader();if(StringUtils.isEmpty(grayUserid)){
????????????grayUserid?=?headers.getFirst(GrayHeaderConstant.GRAY_USER_ACCOUNT);
????????}
????????GrayPropertiesLoader?grayPropertiesLoader?=?GrayContextUtil.getBean(GrayPropertiesLoader.class);/**
?????????*?無法獲取grayUserid和grayDomain的情況下，從緩存灰度配置拿灰度flag:滿足多數用戶的流量走向
?????????*/if(null!=?grayPropertiesLoader){
????????????grayHeader?=?grayPropertiesLoader.calculateGrayHeader(null,?headers.getFirst(GrayHeaderConstant.GRAY_NGINX_IP),?grayUserid);
????????}return?grayHeader;
????}/**
?????*?設置當前header的灰度頭部信息
?????*?@param?headers
?????*?@param?grayHeader
?????*/public?static?HttpHeaders?appendGrayHeader(HttpHeaders?headers,?GrayHeader?grayHeader){try{
????????????Iterator>?iter?=?toMap(grayHeader).entrySet().iterator();while(iter.hasNext()){
????????????????Map.Entry?row?=?iter.next();if(!headers.containsKey(row.getKey())){
????????????????????headers.add(row.getKey(),?row.getValue());
????????????????}
????????????}return?headers;
????????}catch(Exception?e){
????????????log.error("恢復灰度頭部信息失敗",?e);
????????}return?headers;
????}/**
?????*
?????*?@param?headersJson?request?header?json
?????*?@param?grayHeaderString?gray??header?json
?????*?@return
?????*/public?static?String?appendGrayHeader(String?headersJson,?String?grayHeaderString){try{
????????????JSONObject?headParamObj?=?new?JSONObject();if(StringUtils.isNotEmpty(headersJson)){
????????????????headParamObj?=?JSONObject.parseObject(headersJson);
????????????}
????????????JSONObject?grayHeaderJson?=?new?JSONObject();if(StringUtils.isNotEmpty(grayHeaderString)){
????????????????grayHeaderJson?=?JSONObject.parseObject(grayHeaderString);
????????????}
????????????Iterator>?iter?=?grayHeaderJson.entrySet().iterator();while(iter.hasNext()){
????????????????JSONObject.Entry?row?=?iter.next();if(StringUtils.isEmpty(headParamObj.getString(row.getKey()))){
????????????????????headParamObj.put(row.getKey(),?row.getValue());
????????????????}
????????????}return??headParamObj.toJSONString();
????????}catch(Exception?e){
????????????log.error("設置恢復灰度頭部信息失敗",?e);
????????}return?headersJson;
????}/**
?????*?把grayHeader寫入到當前線程local(適合消息隊列等通過消息接收到grayHeader信息后寫入threadlocak，提供給restTemplate發起時抓取并塞入頭部)
?????*?@param?grayHeader
?????*/public?static?void?setGrayHeader2Local(GrayHeader?grayHeader){
????????GrayHeaderHolder.setGrayContext(grayHeader);
????}/**
?????*?清除threadLocal存儲的灰度路由信息
?????*/public?static?void?removeGrayHeader(){
????????GrayHeaderHolder.removeGrayContext();
????}/**
?????*?Gray頭部對象轉map
?????*?@param?grayHeader
?????*?@return
?????*/public?static?Map?toMap(GrayHeader?grayHeader){
????????Map?res?=?new?HashMap();if(null!=grayHeader.getIsGray()){
????????????res.put(GrayHeaderConstant.GRAY_IS,?grayHeader.getIsGray().toString());
????????}if(null!=grayHeader.getGrayUserid()){
????????????res.put(GrayHeaderConstant.GRAY_USER_ACCOUNT,?grayHeader.getGrayUserid());
????????}if(null!=grayHeader.getGrayDomain()){
????????????res.put(GrayHeaderConstant.GRAY_NGINX_IP,?grayHeader.getGrayDomain());
????????}return?res;
????}/**
?????*?map中獲取grayHeader對象
?????*?@param?map
?????*?@return
?????*/public?static?GrayHeader?toGrayHeader(Map?map){
????????GrayHeader?grayHeader?=?new?GrayHeader();if(null!=map.get(GrayHeaderConstant.GRAY_IS)){
????????????grayHeader.setIsGray(Boolean.valueOf(map.get(GrayHeaderConstant.GRAY_IS)));
????????}if(null!=map.get(GrayHeaderConstant.GRAY_USER_ACCOUNT)){
????????????grayHeader.setGrayUserid((String)map.get(GrayHeaderConstant.GRAY_USER_ACCOUNT));
????????}if(null!=map.get(GrayHeaderConstant.GRAY_NGINX_IP)){
????????????grayHeader.setGrayDomain((String)map.get(GrayHeaderConstant.GRAY_NGINX_IP));
????????}return?grayHeader;
????}public?static?boolean?isHeaderContains(String?headerName)?{return?headerName.startsWith("n-d-")?||?requestHeaderList.contains(headerName);
????}public?static?boolean?isHeaderContainsExcludeInner(String?headerName)?{return?isHeaderContains(headerName)?;
????}
}

ServiceGrayUtil

public?class?ServiceGrayUtil?{
????/**
?????*?獲取當前header的灰度頭部信息，優先從discoveryRequest上下文獲取,如果抓不到則從threadLocal獲取
?????*?@return
?????*/
????public?static?GrayHeader?getGrayHeaderFromContext(){
????????String?grayDomain?=?null;
????????String?grayUserid?=?null;
????????GrayHeader?grayHeader?=?new?GrayHeader();
????????try{
????????????/**
?????????????*?從request上下文獲取
?????????????*/
????????????Map?grayHeaderMap?=?new?HashMap();
????????????ServiceStrategyContextHolder?serviceStrategyContextHolder?=?GrayContextUtil.getBean(ServiceStrategyContextHolder.class);
????????????ServiceStrategyRouteFilter?serviceStrategyRouteFilter?=?GrayContextUtil.getBean(ServiceStrategyRouteFilter.class);
????????????ServletRequestAttributes?attributes?=?serviceStrategyContextHolder.getRestAttributes();if?(attributes?!=?null)?{
????????????????HttpServletRequest?previousRequest?=?attributes.getRequest();
????????????????Enumeration?headerNames?=?previousRequest.getHeaderNames();if?(headerNames?!=?null)?{
????????????????????String?routeRegionWeight;while?(headerNames.hasMoreElements())?{
????????????????????????routeRegionWeight?=?(String)?headerNames.nextElement();
????????????????????????String?headerValue?=?previousRequest.getHeader(routeRegionWeight);boolean?isHeaderContains?=?GrayUtil.isHeaderContainsExcludeInner(routeRegionWeight.toLowerCase());if?(isHeaderContains)?{
????????????????????????????grayHeaderMap.put(routeRegionWeight,?headerValue);
????????????????????????}
????????????????????}
????????????????}
????????????}
????????????grayHeader?=?GrayUtil.toGrayHeader(grayHeaderMap);
????????????grayDomain?=?null==grayDomain?grayHeader.getgrayDomain():grayDomain;
????????????grayUserid?=?null==grayUserid?grayHeader.getgrayUserid():grayUserid;if(null!=grayHeader.getIsGray()){
????????????????log.debug("?---?gray?debug:從request頭部獲取灰度信息成功:{}",?JsonUtil.toJson(grayHeader));return?grayHeader;
????????????}
????????}catch(Exception?e){
????????????log.debug("從request頭部獲取灰度信息失敗",?e);
????????}try{/**
?????????????*?從threadlocal獲取:如果request上下文未獲取到isGray標記則進入
?????????????*/if(null!=?GrayHeaderHolder.getGrayContext()){
????????????????grayHeader?=?GrayHeaderHolder.getGrayContext();
????????????????grayDomain?=?null==grayDomain?grayHeader.getgrayDomain():grayDomain;
????????????????grayUserid?=?null==grayUserid?grayHeader.getgrayUserid():grayUserid;
????????????}if(null!=grayHeader.getIsGray()){
????????????????log.debug("?---?gray?debug:從threadlocal獲取灰度信息成功:{}",?JsonUtil.toJson(grayHeader));return?grayHeader;
????????????}
????????}catch(Exception?e){
????????????log.debug("從threadlocal獲取灰度信息失敗",?e);
????????}try{
????????????GrayPropertiesLoader?grayPropertiesLoader?=?GrayContextUtil.getBean(GrayPropertiesLoader.class);/**
?????????????*?無法獲取grayUserid和grayDomain的情況下，從緩存灰度配置拿灰度flag:滿足多數用戶的流量走向
?????????????*/if(null!=?grayPropertiesLoader){
????????????????grayHeader?=?grayPropertiesLoader.calculateGrayHeader(null,?grayDomain,?grayUserid);
????????????}if(null!=grayHeader.getIsGray()){
????????????????log.debug("?---?gray?debug:本地上下文獲取灰度信息失敗，從配置計算灰度信息信息成功:{}?,?grayDomain:{},?grayUserid:{}",?JsonUtil.toJson(grayHeader),grayDomain,grayUserid?);return?grayHeader;
????????????}
????????}catch(Exception?e){
????????????log.debug("從配置獲取灰度標志失敗",?e);
????????}return?grayHeader;
????}public?static?String?getGrayHeaderStrFromContext(){return??JsonUtil.toJson(GrayUtil.toMap(getGrayHeaderFromContext()));
????}
}

GrayLoadBalanceUtil

public?class?GrayLoadBalanceUtil?{
????public?static?GrayLoadBalanceUtil1?grayLoadBalanceUtil;

????@Autowired
????private?DiscoveryClient?discoveryClient;

????@Autowired
????private?Registration?registration;
????@Autowired
????private?GrayPropertiesLoader?grayPropertiesLoader;

????private?static?AntPathMatcher?matcher?=?new?AntPathMatcher();

????@PostConstruct
????public?void?init()?{
????????grayLoadBalanceUtil?=?this;
????}

????public?static?String?getServerUrl(String?serviceName,?Boolean?isGray,?String?grayDomain,?String?grayUserid){

????????List?serviceInstancesList?=?grayLoadBalanceUtil.discoveryClient.getInstances(serviceName);if(null!=serviceInstancesList){for(ServiceInstance?serviceInstance:?serviceInstancesList){try{//計算isGray的過濾器在此入口之后，所以這里自己計算一遍isGray
????????????????????isGray?=?grayLoadBalanceUtil.grayPropertiesLoader.calculateGrayFlag(null==isGray?null:isGray.toString(),?grayDomain,?grayUserid);if(isServerMatch(serviceInstance.getServiceId(),?isGray,?serviceInstance.getHost(),?serviceInstance.getPort())){return?serviceInstance.getUri().toString();
????????????????????}
????????????????}catch(Exception?e){
????????????????????log.info("?---?gray?inf:?獲取版本或者服務名失敗?>?{}?",?JsonUtil.toJson(serviceInstance));return?serviceInstance.getUri().toString();
????????????????}
????????????}
????????}
????????log.error("自定義負載均衡匹配失敗，無法繼續請求服務:{},?{}",?serviceName,?isGray);return?"";
????}/**
?????*?當前節點是否被選中
?????*?@param?isGray?路由模式，true為灰度路由
?????*?@param?serviceName?服務名稱
?????*?@return
?????*/public?static?Boolean?isServerMatch(String?serviceName,?Boolean?isGray,?String?host,?Integer?port){if(!grayLoadBalanceUtil.grayPropertiesLoader.get().getGrayServiceMap().containsKey(serviceName)){//該服務未配置灰度節點則匹配第一個return?true;
????????}else{//該服務配置了灰度節點則需要匹配return?(isGray?&&?isGrayNode(host,port))?||?(!isGray?&&?!isGrayNode(host,port));
????????}
????}/**
?????*?匹配版本規則
?????*?@param?pattern
?????*?@param?value
?????*?@return
?????*/private?static?boolean?match(String?pattern,?String?value)?{return?GrayLoadBalanceUtil1.matcher.match(pattern,?value);
????}/**
?????*?獲取當前服務
?????*?@return
?????*/public?static?ServiceInstance?getLocalServiceInstance()?{return?grayLoadBalanceUtil.registration;
????}/**
?????*?當前節點是否為金絲雀節點
?????*?@return
?????*/public?static?Boolean?isCurrentGrayNode(){return?isGrayNode(grayLoadBalanceUtil.registration.getHost(),?grayLoadBalanceUtil.registration.getPort());
????}public?static?Boolean?isGrayNode(String?host,?Integer?port){try{return?grayLoadBalanceUtil.grayPropertiesLoader.get().getGrayInstanceMap().containsKey(StringUtils.join(host,?":",port));
????????}catch(Exception?e){
????????????log.error("?---?gray?debug:??node?judge??獲取當前節點是否為灰度節點失敗",?e);
????????}return?false;
????}
}

GrayHeaderHolder

public class GrayHeaderHolder {
private static final InheritableThreadLocal grayContextLocal = new InheritableThreadLocal<>();
public static GrayHeader getGrayContext(){
return grayContextLocal.get();
}
public static void setGrayContext(GrayHeader grayHeader){
grayContextLocal.set(grayHeader);
}
public static void removeGrayContext(){
grayContextLocal.remove();
}
}

GrayProperties

public class GrayProperties implements Serializable {

public GrayProperties(){

}
public GrayProperties(Boolean enable,Boolean online, String userWhitelist,String frontNginxipWhitelist, String userWhitelistRemark){
this.enable = enable;
this.online = online;
this.userWhitelist = userWhitelist;
this.frontNginxipWhitelist = frontNginxipWhitelist;
this.userWhitelistRemark = userWhitelistRemark;
}


//金絲雀路由啟用總開關
private Boolean enable = false;
//金絲雀節點匹配流量控制開關true為導流全部流量，false按service金絲雀節點匹配規則導流量
private Boolean online = false;
//用戶金絲雀路由條件----客戶端用戶白名單
private String userWhitelist;
//用戶金絲雀路由條件----客戶端用戶白名單備注
private String userWhitelistRemark;
//用戶金絲雀路由條件----前端nginxip白名單(具體參數為最新前端代碼統一部署的負載均衡ip)
private String frontNginxipWhitelist;

private Map userWhitelistMap = null;
private Map frontNginxipWhitelistMap = null;

private GrayProductVersion productVersion;
/**
* 金絲雀服務清單
*/
private List servicesLocal;
//金絲雀節點清單
private Map grayInstanceMap = null;
//存在金絲雀節點的服務清單
private Map grayServiceMap = null;
@PostConstruct
public void initMap() {
userWhitelistMap = transfer2Map(userWhitelist);
frontNginxipWhitelistMap = transfer2Map(frontNginxipWhitelist);
grayInstanceMap = new HashMap();
grayServiceMap = new HashMap();
if(null!=servicesLocal && servicesLocal.size()>0){
servicesLocal.forEach(serviceLocal->{
if(null!=serviceLocal.getData() && serviceLocal.getData().size()>0){
grayServiceMap.put(serviceLocal.getService(), "");
serviceLocal.getData().forEach(instanceLocal->{
grayInstanceMap.put(StringUtils.join(instanceLocal.getHost(),":", instanceLocal.getPort()), "");
});
}
});
}
}
/**
* 轉換逗號分隔的字符串到map, value為null
* @param str
* @return
*/
Map transfer2Map(String str){
Map map = new HashMap();
if(StringUtils.isNotEmpty(str)){
for(String key: str.trim().split(",")){
map.put(key.toLowerCase(), null);
}
}
return map;
}
}

GrayPropertiesLoader

public class GrayPropertiesLoader {
public static String GRAY_PROPERTIES_SYNC_TOPIC = "gray-properties-topic";
public static String GRAY_PROPERTIES_KEY = "GLOBAL:CONFIG:GRAY_PROPERTIES";

Redisson redisson;

GrayProperties grayProperties = null;

public GrayPropertiesLoader(Redisson redisson){
this.redisson = redisson;
}
/**
* 本地獲取配置
*/
public GrayProperties get(){
if(null!=grayProperties){
return grayProperties;
}else{
log.debug(" --- gray debug: PROPERTIES > {}", "未加載到配置本地緩存，開始從遠程獲取");
return load();
}
}

/**
* 從緩存或db加載配置
* @return
*/
private synchronized GrayProperties load(){
//緩存獲取
GrayProperties cacheGrayProperties = null;
try{
RBucket bucket = redisson.getBucket(GrayPropertiesLoader.GRAY_PROPERTIES_KEY);
cacheGrayProperties = bucket.get();
}catch(Exception e){
log.error("從redis獲取金絲雀配置信息失敗", e);
}
if(null!=cacheGrayProperties){
cacheGrayProperties.initMap();
grayProperties = cacheGrayProperties;
log.debug(" --- gray debug: PROPERTIES > {}, {}", "從緩存加載：", JsonUtil.toJson(grayProperties));
return grayProperties;
}else{
//數據庫獲取(暫不實現，redis重啟后需要修改配置來加載)
log.debug(" --- gray debug: PROPERTIES > {}", "未加載到配置緩存，跳過數據庫加載");
//都沒有則，返回一個空的
return new GrayProperties();
}
}
/**
* 金絲雀灰度設置修改監聽
*/
@PostConstruct
public void listener(){
RTopic topic = redisson.getTopic(GRAY_PROPERTIES_SYNC_TOPIC, new JsonJacksonCodec());
topic.addListener(GrayProperties.class, new MessageListener() {
@Override
public void onMessage(CharSequence charSequence, GrayProperties cacheGrayProperties) {
if(null!=cacheGrayProperties){
cacheGrayProperties.initMap();
grayProperties = cacheGrayProperties;
log.debug(" --- gray debug: PROPERTIES > {}, {}", "訂閱的配置更新了：", JsonUtil.toJson(grayProperties));
}
}
});
}
/**
* 發布配置修改
* @param grayProperties
*/
public void publish(GrayProperties grayProperties){
RTopic topic = redisson.getTopic(GRAY_PROPERTIES_SYNC_TOPIC, new JsonJacksonCodec());
topic.publish(grayProperties);
log.debug(" --- gray debug: PROPERTIES > {}, {}", "修改了配置并發布訂閱：", JsonUtil.toJson(grayProperties));
}
/**
* 根據頭部傳參判斷請求是否走灰度
* @param isGray
* @param nginxIp
* @param userAccount
* @return
*/
public Boolean calculateGrayFlag(String isGray, String nginxIp, String userAccount){
Boolean flag = false;
/**
* 保底方案1: nginxIp,userAccount為空會根據開關狀態來返回灰度標記
*/
if(null!=isGray){
flag = Boolean.valueOf(isGray);
log.debug(" --- gray debug - 【金絲雀標記】 - 優先使用已有的金絲雀標記：{}", flag);
return flag;
}else{
if(this.get().getEnable()){ //金絲雀全局開關開啟--金絲雀發版到上線階段
if(this.get().getOnline()){ //金絲雀上線開關開啟--金絲雀密集上線階段
flag = true;
log.debug(" --- gray debug - 【金絲雀標記】 - 總開關開【開】，金絲雀開關【開】：{}", flag);
return flag;
}else{//金絲雀上線開關關閉--金絲雀驗證階段
log.debug(" --- gray debug: USER > {} {}", nginxIp, userAccount);
if((null!=userAccount && this.get().getUserWhitelistMap().containsKey(userAccount)) || (null!=nginxIp && this.get().getFrontNginxipWhitelistMap().containsKey(nginxIp))){
flag = true;
log.debug(" --- gray debug - 【金絲雀標記】 - 總開關開【開】，金絲雀開關【關】， 用戶和nginxip【匹配成功】：{}", flag);
return flag;
}
/**
* 保底方案2: 根據當前節點是否為金絲雀節點來計算灰度
*/
flag = GrayLoadBalanceUtil.isCurrentGrayNode();
log.debug(" --- gray debug - 【金絲雀標記】 - 總開關開【開】，金絲雀開關【關】， 用戶和nginxip【匹配失敗】，判斷當前CLOUD節點是否為金絲雀節點: {}", flag);
return flag;
}
}else{
log.debug(" --- gray debug - 【金絲雀標記】 - 總開關開【關】：{}", flag);
}
}
return flag;
}
public GrayHeader calculateGrayHeader(String isGray, String nginxIp, String userAccount){
GrayHeader grayHeader = new GrayHeader();
grayHeader.setIsGray(calculateGrayFlag(isGray, nginxIp, userAccount));
if(null!=nginxIp){
grayHeader.setNginxIp(nginxIp);
}
if(null!=userAccount){
grayHeader.setUserAccount(userAccount);
}
return grayHeader;
}
}

GrayHeaderConstant

public class GrayHeaderConstant {

public static String GRAY_IS = "cloudp-is-gray";
public static String GRAY_NGINX_IP= "nginxip";
public static String GRAY_USER_ACCOUNT= "user-account";


public static String STRATEGY_REQUEST_HEADERS="sf-domain;cloudp-is-gray;user-account;nginxip;access_token;authorization;refresh_token;redirectUrl;cloudType;Referer;back-pass";


}

GrayHeader

public?class?GrayHeader?implements?Serializable?{
????public?GrayHeader(){

????}
????public?GrayHeader(Boolean?isGray,?String?nginxIp,?String?userAccount){
????????this.isGray?=?isGray;
????????this.nginxIp?=?nginxIp;
????????this.userAccount?=?userAccount;
????}

????//是否為灰度
????Boolean?isGray;
????//前端路由ip
????String?nginxIp;
????//用戶賬號
????String?userAccount;

}

自定義負載均衡算法參考

/**
* 根據灰度規則在服務器節點中選擇節點
* @return
*/
private NodeCacheVO selectServer(ProviderCacheVO providerCacheVO, String isGray, String userAccount, String nginxIp){
NodeCacheVO nodeCacheVO = null;
GrayProperties grayProperties = grayPropertiesLoader.get();
/* 第①部分 */
//總開關關閉，則所有節點隨機跳轉
if(!grayProperties.getEnable()){
return providerCacheVO.randomNodeByWeightInAll();
}

/* 第②部分*/
Boolean gray = grayPropertiesLoader.calculateGrayFlag(isGray, nginxIp, userAccount);
if(gray){//走灰度
nodeCacheVO = providerCacheVO.randomNodeByWeightInCanary();
}else{//走正常節點
nodeCacheVO = providerCacheVO.randomNodeByWeightInStandard();
}

/* 第③部分*/
if(null==nodeCacheVO){//兜底：如果總開關開，灰度開關關，找不到節點時在所有節點隨機選擇
nodeCacheVO = providerCacheVO.randomNodeByWeightInAll();
}
return nodeCacheVO;
}

灰度路由算法保底方案

• getGrayHeaderFromContext()實現原理

ServiceGrayUtil.getGrayHeaderFromContext()?實現了保底方案
//①?GrayHeader從request上下文獲取(isGray有就直接取，userAccount/nginxIp有就傳遞下去)
//②?GrayHeader從GrayHeaderHolder的ThreadLocal中獲取(isGray有就直接取，userAccount/nginxIp有就傳遞下去)
//③?GrayHeader從GrayPropertiesLoader進入重新計算灰度標記模式

• 灰度標記算法實現

  GrayPropertiesLoader.calculateGrayFlag

?

實現了重新計算灰度標記，其中采用了保底方案：保底方案1：計算時userAccout,nginxIp不存在則根據開關規則來判斷 保底方案2：則根據當前節點是不是灰度節點來設置后續節點是不是走灰度

?

部署方案對比(此小節來自網絡)

藍綠部署

圖片來源知乎@稻草先生

所謂藍綠部署，是指同時運行兩個版本的應用，如上圖所示，藍綠部署的時候，并不停止掉老版本，而是直接部署一套新版本，等新版本運行起來后，再將流量切換到新版本上。但是藍綠部署要求在升級過程中，同時運行兩套程序，對硬件的要求就是日常所需的二倍，比如日常運行時，需要10臺服務器支撐業務，那么使用藍綠部署，你就需要購置二十臺服務器。

「滾動發布」

滾動發布能夠解決掉藍綠部署時對硬件要求增倍的問題。

圖片來源知乎@稻草先生

所謂滾動升級，就是在升級過程中，并不一下子啟動所有新版本，是先啟動一臺新版本，再停止一臺老版本，然后再啟動一臺新版本，再停止一臺老版本，直到升級完成，這樣的話，如果日常需要10臺服務器，那么升級過程中也就只需要11臺就行了。

但是滾動升級有一個問題，在開始滾動升級后，流量會直接流向已經啟動起來的新版本，但是這個時候，新版本是不一定可用的，比如需要進一步的測試才能確認。那么在滾動升級期間，整個系統就處于非常不穩定的狀態，如果發現了問題，也比較難以確定是新版本還是老版本造成的問題。

為了解決這個問題，我們需要為滾動升級實現流量控制能力。

「灰度發布」

灰度發布也叫金絲雀發布，起源是，礦井工人發現，金絲雀對瓦斯氣體很敏感，礦工會在下井之前，先放一只金絲雀到井中，如果金絲雀不叫了，就代表瓦斯濃度高。

圖片來源知乎@稻草先生

在灰度發布開始后，先啟動一個新版本應用，但是并不直接將流量切過來，而是測試人員對新版本進行線上測試，啟動的這個新版本應用，就是我們的金絲雀。如果沒有問題，那么可以將少量的用戶流量導入到新版本上，然后再對新版本做運行狀態觀察，收集各種運行時數據，如果此時對新舊版本做各種數據對比，就是所謂的A/B測試。

當確認新版本運行良好后，再逐步將更多的流量導入到新版本上，在此期間，還可以不斷地調整新舊兩個版本的運行的服務器副本數量，以使得新版本能夠承受越來越大的流量壓力。直到將100%的流量都切換到新版本上，最后關閉剩下的老版本服務，完成灰度發布。

如果在灰度發布過程中(灰度期)發現了新版本有問題，就應該立即將流量切回老版本上，這樣，就會將負面影響控制在最小范圍內。

推薦實戰文章

Java實現圖片水印+壓縮So easy!分布式定時任務xxJob的常用姿勢都集齊了，So Easy!這個輪子讓SpringBoot實現api加密So Easy!實戰：SpringBoot集成xxl-sso實現單點登錄神奇！不需要服務器，搭建免費個人Blog，so easy！(待發布)分布式事務框架...

總結

以上是生活随笔為你收集整理的基于spring cloud 的灰度发布实践_【收藏】基于spring cloud灰度发版方案的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。