當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

ceph rgw：bucket policy实现

發布時間：2023/12/14 编程问答 36 豆豆

生活随笔收集整理的這篇文章主要介紹了 ceph rgw：bucket policy实现小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

ceph rgw：bucket policy實現

相比于aws，rgw的bucket policy實現的還不是很完善，有很多細節都不支持，并且已支持的特性也在很多細節方面與s3不同，尤其是因為rgw不支持類似s3的account user結構，而使用tenant作為替代而導致的一些不同。

并且在文檔中還提及，為了修正這種不同，以及支持更多特性，在不久后會重寫rgw的 Authentication/Authorization subsystem。到時候可能導致一些兼容問題？

差異性，主要有以下幾點：

顧名思義，只支持為bucket設置policy，不能將policy設置到user等其他資源上。
指定Principal使用如下格式：?"Principal":{"AWS":"arn:aws:iam::<tenant>:user/<username>"}；因為目前RGW use ‘tenant’ identifier in place of the Amazon twelve-digit account ID。
在policy json文件中不支持變量的使用，比如${aws:username}。
rgw支持的Action、Condition是aws的子集，文檔中有列出。附錄中的RGWListBucket::verify_permission()和rgw_build_iam_environment(...)也能看出被支持的Condition有哪些。
Under AWS, all tenants share a single namespace. RGW gives every tenant its own namespace of buckets. There may be an option to enable an AWS-like ‘flat’ bucket namespace?in future versions. At present, to access a bucket belonging to another tenant, address it as?“tenant:bucket”?in the S3 request.
In AWS, a bucket policy can grant access to another account, and that account owner can then grant access to individual users with user permissions. Since we do not yet support user, role, and group permissions, account owners will currently need to grant access directly to individual users, and granting an entire account access to a bucket grants access to all users in that account.

實現

為一個bucket設置bucket policy，就是向該bucket對應的bucket.instance對象的xattr中以user.rgw.iam-policy為key將上傳的policy json文本存入。之后使用時從xattr中查詢并解析。

而對于policy的使用，則是在rgw_process.cc/process_request(...)函數中開始的。這個函數就是rgw frontend回調函數最終驗證并執行請求的地方，它屬于REST API通用處理層，這一層以process_request函數作為入口，其主要步驟大概分為用戶認證、桶/對象acl/policy檢查、用戶/桶配額檢查、執行操作等。

bucket policy的驗證，具體是在process_request調用的rgw_process_authenticated函數中，該函數先后調用了init_permissions和read_permissions，這兩個函數都包含讀取bucket policy到req_state.iam_policy的語句。

最后在op->verify_permission函數中，根據不同操作進行權限驗證，也包括了policy的驗證。驗證過程大體如下：

將被驗證請求的主體和操作，轉換成policy的Principle和Action格式，存入對應的對象，對象變量名分別為ida和res。

判斷ida與bucket policy中的Principle是否匹配，如果沒有發現匹配的，則返回Effect::Pass，表示沒有匹配的policy授權，那么此時需要根據其他授權機制判斷請求是否執行。（其他兩個狀態是Effect::Allow和Effect::Deny，分別表示同意和阻止）

判斷res和policy的Resource是否匹配（以及res和policy的notResource是否不匹配），如果否，返回Effect::Pass。

判斷請求的操作與policy的Action是否匹配，如果否，返回Effect::Pass。

判斷請求是否滿足policy的所有Condition，如果滿足，返回Effect::Allow，不滿足，返回Effect::Deny。

其中Condition可以包括兩部分的限制，一個是要求請求有指定的header項，另一個是要求請求帶有指定的路徑參數，在驗證用戶請求時，前者在rgw_build_iam_environment函數中被存入req_state::env?中；后者先被存入RGWListBucket(或RGWListBucketMultiparts等其他需要驗證這些參數的對象)的成員變量中，在RGWListBucket::verify_permission()函數調用時才被存入req_state::env。req_state::env則在Condition.eval(...)中被用于比較。

有關Condition需要的參數準備過程的代碼，見附錄后面幾個函數。

簡單驗證下：
創建一個名為testbucket的桶，使用s3cmd為其設置policy，發現該桶對應的bucket.instance對象的xattr中增加了相關的屬性user.rgw.iam-policy，可以使用下面列出對象的所有xattr。

$ ./bin/rados -p default.rgw.meta --namespace=root listxattr .bucket.meta.testbucket:f52fe9ac-581e-432f-a8d2-363748a54fa8.4167.1

然后使用下面的命令來獲得指定key的屬性值，你會發現，里面存儲的直接就是我們上傳的json文本。

$ ./bin/rados -p default.rgw.meta --namespace=root getxattr .bucket.meta.testbucket:f52fe9ac-581e-432f-a8d2-363748a54fa8.4167.1 user.rgw.iam-policy

功能測試

基本的PUT Policy和DELETE Policy通過s3cmd測試沒有問題。

下面測試了幾個常用的場景用法。在測試前，先創建幾個用戶：
屬于默認tenant（即為空）的testid 和 testid2
屬于tenantone的userone和usertwo
屬于tenanttwo的userthree

下面使用s3cmd測試，僅在第一個case列出完整命令，之后省略。

給所有用戶授予指定權限

? cmh@ubuntu:~/code/files$ cat policy.json {"Version": "2012-10-17","Statement": [{"Action": "s3:ListBucket","Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS": "*"}}] }? cmh@ubuntu:~$ cp .s3cfg_userone .s3cfg ? cmh@ubuntu:~/code/files$ s3cmd setpolicy policy.json s3://bucketone? cmh@ubuntu:~$ cp .s3cfg_usertwo .s3cfg ? cmh@ubuntu:~/code/files$ s3cmd ls s3://bucketone 2017-12-07 07:53 977 s3://bucketone/objone 2017-12-07 07:53 977 s3://bucketone/objtwo? cmh@ubuntu:~$ cp .s3cfg_userthree .s3cfg ? cmh@ubuntu:~/code/files$ s3cmd ls s3://bucketone ERROR: Bucket 'bucketone' does not exist ERROR: S3 error: 404 (NoSuchBucket) ? cmh@ubuntu:~/code/files$ s3cmd ls s3://tenantone:bucketone 2017-12-07 07:53 977 s3://tenantone:bucketone/objone 2017-12-07 07:53 977 s3://tenantone:bucketone/objtwo

給指定用戶授予指定權限

? cmh@ubuntu:~/code/files$ cat policy.json {"Version": "2012-10-17","Statement": [{"Action": ["s3:ListBucket","s3:GetObject"],"Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS": ["arn:aws:iam:::user/testid2","arn:aws:iam::tenanttwo:user/userthree"]}}] }

給指定用戶授予所有權限

? cmh@ubuntu:~/code/files$ cat policy.json {"Version": "2012-10-17","Statement": [{"Action": "s3:*","Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS": ["arn:aws:iam:::user/testid2","arn:aws:iam::tenanttwo:user/userthree"]}}] }

給所有用戶授予所有權限

配合Condition，給指定用戶授予指定權限，并要求請求中帶有指定header

{"Version": "2012-10-17","Statement": [{"Sid": "statement1","Effect": "Allow","Principal": {"AWS": "arn:aws:iam::tenantone:user/usertwo"},"Action": ["s3:ListBucket"],"Resource": ["arn:aws:s3:::bucketone"],"Condition": {"StringEquals": {"aws:UserAgent": "cmh-test"}}},{"Sid": "statement2","Effect": "Deny","Principal": {"AWS": "arn:aws:iam::tenantone:user/usertwo"},"Action": ["s3:ListBucket"],"Resource": ["arn:aws:s3:::bucketone"],"Condition": {"StringNotEquals": {"aws:UserAgent": "cmh-test"}}}] }

### 配合Condition，給指定用戶授予指定權限，并要求請求帶有指定路徑參數

目前只支持ListBucket的s3:prefix 、 s3:delimiter 和 s3:max-keys 。

L版本驗證失敗，Master分支代碼驗證通過。

用戶1設置policy

{"Version": "2012-10-17","Statement": [{"Action": "s3:ListBucket","Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS":"*" },"Condition":{"NumericEquals": {"s3:max-keys": "10"}}}] }

使用用戶2發起請求

#!/bin/bashaccess_key="usertwo123" secret_key="usertwo123" date=$(date -R -u) string_to_sign="GET\n\n\n${date}\n/bucketone/" signature=$(echo -en ${string_to_sign} | openssl sha1 -hmac ${secret_key} -binary | base64)curl "http://127.0.0.1:8000/bucketone/?max-keys=10" \-H "Date: ${date}" \-H "User-Agent: cmh-test" \-H "Authorization: AWS ${access_key}:${signature}" \-X GET -v

附錄：代碼片段

注：以下代碼為master分支代碼，不是L版本

RGWPutBucketPolicy::execute()

上傳policy的請求執行函數

void RGWPutBucketPolicy::execute() {op_ret = get_params();if (op_ret < 0) {return;}bufferlist in_data = bufferlist::static_from_mem(data, len);if (!store->is_meta_master()) {op_ret = forward_request_to_master(s, NULL, store, in_data, nullptr);if (op_ret < 0) {ldout(s->cct, 20) << "forward_request_to_master returned ret=" << op_ret << dendl;return;}}try {Policy p(s->cct, s->bucket_tenant, in_data);// 將bucket原有的policy刪除，將新的加入進去auto attrs = s->bucket_attrs;attrs[RGW_ATTR_IAM_POLICY].clear();attrs[RGW_ATTR_IAM_POLICY].append(p.text);op_ret = rgw_bucket_set_attrs(store, s->bucket_info, attrs,&s->bucket_info.objv_tracker);if (op_ret == -ECANCELED) {op_ret = 0; /* lost a race, but it's ok because policies are immutable */}} catch (rgw::IAM::PolicyParseException& e) {ldout(s->cct, 20) << "failed to parse policy: " << e.what() << dendl;op_ret = -EINVAL;} }

rgw_process_authenticated(...)

進行權限認證到執行的入口函數

int rgw_process_authenticated(RGWHandler_REST * const handler,RGWOp *& op,RGWRequest * const req,req_state * const s,const bool skip_retarget) {req->log(s, "init permissions");// init_permissions 將acl、policy等信息從xattr讀入內存// 它調用了do_init_permissions函數// do_init_permissions又調用了rgw_build_bucket_policies// rgw_build_bucket_policies的末尾部分，調用了get_iam_policy_from_attr函數// 將bucket policy存入了req_state.iam_policy變量中int ret = handler->init_permissions(op);if (ret < 0) {return ret;}/*** Only some accesses support website mode, and website mode does NOT apply* if you are using the REST endpoint either (ergo, no authenticated access)*/if (! skip_retarget) {req->log(s, "recalculating target");ret = handler->retarget(op, &op);if (ret < 0) {return ret;}req->op = op;} else {req->log(s, "retargeting skipped because of SubOp mode");}/* If necessary extract object ACL and put them into req_state. */req->log(s, "reading permissions");// 該函數同樣調用了get_iam_policy_from_attr函數// 將bucket policy存入了req_state.iam_policy變量中ret = handler->read_permissions(op);if (ret < 0) {return ret;}req->log(s, "init op");ret = op->init_processing();if (ret < 0) {return ret;}req->log(s, "verifying op mask");ret = op->verify_op_mask();if (ret < 0) {return ret;}req->log(s, "verifying op permissions");// 最終驗證ret = op->verify_permission();if (ret < 0) {if (s->system_request) {dout(2) << "overriding permissions due to system operation" << dendl;} else if (s->auth.identity->is_admin_of(s->user->user_id)) {dout(2) << "overriding permissions due to admin operation" << dendl;} else {return ret;}}req->log(s, "verifying op params");ret = op->verify_params();if (ret < 0) {return ret;}// 執行具體的請求并返回結果給客戶端req->log(s, "pre-executing");op->pre_exec();req->log(s, "executing");op->execute();req->log(s, "completing");op->complete();return 0; }

rgw_build_iam_environment(...)

根據請求中的header，將Condition支持的header項存入req_state::env中

rgw::IAM::Environment rgw_build_iam_environment(RGWRados* store,struct req_state* s) {rgw::IAM::Environment e;const auto& m = s->info.env->get_map();auto t = ceph::real_clock::now();e.emplace("aws:CurrentTime", std::to_string(ceph::real_clock::to_time_t(t)));e.emplace("aws:EpochTime", ceph::to_iso_8601(t));// TODO: This is fine for now, but once we have STS we'll need to// look and see. Also this won't work with the IdentityApplier// model, since we need to know the actual credential.e.emplace("aws:PrincipalType", "User");auto i = m.find("HTTP_REFERER");if (i != m.end()) {e.emplace("aws:Referer", i->second);}// These seem to be the semantics, judging from rest_rgw_s3.cci = m.find("SERVER_PORT_SECURE");if (i != m.end()) {e.emplace("aws:SecureTransport", "true");}i = m.find("HTTP_HOST");if (i != m.end()) {e.emplace("aws:SourceIp", i->second);}i = m.find("HTTP_USER_AGENT"); {if (i != m.end())e.emplace("aws:UserAgent", i->second);}if (s->user) {// What to do about aws::userid? One can have multiple access// keys so that isn't really suitable. Do we have a durable// identifier that can persist through name changes?e.emplace("aws:username", s->user->user_id.id);}return e; }

RGWListBucket_ObjStore_S3::get_params()

從請求中解析出prefix、marker、max_keys、delimiter等參數，存入RGWListBucket的成員變量中。

int RGWListBucket_ObjStore_S3::get_params() {list_versions = s->info.args.exists("versions");prefix = s->info.args.get("prefix");if (!list_versions) {marker = s->info.args.get("marker");} else {marker.name = s->info.args.get("key-marker");marker.instance = s->info.args.get("version-id-marker");}max_keys = s->info.args.get("max-keys");op_ret = parse_max_keys();if (op_ret < 0) {return op_ret;}delimiter = s->info.args.get("delimiter");encoding_type = s->info.args.get("encoding-type");if (s->system_request) {s->info.args.get_bool("objs-container", &objs_container, false);const char *shard_id_str = s->info.env->get("HTTP_RGWX_SHARD_ID");if (shard_id_str) {string err;shard_id = strict_strtol(shard_id_str, 10, &err);if (!err.empty()) {ldout(s->cct, 5) << "bad shard id specified: " << shard_id_str << dendl;return -EINVAL;}} else {shard_id = s->bucket_instance_shard_id;}}return 0; }

RGWListBucket::verify_permission()

將RGWListBucket成員變量中的prefix、delimiter、max-keys三者被Condition的參數，存入req_state::env中，用于之后的Condition::eval()

int RGWListBucket::verify_permission() {op_ret = get_params();if (op_ret < 0) {return op_ret;}if (!prefix.empty())s->env.emplace("s3:prefix", prefix);if (!delimiter.empty())s->env.emplace("s3:delimiter", delimiter);s->env.emplace("s3:max-keys", std::to_string(max));if (!verify_bucket_permission(s,list_versions ?rgw::IAM::s3ListBucketVersions :rgw::IAM::s3ListBucket)) {return -EACCES;}return 0; }