當(dāng)前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

ceph rgw：bucket policy实现

發(fā)布時(shí)間：2023/12/14 编程问答 32 豆豆

生活随笔收集整理的這篇文章主要介紹了 ceph rgw：bucket policy实现小編覺得挺不錯(cuò)的,現(xiàn)在分享給大家,幫大家做個(gè)參考.

ceph rgw：bucket policy實(shí)現(xiàn)

相比于aws，rgw的bucket policy實(shí)現(xiàn)的還不是很完善，有很多細(xì)節(jié)都不支持，并且已支持的特性也在很多細(xì)節(jié)方面與s3不同，尤其是因?yàn)閞gw不支持類似s3的account user結(jié)構(gòu)，而使用tenant作為替代而導(dǎo)致的一些不同。

并且在文檔中還提及，為了修正這種不同，以及支持更多特性，在不久后會(huì)重寫rgw的 Authentication/Authorization subsystem。到時(shí)候可能導(dǎo)致一些兼容問題？

差異性，主要有以下幾點(diǎn)：

顧名思義，只支持為bucket設(shè)置policy，不能將policy設(shè)置到user等其他資源上。
指定Principal使用如下格式：?"Principal":{"AWS":"arn:aws:iam::<tenant>:user/<username>"}；因?yàn)槟壳癛GW use ‘tenant’ identifier in place of the Amazon twelve-digit account ID。
在policy json文件中不支持變量的使用，比如${aws:username}。
rgw支持的Action、Condition是aws的子集，文檔中有列出。附錄中的RGWListBucket::verify_permission()和rgw_build_iam_environment(...)也能看出被支持的Condition有哪些。
Under AWS, all tenants share a single namespace. RGW gives every tenant its own namespace of buckets. There may be an option to enable an AWS-like ‘flat’ bucket namespace?in future versions. At present, to access a bucket belonging to another tenant, address it as?“tenant:bucket”?in the S3 request.
In AWS, a bucket policy can grant access to another account, and that account owner can then grant access to individual users with user permissions. Since we do not yet support user, role, and group permissions, account owners will currently need to grant access directly to individual users, and granting an entire account access to a bucket grants access to all users in that account.

實(shí)現(xiàn)

為一個(gè)bucket設(shè)置bucket policy，就是向該bucket對(duì)應(yīng)的bucket.instance對(duì)象的xattr中以u(píng)ser.rgw.iam-policy為key將上傳的policy json文本存入。之后使用時(shí)從xattr中查詢并解析。

而對(duì)于policy的使用，則是在rgw_process.cc/process_request(...)函數(shù)中開始的。這個(gè)函數(shù)就是rgw frontend回調(diào)函數(shù)最終驗(yàn)證并執(zhí)行請(qǐng)求的地方，它屬于REST API通用處理層，這一層以process_request函數(shù)作為入口，其主要步驟大概分為用戶認(rèn)證、桶/對(duì)象acl/policy檢查、用戶/桶配額檢查、執(zhí)行操作等。

bucket policy的驗(yàn)證，具體是在process_request調(diào)用的rgw_process_authenticated函數(shù)中，該函數(shù)先后調(diào)用了init_permissions和read_permissions，這兩個(gè)函數(shù)都包含讀取bucket policy到req_state.iam_policy的語句。

最后在op->verify_permission函數(shù)中，根據(jù)不同操作進(jìn)行權(quán)限驗(yàn)證，也包括了policy的驗(yàn)證。驗(yàn)證過程大體如下：

將被驗(yàn)證請(qǐng)求的主體和操作，轉(zhuǎn)換成policy的Principle和Action格式，存入對(duì)應(yīng)的對(duì)象，對(duì)象變量名分別為ida和res。

判斷ida與bucket policy中的Principle是否匹配，如果沒有發(fā)現(xiàn)匹配的，則返回Effect::Pass，表示沒有匹配的policy授權(quán)，那么此時(shí)需要根據(jù)其他授權(quán)機(jī)制判斷請(qǐng)求是否執(zhí)行。（其他兩個(gè)狀態(tài)是Effect::Allow和Effect::Deny，分別表示同意和阻止）

判斷res和policy的Resource是否匹配（以及res和policy的notResource是否不匹配），如果否，返回Effect::Pass。

判斷請(qǐng)求的操作與policy的Action是否匹配，如果否，返回Effect::Pass。

判斷請(qǐng)求是否滿足policy的所有Condition，如果滿足，返回Effect::Allow，不滿足，返回Effect::Deny。

其中Condition可以包括兩部分的限制，一個(gè)是要求請(qǐng)求有指定的header項(xiàng)，另一個(gè)是要求請(qǐng)求帶有指定的路徑參數(shù)，在驗(yàn)證用戶請(qǐng)求時(shí)，前者在rgw_build_iam_environment函數(shù)中被存入req_state::env?中；后者先被存入RGWListBucket(或RGWListBucketMultiparts等其他需要驗(yàn)證這些參數(shù)的對(duì)象)的成員變量中，在RGWListBucket::verify_permission()函數(shù)調(diào)用時(shí)才被存入req_state::env。req_state::env則在Condition.eval(...)中被用于比較。

有關(guān)Condition需要的參數(shù)準(zhǔn)備過程的代碼，見附錄后面幾個(gè)函數(shù)。

簡單驗(yàn)證下：
創(chuàng)建一個(gè)名為testbucket的桶，使用s3cmd為其設(shè)置policy，發(fā)現(xiàn)該桶對(duì)應(yīng)的bucket.instance對(duì)象的xattr中增加了相關(guān)的屬性u(píng)ser.rgw.iam-policy，可以使用下面列出對(duì)象的所有xattr。

$ ./bin/rados -p default.rgw.meta --namespace=root listxattr .bucket.meta.testbucket:f52fe9ac-581e-432f-a8d2-363748a54fa8.4167.1

然后使用下面的命令來獲得指定key的屬性值，你會(huì)發(fā)現(xiàn)，里面存儲(chǔ)的直接就是我們上傳的json文本。

$ ./bin/rados -p default.rgw.meta --namespace=root getxattr .bucket.meta.testbucket:f52fe9ac-581e-432f-a8d2-363748a54fa8.4167.1 user.rgw.iam-policy

功能測(cè)試

基本的PUT Policy和DELETE Policy通過s3cmd測(cè)試沒有問題。

下面測(cè)試了幾個(gè)常用的場景用法。在測(cè)試前，先創(chuàng)建幾個(gè)用戶：
屬于默認(rèn)tenant（即為空）的testid 和 testid2
屬于tenantone的userone和usertwo
屬于tenanttwo的userthree

下面使用s3cmd測(cè)試，僅在第一個(gè)case列出完整命令，之后省略。

給所有用戶授予指定權(quán)限

? cmh@ubuntu:~/code/files$ cat policy.json {"Version": "2012-10-17","Statement": [{"Action": "s3:ListBucket","Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS": "*"}}] }? cmh@ubuntu:~$ cp .s3cfg_userone .s3cfg ? cmh@ubuntu:~/code/files$ s3cmd setpolicy policy.json s3://bucketone? cmh@ubuntu:~$ cp .s3cfg_usertwo .s3cfg ? cmh@ubuntu:~/code/files$ s3cmd ls s3://bucketone 2017-12-07 07:53 977 s3://bucketone/objone 2017-12-07 07:53 977 s3://bucketone/objtwo? cmh@ubuntu:~$ cp .s3cfg_userthree .s3cfg ? cmh@ubuntu:~/code/files$ s3cmd ls s3://bucketone ERROR: Bucket 'bucketone' does not exist ERROR: S3 error: 404 (NoSuchBucket) ? cmh@ubuntu:~/code/files$ s3cmd ls s3://tenantone:bucketone 2017-12-07 07:53 977 s3://tenantone:bucketone/objone 2017-12-07 07:53 977 s3://tenantone:bucketone/objtwo

給指定用戶授予指定權(quán)限

? cmh@ubuntu:~/code/files$ cat policy.json {"Version": "2012-10-17","Statement": [{"Action": ["s3:ListBucket","s3:GetObject"],"Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS": ["arn:aws:iam:::user/testid2","arn:aws:iam::tenanttwo:user/userthree"]}}] }

給指定用戶授予所有權(quán)限

? cmh@ubuntu:~/code/files$ cat policy.json {"Version": "2012-10-17","Statement": [{"Action": "s3:*","Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS": ["arn:aws:iam:::user/testid2","arn:aws:iam::tenanttwo:user/userthree"]}}] }

給所有用戶授予所有權(quán)限

配合Condition，給指定用戶授予指定權(quán)限，并要求請(qǐng)求中帶有指定header

{"Version": "2012-10-17","Statement": [{"Sid": "statement1","Effect": "Allow","Principal": {"AWS": "arn:aws:iam::tenantone:user/usertwo"},"Action": ["s3:ListBucket"],"Resource": ["arn:aws:s3:::bucketone"],"Condition": {"StringEquals": {"aws:UserAgent": "cmh-test"}}},{"Sid": "statement2","Effect": "Deny","Principal": {"AWS": "arn:aws:iam::tenantone:user/usertwo"},"Action": ["s3:ListBucket"],"Resource": ["arn:aws:s3:::bucketone"],"Condition": {"StringNotEquals": {"aws:UserAgent": "cmh-test"}}}] }

### 配合Condition，給指定用戶授予指定權(quán)限，并要求請(qǐng)求帶有指定路徑參數(shù)

目前只支持ListBucket的s3:prefix 、 s3:delimiter 和 s3:max-keys 。

L版本驗(yàn)證失敗，Master分支代碼驗(yàn)證通過。

用戶1設(shè)置policy

{"Version": "2012-10-17","Statement": [{"Action": "s3:ListBucket","Resource": ["arn:aws:s3:::bucketone","arn:aws:s3:::bucketone/*"],"Effect": "Allow","Principal": {"AWS":"*" },"Condition":{"NumericEquals": {"s3:max-keys": "10"}}}] }

使用用戶2發(fā)起請(qǐng)求

#!/bin/bashaccess_key="usertwo123" secret_key="usertwo123" date=$(date -R -u) string_to_sign="GET\n\n\n${date}\n/bucketone/" signature=$(echo -en ${string_to_sign} | openssl sha1 -hmac ${secret_key} -binary | base64)curl "http://127.0.0.1:8000/bucketone/?max-keys=10" \-H "Date: ${date}" \-H "User-Agent: cmh-test" \-H "Authorization: AWS ${access_key}:${signature}" \-X GET -v

附錄：代碼片段

注：以下代碼為master分支代碼，不是L版本

RGWPutBucketPolicy::execute()

上傳policy的請(qǐng)求執(zhí)行函數(shù)

void RGWPutBucketPolicy::execute() {op_ret = get_params();if (op_ret < 0) {return;}bufferlist in_data = bufferlist::static_from_mem(data, len);if (!store->is_meta_master()) {op_ret = forward_request_to_master(s, NULL, store, in_data, nullptr);if (op_ret < 0) {ldout(s->cct, 20) << "forward_request_to_master returned ret=" << op_ret << dendl;return;}}try {Policy p(s->cct, s->bucket_tenant, in_data);// 將bucket原有的policy刪除，將新的加入進(jìn)去auto attrs = s->bucket_attrs;attrs[RGW_ATTR_IAM_POLICY].clear();attrs[RGW_ATTR_IAM_POLICY].append(p.text);op_ret = rgw_bucket_set_attrs(store, s->bucket_info, attrs,&s->bucket_info.objv_tracker);if (op_ret == -ECANCELED) {op_ret = 0; /* lost a race, but it's ok because policies are immutable */}} catch (rgw::IAM::PolicyParseException& e) {ldout(s->cct, 20) << "failed to parse policy: " << e.what() << dendl;op_ret = -EINVAL;} }

rgw_process_authenticated(...)

進(jìn)行權(quán)限認(rèn)證到執(zhí)行的入口函數(shù)

int rgw_process_authenticated(RGWHandler_REST * const handler,RGWOp *& op,RGWRequest * const req,req_state * const s,const bool skip_retarget) {req->log(s, "init permissions");// init_permissions 將acl、policy等信息從xattr讀入內(nèi)存// 它調(diào)用了do_init_permissions函數(shù)// do_init_permissions又調(diào)用了rgw_build_bucket_policies// rgw_build_bucket_policies的末尾部分，調(diào)用了get_iam_policy_from_attr函數(shù)// 將bucket policy存入了req_state.iam_policy變量中int ret = handler->init_permissions(op);if (ret < 0) {return ret;}/*** Only some accesses support website mode, and website mode does NOT apply* if you are using the REST endpoint either (ergo, no authenticated access)*/if (! skip_retarget) {req->log(s, "recalculating target");ret = handler->retarget(op, &op);if (ret < 0) {return ret;}req->op = op;} else {req->log(s, "retargeting skipped because of SubOp mode");}/* If necessary extract object ACL and put them into req_state. */req->log(s, "reading permissions");// 該函數(shù)同樣調(diào)用了get_iam_policy_from_attr函數(shù)// 將bucket policy存入了req_state.iam_policy變量中ret = handler->read_permissions(op);if (ret < 0) {return ret;}req->log(s, "init op");ret = op->init_processing();if (ret < 0) {return ret;}req->log(s, "verifying op mask");ret = op->verify_op_mask();if (ret < 0) {return ret;}req->log(s, "verifying op permissions");// 最終驗(yàn)證ret = op->verify_permission();if (ret < 0) {if (s->system_request) {dout(2) << "overriding permissions due to system operation" << dendl;} else if (s->auth.identity->is_admin_of(s->user->user_id)) {dout(2) << "overriding permissions due to admin operation" << dendl;} else {return ret;}}req->log(s, "verifying op params");ret = op->verify_params();if (ret < 0) {return ret;}// 執(zhí)行具體的請(qǐng)求并返回結(jié)果給客戶端req->log(s, "pre-executing");op->pre_exec();req->log(s, "executing");op->execute();req->log(s, "completing");op->complete();return 0; }

rgw_build_iam_environment(...)

根據(jù)請(qǐng)求中的header，將Condition支持的header項(xiàng)存入req_state::env中

rgw::IAM::Environment rgw_build_iam_environment(RGWRados* store,struct req_state* s) {rgw::IAM::Environment e;const auto& m = s->info.env->get_map();auto t = ceph::real_clock::now();e.emplace("aws:CurrentTime", std::to_string(ceph::real_clock::to_time_t(t)));e.emplace("aws:EpochTime", ceph::to_iso_8601(t));// TODO: This is fine for now, but once we have STS we'll need to// look and see. Also this won't work with the IdentityApplier// model, since we need to know the actual credential.e.emplace("aws:PrincipalType", "User");auto i = m.find("HTTP_REFERER");if (i != m.end()) {e.emplace("aws:Referer", i->second);}// These seem to be the semantics, judging from rest_rgw_s3.cci = m.find("SERVER_PORT_SECURE");if (i != m.end()) {e.emplace("aws:SecureTransport", "true");}i = m.find("HTTP_HOST");if (i != m.end()) {e.emplace("aws:SourceIp", i->second);}i = m.find("HTTP_USER_AGENT"); {if (i != m.end())e.emplace("aws:UserAgent", i->second);}if (s->user) {// What to do about aws::userid? One can have multiple access// keys so that isn't really suitable. Do we have a durable// identifier that can persist through name changes?e.emplace("aws:username", s->user->user_id.id);}return e; }

RGWListBucket_ObjStore_S3::get_params()

從請(qǐng)求中解析出prefix、marker、max_keys、delimiter等參數(shù)，存入RGWListBucket的成員變量中。

int RGWListBucket_ObjStore_S3::get_params() {list_versions = s->info.args.exists("versions");prefix = s->info.args.get("prefix");if (!list_versions) {marker = s->info.args.get("marker");} else {marker.name = s->info.args.get("key-marker");marker.instance = s->info.args.get("version-id-marker");}max_keys = s->info.args.get("max-keys");op_ret = parse_max_keys();if (op_ret < 0) {return op_ret;}delimiter = s->info.args.get("delimiter");encoding_type = s->info.args.get("encoding-type");if (s->system_request) {s->info.args.get_bool("objs-container", &objs_container, false);const char *shard_id_str = s->info.env->get("HTTP_RGWX_SHARD_ID");if (shard_id_str) {string err;shard_id = strict_strtol(shard_id_str, 10, &err);if (!err.empty()) {ldout(s->cct, 5) << "bad shard id specified: " << shard_id_str << dendl;return -EINVAL;}} else {shard_id = s->bucket_instance_shard_id;}}return 0; }

RGWListBucket::verify_permission()

將RGWListBucket成員變量中的prefix、delimiter、max-keys三者被Condition的參數(shù)，存入req_state::env中，用于之后的Condition::eval()

int RGWListBucket::verify_permission() {op_ret = get_params();if (op_ret < 0) {return op_ret;}if (!prefix.empty())s->env.emplace("s3:prefix", prefix);if (!delimiter.empty())s->env.emplace("s3:delimiter", delimiter);s->env.emplace("s3:max-keys", std::to_string(max));if (!verify_bucket_permission(s,list_versions ?rgw::IAM::s3ListBucketVersions :rgw::IAM::s3ListBucket)) {return -EACCES;}return 0; }