ffmpeg內存模型及AVPacket和AVFrame API解釋

---

目錄

ffmpeg內存模型

AVPacket常用API

AVPacket Demo

AVFrame常用API

---

1. ffmpeg內存模型

/*** Supply raw packet data as input to a decoder.** Internally, this call will copy relevant AVCodecContext fields, which can* influence decoding per-packet, and apply them when the packet is actually* decoded. (For example AVCodecContext.skip_frame, which might direct the* decoder to drop the frame contained by the packet sent with this function.)** @warning The input buffer, avpkt->data must be AV_INPUT_BUFFER_PADDING_SIZE* larger than the actual read bytes because some optimized bitstream* readers read 32 or 64 bits at once and could read over the end.** @warning Do not mix this API with the legacy API (like avcodec_decode_video2())* on the same AVCodecContext. It will return unexpected results now* or in future libavcodec versions.** @note The AVCodecContext MUST have been opened with @ref avcodec_open2()* before packets may be fed to the decoder.** @param avctx codec context* @param[in] avpkt The input AVPacket. Usually, this will be a single video* frame, or several complete audio frames.* Ownership of the packet remains with the caller, and the* decoder will not write to the packet. The decoder may create* a reference to the packet data (or copy it if the packet is* not reference-counted).* Unlike with older APIs, the packet is always fully consumed,* and if it contains multiple frames (e.g. some audio codecs),* will require you to call avcodec_receive_frame() multiple* times afterwards before you can send a new packet.* It can be NULL (or an AVPacket with data set to NULL and* size set to 0); in this case, it is considered a flush* packet, which signals the end of the stream. Sending the* first flush packet will return success. Subsequent ones are* unnecessary and will return AVERROR_EOF. If the decoder* still has frames buffered, it will return them after sending* a flush packet.** @return 0 on success, otherwise negative error code:* AVERROR(EAGAIN): input is not accepted in the current state - user* must read output with avcodec_receive_frame() (once* all output is read, the packet should be resent, and* the call will not fail with EAGAIN).* AVERROR_EOF: the decoder has been flushed, and no new packets can* be sent to it (also returned if more than 1 flush* packet is sent)* AVERROR(EINVAL): codec not opened, it is an encoder, or requires flush* AVERROR(ENOMEM): failed to add packet to internal queue, or similar* other errors: legitimate decoding errors*/ int avcodec_send_packet(AVCodecContext *avctx, const AVPacket *avpkt); /*** Return decoded output data from a decoder.** @param avctx codec context* @param frame This will be set to a reference-counted video or audio* frame (depending on the decoder type) allocated by the* decoder. Note that the function will always call* av_frame_unref(frame) before doing anything else.** @return* 0: success, a frame was returned* AVERROR(EAGAIN): output is not available in this state - user must try* to send new input* AVERROR_EOF: the decoder has been fully flushed, and there will be* no more output frames* AVERROR(EINVAL): codec not opened, or it is an encoder* AVERROR_INPUT_CHANGED: current decoded frame has changed parameters* with respect to first decoded frame. Applicable* when flag AV_CODEC_FLAG_DROPCHANGED is set.* other negative values: legitimate decoding errors*/ int avcodec_receive_frame(AVCodecContext *avctx, AVFrame *frame);

2. 從av_read_frame讀取到一個AVPacket后怎么放入隊列？從avcodec_recevice_frame讀取到一個AVFrame后又怎么放入隊列？

從現有的Packet拷貝一個新Packet的時候，有兩種情況：

兩個Packet的buf引用的是同一數據緩存空間，這時候要注意數據緩存空間的釋放問題；

兩個Packet的buf引用不同的數據緩存空間，每個Packet都有數據緩存空間的copy；

AVPacket和AVFrame內部都封裝了AVBufferRef

AVBufferRef真正存儲數據的是AVBuffer

AVBuffer的data是真正存數據的，refcount是引用計數

更精確一點

對于多個AVPacket共享同一個緩存空間， FFmpeg使用的引用計數的機制（reference-count） ：

初始化引用計數為0，只有真正分配AVBuffer的時候，引用計數初始化為1;

當有新的Packet引用共享的緩存空間時， 就將引用計數+1；

當釋放了引用共享空間的Packet，就將引用計數-1；引用計數為0時，就釋放掉引用的緩存空間AVBuffer。

AVFrame也是采用同樣的機制。

---

2. AVPacket常用API

函數原型說明

AVPacket *av_packet_alloc(void);	分配AVPacket，這個時候和buffer沒有關系
void av_packet_free(AVPacket **pkt);	釋放AVPacket和_alloc對應
void av_init_packet(AVPacket *pkt);	初始化AVPacket，只是單純初始化pkt字段
int av_new_packet(AVPacket *pkt, int size);	給AVPacket的buf分配內存， 引用計數初始化為1
int av_packet_ref(AVPacket *dst, const AVPacket *src);	增加引用計數
void av_packet_unref(AVPacket *pkt);	減少引用計數
void av_packet_move_ref(AVPacket *dst, AVPacket *src);	轉移引用計數
AVPacket *av_packet_clone(const AVPacket *src);	等于av_packet_alloc()+av_packet_ref()

0. AVPacket結構體

/*** This structure stores compressed data. It is typically exported by demuxers* and then passed as input to decoders, or received as output from encoders and* then passed to muxers.** For video, it should typically contain one compressed frame. For audio it may* contain several compressed frames. Encoders are allowed to output empty* packets, with no compressed data, containing only side data* (e.g. to update some stream parameters at the end of encoding).** AVPacket is one of the few structs in FFmpeg, whose size is a part of public* ABI. Thus it may be allocated on stack and no new fields can be added to it* without libavcodec and libavformat major bump.** The semantics of data ownership depends on the buf field.* If it is set, the packet data is dynamically allocated and is* valid indefinitely until a call to av_packet_unref() reduces the* reference count to 0.** If the buf field is not set av_packet_ref() would make a copy instead* of increasing the reference count.** The side data is always allocated with av_malloc(), copied by* av_packet_ref() and freed by av_packet_unref().** @see av_packet_ref* @see av_packet_unref*/ typedef struct AVPacket {/*** A reference to the reference-counted buffer where the packet data is* stored.* May be NULL, then the packet data is not reference-counted.*/AVBufferRef *buf;/*** Presentation timestamp in AVStream->time_base units; the time at which* the decompressed packet will be presented to the user.* Can be AV_NOPTS_VALUE if it is not stored in the file.* pts MUST be larger or equal to dts as presentation cannot happen before* decompression, unless one wants to view hex dumps. Some formats misuse* the terms dts and pts/cts to mean something different. Such timestamps* must be converted to true pts/dts before they are stored in AVPacket.*/int64_t pts;/*** Decompression timestamp in AVStream->time_base units; the time at which* the packet is decompressed.* Can be AV_NOPTS_VALUE if it is not stored in the file.*/int64_t dts;uint8_t *data;int size;int stream_index;/*** A combination of AV_PKT_FLAG values*/int flags;/*** Additional packet data that can be provided by the container.* Packet can contain several types of side information.*/AVPacketSideData *side_data;int side_data_elems;/*** Duration of this packet in AVStream->time_base units, 0 if unknown.* Equals next_pts - this_pts in presentation order.*/int64_t duration;int64_t pos; ///< byte position in stream, -1 if unknown#if FF_API_CONVERGENCE_DURATION/*** @deprecated Same as the duration field, but as int64_t. This was required* for Matroska subtitles, whose duration values could overflow when the* duration field was still an int.*/attribute_deprecatedint64_t convergence_duration; #endif } AVPacket;

1. AVPacket *av_packet_alloc(void);

解釋

簡單的創建一個AVPacket，將其字段設為默認值（data為空，沒有數據緩存空間），data的指針需要另外去賦值。

分配AVPacket，初始化pkt字段，這個時候和buffer沒有關系，內部調用了void av_init_packet(AVPacket *pkt);

源碼

AVPacket *av_packet_alloc(void) {AVPacket *pkt = av_mallocz(sizeof(AVPacket));if (!pkt)return pkt;av_init_packet(pkt);return pkt; }

2. void av_packet_free(AVPacket **pkt);

解釋

釋放AVPacket和_alloc對應。內部調用了void av_packet_unref(AVPacket *pkt);

源碼

void av_packet_free(AVPacket **pkt) {if (!pkt || !*pkt)return;av_packet_unref(*pkt);av_freep(pkt); }

3. void av_init_packet(AVPacket *pkt);

解釋

初始化AVPacket，只是單純初始化pkt字段.

源碼

void av_init_packet(AVPacket *pkt) {pkt->pts = AV_NOPTS_VALUE;pkt->dts = AV_NOPTS_VALUE;pkt->pos = -1;pkt->duration = 0; #if FF_API_CONVERGENCE_DURATION FF_DISABLE_DEPRECATION_WARNINGSpkt->convergence_duration = 0; FF_ENABLE_DEPRECATION_WARNINGS #endifpkt->flags = 0;pkt->stream_index = 0;pkt->buf = NULL;pkt->side_data = NULL;pkt->side_data_elems = 0; }

4. int av_new_packet(AVPacket *pkt, int size);

解釋

給AVPacket的AVBufferRef分配內存， 引用計數初始化為1。內部調用了void av_init_packet(AVPacket *pkt)

源碼

int av_new_packet(AVPacket *pkt, int size) {AVBufferRef *buf = NULL;int ret = packet_alloc(&buf, size);if (ret < 0)return ret;av_init_packet(pkt);pkt->buf = buf;pkt->data = buf->data;pkt->size = size;return 0; }

5. int av_packet_ref(AVPacket *dst, const AVPacket *src);

解釋

使用引用計數的淺拷貝

該函數會先拷貝所有非緩存類數據，然后創建一個src->buf的新的引用計數。如果src已經設置了引用計數發（src->buf不為空），則直接將其引用計數+1；
如果src沒有設置引用計數（src->buf為空），則為dst創建一個新的引用計數buf，并復制src->data到dst->buf->data和dst-data中。

最后，復制src的其他字段到dst中。所以av_packet_ref()是將2個AVPacket共用一個緩存的。

源碼

int av_packet_ref(AVPacket *dst, const AVPacket *src) {int ret;dst->buf = NULL;ret = av_packet_copy_props(dst, src);if (ret < 0)goto fail;if (!src->buf) {ret = packet_alloc(&dst->buf, src->size);if (ret < 0)goto fail;av_assert1(!src->size || src->data);if (src->size)memcpy(dst->buf->data, src->data, src->size);dst->data = dst->buf->data;} else {dst->buf = av_buffer_ref(src->buf);if (!dst->buf) {ret = AVERROR(ENOMEM);goto fail;}dst->data = src->data;}dst->size = src->size;return 0; fail:av_packet_unref(dst);return ret; }

6. void av_packet_unref(AVPacket *pkt);

解釋

使用引用計數清理數據

將緩存空間的引用計數-1，并將Packet中的其他字段設為初始值。如果引用計數為0，自動的釋放緩存空間。

源碼

void av_packet_unref(AVPacket *pkt) {av_packet_free_side_data(pkt);av_buffer_unref(&pkt->buf);av_init_packet(pkt);pkt->data = NULL;pkt->size = 0; }

7. void av_packet_move_ref(AVPacket *dst, AVPacket *src);

解釋：

轉移引用計數

把src整個結構體直接賦值給dst，所以引用計數沒有發生變化，并且src被av_init_packet重置

源碼

void av_packet_move_ref(AVPacket *dst, AVPacket *src) {*dst = *src;av_init_packet(src);src->data = NULL;src->size = 0; }

8. AVPacket *av_packet_clone(const AVPacket *src);

解釋

先創建一個新的AVPacket，然后再進行計數引用+數據拷貝，使得新的AVPacket指向老的AVPacket同一個data。

等于av_packet_alloc()+av_packet_ref()

源碼

AVPacket *av_packet_clone(const AVPacket *src) {AVPacket *ret = av_packet_alloc();if (!ret)return ret;if (av_packet_ref(ret, src))av_packet_free(&ret);return ret; }

3. AVPacket Demo

#define MEM_ITEM_SIZE (20*1024*102) #define AVPACKET_LOOP_COUNT 1000 // 測試 內存泄漏 /*** @brief 測試av_packet_alloc和av_packet_free的配對使用*/ void av_packet_test1() {AVPacket *pkt = NULL;int ret = 0;pkt = av_packet_alloc();ret = av_new_packet(pkt, MEM_ITEM_SIZE); // 引用計數初始化為1memccpy(pkt->data, (void *) &av_packet_test1, 1, MEM_ITEM_SIZE);av_packet_unref(pkt); // 要不要調用av_packet_free(&pkt); // 如果不free將會發生內存泄漏,內部調用了 av_packet_unref }/*** @brief 測試誤用av_init_packet將會導致內存泄漏*/ void av_packet_test2() {AVPacket *pkt = NULL;int ret = 0;pkt = av_packet_alloc();ret = av_new_packet(pkt, MEM_ITEM_SIZE);memccpy(pkt->data, (void *) &av_packet_test1, 1, MEM_ITEM_SIZE); // av_init_packet(pkt); // 這個時候init就會導致內存無法釋放av_packet_free(&pkt); }/*** @brief 測試av_packet_move_ref后，可以av_init_packet*/ void av_packet_test3() {AVPacket *pkt = NULL;AVPacket *pkt2 = NULL;int ret = 0;pkt = av_packet_alloc();ret = av_new_packet(pkt, MEM_ITEM_SIZE);memccpy(pkt->data, (void *) &av_packet_test1, 1, MEM_ITEM_SIZE);pkt2 = av_packet_alloc(); // 必須先allocav_packet_move_ref(pkt2, pkt);//內部其實也調用了av_init_packetav_init_packet(pkt);av_packet_free(&pkt);av_packet_free(&pkt2); }/*** @brief 測試av_packet_clone*/ void av_packet_test4() {AVPacket *pkt = NULL;// av_packet_alloc()沒有必要，因為av_packet_clone內部有調用 av_packet_allocAVPacket *pkt2 = NULL;int ret = 0;pkt = av_packet_alloc();ret = av_new_packet(pkt, MEM_ITEM_SIZE);memccpy(pkt->data, (void *) &av_packet_test1, 1, MEM_ITEM_SIZE);pkt2 = av_packet_clone(pkt); // av_packet_alloc()+av_packet_ref()av_init_packet(pkt);av_packet_free(&pkt);av_packet_free(&pkt2); }/*** @brief 測試av_packet_ref*/ void av_packet_test5() {AVPacket *pkt = NULL;AVPacket *pkt2 = NULL;int ret = 0;pkt = av_packet_alloc(); //if (pkt->buf) // 打印referenc-counted，必須保證傳入的是有效指針{printf("%s(%d) ref_count(pkt) = %d\n", __FUNCTION__, __LINE__,av_buffer_get_ref_count(pkt->buf));}ret = av_new_packet(pkt, MEM_ITEM_SIZE);if (pkt->buf) // 打印referenc-counted，必須保證傳入的是有效指針{printf("%s(%d) ref_count(pkt) = %d\n", __FUNCTION__, __LINE__,av_buffer_get_ref_count(pkt->buf));}memccpy(pkt->data, (void *) &av_packet_test1, 1, MEM_ITEM_SIZE);pkt2 = av_packet_alloc(); // 必須先allocav_packet_move_ref(pkt2, pkt); // av_packet_move_ref // av_init_packet(pkt); //av_packet_move_refav_packet_ref(pkt, pkt2);av_packet_ref(pkt, pkt2); // 多次ref如果沒有對應多次unref將會內存泄漏if (pkt->buf) // 打印referenc-counted，必須保證傳入的是有效指針{printf("%s(%d) ref_count(pkt) = %d\n", __FUNCTION__, __LINE__,av_buffer_get_ref_count(pkt->buf));}if (pkt2->buf) // 打印referenc-counted，必須保證傳入的是有效指針{printf("%s(%d) ref_count(pkt) = %d\n", __FUNCTION__, __LINE__,av_buffer_get_ref_count(pkt2->buf));}av_packet_unref(pkt); // 將為2av_packet_unref(pkt); // 做第二次是沒有用的if (pkt->buf)printf("pkt->buf沒有被置NULL\n");elseprintf("pkt->buf已經被置NULL\n");if (pkt2->buf) // 打印referenc-counted，必須保證傳入的是有效指針{printf("%s(%d) ref_count(pkt) = %d\n", __FUNCTION__, __LINE__,av_buffer_get_ref_count(pkt2->buf));}av_packet_unref(pkt2);av_packet_free(&pkt);av_packet_free(&pkt2); }/*** @brief 測試AVPacket整個結構體賦值, 和av_packet_move_ref類似*/ void av_packet_test6() {AVPacket *pkt = NULL;AVPacket *pkt2 = NULL;int ret = 0;pkt = av_packet_alloc();ret = av_new_packet(pkt, MEM_ITEM_SIZE);memccpy(pkt->data, (void *) &av_packet_test1, 1, MEM_ITEM_SIZE);pkt2 = av_packet_alloc(); // 必須先alloc*pkt2 = *pkt; // 有點類似 pkt可以重新分配內存av_init_packet(pkt);av_packet_free(&pkt);av_packet_free(&pkt2); }

---

3. AVFrame常用API

函數原型說明

AVFrame *av_frame_alloc(void);	分配AVFrame
void av_frame_free(AVFrame **frame);	釋放AVFrame
int av_frame_ref(AVFrame *dst, const AVFrame *src);	增加引用計數
void av_frame_unref(AVFrame *frame);	減少引用計數
void av_frame_move_ref(AVFrame *dst, AVFrame *src);	轉移引用計數
int av_frame_get_buffer(AVFrame *frame, int align);	根據AVFrame分配內存
AVFrame *av_frame_clone(const AVFrame *src);	等于av_frame_alloc()+av_frame_ref()

解釋同上

超強干貨來襲 云風專訪：近40年碼齡，通宵達旦的技術人生

總結

以上是生活随笔為你收集整理的ffmpeg内存模型及AVPacket和AVFrame API基本使用的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。