RDMA的ibv_post_send() 函数

函数原型为

int ibv_post_send(struct ibv_qp *qp, struct ibv_send_wr *wr,
                  struct ibv_send_wr **bad_wr);

其中struct ibv_send_wr结构体的定义为：

struct ibv_send_wr {
    uint64_t        wr_id;
    struct ibv_send_wr     *next;
    struct ibv_sge           *sg_list;
    int            num_sge;
    enum ibv_wr_opcode    opcode;
    int            send_flags;
    uint32_t        imm_data;
    union {
        struct {
            uint64_t    remote_addr;
            uint32_t    rkey;
        } rdma;
        struct {
            uint64_t    remote_addr;
            uint64_t    compare_add;
            uint64_t    swap;
            uint32_t    rkey;
        } atomic;
        struct {
            struct ibv_ah  *ah;
            uint32_t    remote_qpn;
            uint32_t    remote_qkey;
        } ud;
    } wr;
};

在ibv_send_wr 结构体中opcode参数决定了数据传输类型，比如说：

IBV_WR_SEND——在这种类型中，The content of the local memory buffers specified in sg_list is being sent to the remote QP. The sender doesn’t know where the data will be written in the remote node. A Receive Request will be consumed from the head of remote QP's Receive Queue and sent data will be written to the memory buffers which are specified in that Receive Request.当前内存buffer中在sg_list中的内容会被发送给远方的QP。发送方并不会知道数据会写到远方节点的何处。接收方要post_recv，并且接收到的数据要放到指定的地址中。

IBV_WR_RDMA_WRITE——在这种类型中，The content of the local memory buffers specified in sg_list is being sent and written to a contiguous block of memory range in the remote QP's virtual space. This doesn't necessarily means that the remote memory is physically contiguous. No Receive Request will be consumed in the remote QP.本地内存buffer中sg_list中的内容会被发送和写到远方节点的QP的虚拟空间中的一段连续内存块中——这并不意味着远方的内存在物理上也是连续的。并且remote QP也不需要post_recv。