[dev][dpdk][crypto] dpdk加解密設備與IPSEC
概述
分三部分，加解密框架（crypto framework），加解密設備（crypto dev），安全協議（Security Framework）
× API，設計思路等，都在加解密框架里：見文檔：http://doc.dpdk.org/guides-18.11/prog_guide/cryptodev_lib.html
× 設備層的事情，加解密設備的分類，調度，主備等，見文檔：http://doc.dpdk.org/guides-18.11/cryptodevs/index.html
× ESP，PDCP等網絡加密協議相關的事情，主要功能是整合NIC pmd和Crypto pmd，見文檔：http://doc.dpdk.org/guides-18.11/prog_guide/rte_security.html
另外還有一個追加的部分，ipsec （rte_ipsec）
* 這一部分主要是對前面三個部分的整合封裝調用，用來專門處理ipsec報文。見文檔：http://doc.dpdk.org/guides/prog_guide/ipsec_lib.html
畫個圖:
如果，以上三個文檔，你都讀完了，那么請不要再往下讀了。內容重復。且本人懶惰，多以摘要為主，寫的也不好 ：）
一般編程模型
畫了一張圖，還沒完全整理好，后續有可能會再更新，也有可能不更新了。
是參考dpdk的這個例子里的流程畫的：http://doc.dpdk.org/guides/sample_app_ug/ipsec_secgw.html
初始化加密設備的一般流程。
設備就緒后的一般使用流程。（需要關注的是：在分析rte_ipsec的源碼過程中，并沒有發現對API rte_crypto_op_attach_sym_session的調用。奇怪。。。）
// TODO init dpdk crypto things. // 1. init sa->xforms like function sa_add_rules() // 2. init two parameter ipsec_xform/crypto_xform // 2. ipsec_sa_init() // 2.1 rte_ipsec_sa_init() // 3. create_session // 3.1 rte_security_session_create() // 3.2 rte_cryptodev_sym_session_create() // 3.3 rte_cryptodev_sym_session_init() // 3.4 rte_ipsec_session_prepare() // 3.4.1 sa->ips->pkt_func = ipsec_sa_pkt_func_select() // 3.5 rte_ipsec_pkt_process() // 3.5.1 sa->ips->pkt_func.process() // 3.6 rte_ipsec_pkt_crypto_prepare() // 3.6.1 sa->ips->pkt_func.prepare() // 7 rte_cryptodev_enqueue_burst() // 8.rte_cryptodev_dequeue_burst() // 9 rte_ipsec_pkt_crypto_group() // 10 rte_ipsec_pkt_process() // 3.5.1 sa->ips->pkt_func.process() // outbound // 4 rte_security_attach_session() // 5 rte_crypto_op_attach_sym_session() // 6 rte_security_set_pkt_metadata() // 7.rte_cryptodev_enqueue_burst() // 8 rte_cryptodev_dequeue_burst()
第二部分，設備（crypto dev）
所有的加解密設備大概分為以下幾種：openssl，null，硬件架構相關的設備。
null為純軟件的最小實現，可以用來調試等。
硬件相關的主要包括，Intel，arm，NXP，AMD，QAT卡等。
還有一種，基于VIRTIO的PMD設備。
不同的設備，所支持的加解密算法也各有不同，有一個詳細的對比列表，見：
http://doc.dpdk.org/guides-18.11/cryptodevs/overview.html
設備調度
在各設備的上層，還有一個調度設備，用于管理和協調多個加解密設備直接的數據流。叫做 cryptodev scheduler PMD
http://doc.dpdk.org/guides-18.11/cryptodevs/scheduler.html
接口API
加解密設備的調度API（schduler）
http://doc.dpdk.org/api-18.11/rte__cryptodev__scheduler_8h.html
加解密設備的使用API （cryptodev）
http://doc.dpdk.org/api-18.11/rte__cryptodev_8h.html
第一部分 框架（crypto framework）
架構設計文檔：http://doc.dpdk.org/guides-18.11/prog_guide/cryptodev_lib.html
設計理念
The cryptodev library follows the same basic principles as those used in DPDKs Ethernet Device framework.
The Crypto framework provides a generic Crypto device framework which supports both physical (hardware)
and virtual (software) Crypto devices as well as a generic Crypto API which allows Crypto devices to be
managed and configured and supports Crypto operations to be provisioned on Crypto poll mode driver.
設備初始化方法
A。 初始化設備
實體設備：與PCI網卡相同。
虛擬設備：1. 可以使用 --vdev參數在dpdk程序啟動時使用。2. 在運行時使用api：
rte_vdev_init("crypto_aesni_mb", "max_nb_queue_pairs=2,socket_id=0")
B。配置設備
設備識別：用Id或name
設備配置：使用api：
int rte_cryptodev_configure(uint8_t dev_id, struct rte_cryptodev_config *config) struct rte_cryptodev_config { int socket_id; /**< Socket to allocate resources on */ uint16_t nb_queue_pairs; /**< Number of queue pairs to configure on device */ };
C。配置queue
int rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, const struct rte_cryptodev_qp_conf *qp_conf, int socket_id) struct rte_cryptodev_qp_conf { uint32_t nb_descriptors; /**< Number of descriptors per queue pair */ };
D。數據流程
1. 在queue pair 上burst 提取數據。
uint16_t rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op **ops, uint16_t nb_ops) uint16_t rte_cryptodev_dequeue_burst(uint8_t dev_id, uint16_t qp_id, struct rte_crypto_op **ops, uint16_t nb_ops)
2. 私有數據的存儲。分基于session的和不基于session的兩種情況。
3. 關鍵的用于加解密操作的數據結構 rte_crypto_op, 可以把op理解為mbuf，op是mempool管理的。
4. op的mempool的API
extern struct rte_mempool * rte_crypto_op_pool_create(const char *name, enum rte_crypto_op_type type, unsigned nb_elts, unsigned cache_size, uint16_t priv_size, int socket_id); struct rte_crypto_op *rte_crypto_op_alloc(struct rte_mempool *mempool, enum rte_crypto_op_type type) unsigned rte_crypto_op_bulk_alloc(struct rte_mempool *mempool, enum rte_crypto_op_type type, struct rte_crypto_op **ops, uint16_t nb_ops) void rte_crypto_op_free(struct rte_crypto_op *op)
5. session
session用來存儲加密過程中的key，以及分組信息等。
用來存儲這些的叫做private session data
api：
rte_cryptodev_sym_session_create()
rte_cryptodev_sym_session_init()
rte_cryptodev_sym_session_clear()
rte_cryptodev_sym_session_free（）
6. transforms
Currently there are three transforms types cipher, authentication and AEAD.
Also it is important to note that the order in which the transforms are passed indicates the order of the chaining.
struct rte_crypto_sym_xform { struct rte_crypto_sym_xform *next; /**< next xform in chain */ enum rte_crypto_sym_xform_type type; /**< xform type */ union { struct rte_crypto_auth_xform auth; /**< Authentication / hash xform */ struct rte_crypto_cipher_xform cipher; /**< Cipher xform */ struct rte_crypto_aead_xform aead; /**< AEAD xform */ }; };
7. 運行狀態下的最小配置
As a minimum the symmetric operation must have a source data buffer (m_src), a valid session (or transform chain if in session-less mode) and the minimum authentication/ cipher/ AEAD parameters
required depending on the type of operation specified in the session or the transform chain.
示例程序：
/* * Simple example to encrypt several buffers with AES-CBC using * the Cryptodev APIs. */ #define MAX_SESSIONS 1024 #define NUM_MBUFS 1024 #define POOL_CACHE_SIZE 128 #define BURST_SIZE 32 #define BUFFER_SIZE 1024 #define AES_CBC_IV_LENGTH 16 #define AES_CBC_KEY_LENGTH 16 #define IV_OFFSET (sizeof(struct rte_crypto_op) + \ sizeof(struct rte_crypto_sym_op)) struct rte_mempool *mbuf_pool, *crypto_op_pool, *session_pool; unsigned int session_size; int ret; /* Initialize EAL. */ ret = rte_eal_init(argc, argv); if (ret < 0) rte_exit(EXIT_FAILURE, "Invalid EAL arguments\n"); uint8_t socket_id = rte_socket_id(); /* Create the mbuf pool. */ mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, POOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, socket_id); if (mbuf_pool == NULL) rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n"); /* * The IV is always placed after the crypto operation, * so some private data is required to be reserved. */ unsigned int crypto_op_private_data = AES_CBC_IV_LENGTH; /* Create crypto operation pool. */ crypto_op_pool = rte_crypto_op_pool_create("crypto_op_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, NUM_MBUFS, POOL_CACHE_SIZE, crypto_op_private_data, socket_id); if (crypto_op_pool == NULL) rte_exit(EXIT_FAILURE, "Cannot create crypto op pool\n"); /* Create the virtual crypto device. */ char args[128]; const char *crypto_name = "crypto_aesni_mb0"; snprintf(args, sizeof(args), "socket_id=%d", socket_id); ret = rte_vdev_init(crypto_name, args); if (ret != 0) rte_exit(EXIT_FAILURE, "Cannot create virtual device"); uint8_t cdev_id = rte_cryptodev_get_dev_id(crypto_name); /* Get private session data size. */ session_size = rte_cryptodev_sym_get_private_session_size(cdev_id); /* * Create session mempool, with two objects per session, * one for the session header and another one for the * private session data for the crypto device. */ session_pool = rte_mempool_create("session_pool", MAX_SESSIONS * 2, session_size, POOL_CACHE_SIZE, 0, NULL, NULL, NULL, NULL, socket_id, 0); /* Configure the crypto device. */ struct rte_cryptodev_config conf = { .nb_queue_pairs = 1, .socket_id = socket_id }; struct rte_cryptodev_qp_conf qp_conf = { .nb_descriptors = 2048 }; if (rte_cryptodev_configure(cdev_id, &conf) < 0) rte_exit(EXIT_FAILURE, "Failed to configure cryptodev %u", cdev_id); if (rte_cryptodev_queue_pair_setup(cdev_id, 0, &qp_conf, socket_id, session_pool) < 0) rte_exit(EXIT_FAILURE, "Failed to setup queue pair\n"); if (rte_cryptodev_start(cdev_id) < 0) rte_exit(EXIT_FAILURE, "Failed to start device\n"); /* Create the crypto transform. */ uint8_t cipher_key[16] = {0}; struct rte_crypto_sym_xform cipher_xform = { .next = NULL, .type = RTE_CRYPTO_SYM_XFORM_CIPHER, .cipher = { .op = RTE_CRYPTO_CIPHER_OP_ENCRYPT, .algo = RTE_CRYPTO_CIPHER_AES_CBC, .key = { .data = cipher_key, .length = AES_CBC_KEY_LENGTH }, .iv = { .offset = IV_OFFSET, .length = AES_CBC_IV_LENGTH } } }; /* Create crypto session and initialize it for the crypto device. */ struct rte_cryptodev_sym_session *session; session = rte_cryptodev_sym_session_create(session_pool); if (session == NULL) rte_exit(EXIT_FAILURE, "Session could not be created\n"); if (rte_cryptodev_sym_session_init(cdev_id, session, &cipher_xform, session_pool) < 0) rte_exit(EXIT_FAILURE, "Session could not be initialized " "for the crypto device\n"); /* Get a burst of crypto operations. */ struct rte_crypto_op *crypto_ops[BURST_SIZE]; if (rte_crypto_op_bulk_alloc(crypto_op_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, crypto_ops, BURST_SIZE) == 0) rte_exit(EXIT_FAILURE, "Not enough crypto operations available\n"); /* Get a burst of mbufs. */ struct rte_mbuf *mbufs[BURST_SIZE]; if (rte_pktmbuf_alloc_bulk(mbuf_pool, mbufs, BURST_SIZE) < 0) rte_exit(EXIT_FAILURE, "Not enough mbufs available"); /* Initialize the mbufs and append them to the crypto operations. */ unsigned int i; for (i = 0; i < BURST_SIZE; i++) { if (rte_pktmbuf_append(mbufs[i], BUFFER_SIZE) == NULL) rte_exit(EXIT_FAILURE, "Not enough room in the mbuf\n"); crypto_ops[i]->sym->m_src = mbufs[i]; } /* Set up the crypto operations. */ for (i = 0; i < BURST_SIZE; i++) { struct rte_crypto_op *op = crypto_ops[i]; /* Modify bytes of the IV at the end of the crypto operation */ uint8_t *iv_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, IV_OFFSET); generate_random_bytes(iv_ptr, AES_CBC_IV_LENGTH); op->sym->cipher.data.offset = 0; op->sym->cipher.data.length = BUFFER_SIZE; /* Attach the crypto session to the operation */ rte_crypto_op_attach_sym_session(op, session); } /* Enqueue the crypto operations in the crypto device. */ uint16_t num_enqueued_ops = rte_cryptodev_enqueue_burst(cdev_id, 0, crypto_ops, BURST_SIZE); /* * Dequeue the crypto operations until all the operations * are proccessed in the crypto device. */ uint16_t num_dequeued_ops, total_num_dequeued_ops = 0; do { struct rte_crypto_op *dequeued_ops[BURST_SIZE]; num_dequeued_ops = rte_cryptodev_dequeue_burst(cdev_id, 0, dequeued_ops, BURST_SIZE); total_num_dequeued_ops += num_dequeued_ops; /* Check if operation was processed successfully */ for (i = 0; i < num_dequeued_ops; i++) { if (dequeued_ops[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) rte_exit(EXIT_FAILURE, "Some operations were not processed correctly"); } rte_mempool_put_bulk(crypto_op_pool, (void **)dequeued_ops, num_dequeued_ops); } while (total_num_dequeued_ops < num_enqueued_ops);
第三部分，Security Framework
前邊的內容，都在講加解密。但是實際上呢，我是想要處理IPSEC的報文。下面這個庫，將搞定這個事情：
http://doc.dpdk.org/guides-18.11/prog_guide/rte_security.html
既然是ipsec，所以，自然用到nic和crypto兩部分的pmd
+---------------+
| rte_security |
+---------------+
\ /
+-----------+ +--------------+
| NIC PMD | | CRYPTO PMD |
+-----------+ +--------------+
硬件offload
加解密offload
就是說收到的包就解好了密，發包之前不需要進行包加密的意思。這個是硬件offload支持的，不支持的話，還是需要軟件去把包送給crypto dev 處理。
協議offload
區別于加解密offload，ESP的包頭之類的，也可以offload的給硬件。
lookaside offload
處理前邊提到那兩個，就連sa的管理，也可以直接offload。
設備能力
ipsec的封解包，是由device決定的。不同的device可能不一定支持，需要通過下面的方法具體查看。這里的device即包括了eth也包括了crypto
The device (crypto or ethernet) capabilities which support security operations, are defined by the security action type, security protocol,
protocol capabilities and corresponding crypto capabilities for security. For the full scope of the Security capability see definition of
rte_security_capability structure in the DPDK API Reference.
API：
const struct rte_security_capability *rte_security_capabilities_get(uint16_t id);
安全會話（security session）
Security Sessions are created to store the immutable fields of a particular Security Association for a particular protocol which is defined by a security session
configuration structure which is used in the operation processing of a packet flow. Sessions are used to manage protocol specific information as well as crypto
parameters. Security sessions cache this immutable data in a optimal way for the underlying PMD and this allows further acceleration of the offload of Crypto workloads.
session也是mempool里的內存
sessions are mempool objects.
API：
http://doc.dpdk.org/api/rte__security_8h.html
rte_security_session_create()
rte_cryptodev_get_sec_ctx() or rte_eth_dev_get_sec_ctx（）
rte_security_session_destroy()
關鍵結構體
rte_security_session_conf
struct rte_security_session_conf { enum rte_security_session_action_type action_type; /**< Type of action to be performed on the session */ enum rte_security_session_protocol protocol; /**< Security protocol to be configured */ union { struct rte_security_ipsec_xform ipsec; struct rte_security_macsec_xform macsec; struct rte_security_pdcp_xform pdcp; }; /**< Configuration parameters for security session */ struct rte_crypto_sym_xform *crypto_xform; /**< Security Session Crypto Transformations */ void *userdata; /**< Application specific userdata to be saved with session */ };
enum rte_security_session_action_type { RTE_SECURITY_ACTION_TYPE_NONE, /**< No security actions */ RTE_SECURITY_ACTION_TYPE_INLINE_CRYPTO, /**< Crypto processing for security protocol is processed inline * during transmission */ RTE_SECURITY_ACTION_TYPE_INLINE_PROTOCOL, /**< All security protocol processing is performed inline during * transmission */ RTE_SECURITY_ACTION_TYPE_LOOKASIDE_PROTOCOL /**< All security protocol processing including crypto is performed * on a lookaside accelerator */ };
其他
如果是NIC設備的offload，必須注意flow和加解密序列和port直接的對應關系。
In the case of NIC based offloads, the security session specified in the ‘rte_flow_action_security’
must be created on the same port as the flow action that is being specified.
第四部分 RTE_IPSEC
文檔： http://doc.dpdk.org/guides/prog_guide/ipsec_lib.html
一般使用流程
/* enqueue for processing by crypto-device */ rte_ipsec_pkt_crypto_prepare(...); rte_cryptodev_enqueue_burst(...); /* dequeue from crypto-device and do final processing (if any) */ rte_cryptodev_dequeue_burst(...); rte_ipsec_pkt_crypto_group(...); /* optional */ rte_ipsec_pkt_process(...);
api：
這個還沒發布。。。 在master分支上，模塊名字叫librte_ipsec
詳細的用法可以見頭文件： http://git.dpdk.org/dpdk/tree/lib/librte_ipsec/rte_ipsec.h?h=releases
浙公網安備 33010602011771號