Easy RoCE

easy rdma over converged ethernet \<font color="#1d50a2">urls summary\</font> url get post patch put delete /rest/v1/rpc/qos roce no accept from interface n y n n n /rest/v1/rpc/show counters qos roce n y n n n /rest/v1/rpc/clear counters qos roce n y n n n /rest/v1/rpc/qos roce bind interface n y n n n /rest/v1/rpc/qos roce no bind interface n y n n n /rest/v1/rpc/qos roce publish via interface n y n n n /rest/v1/rpc/qos roce no publish via interface n y n n n /rest/v1/rpc/show qos roce all n y n n n /rest/v1/rpc/show qos roce summary n y n n n /rest/v1/running/roce y n y n n /rest/v1/running/roce/roce accept y n y n n /rest/v1/running/roce/roce profiles/{name} y y y n y \<font color="#1d50a2">examples\</font> \<font color="#1d50a2">get the roce configuration \</font> get https //{switch ip}/rest/v1/running/roce response { "roce accept" {}, "roce profiles" \[ { "name" "roce lossless 5m medium", "mode" "lossless", "bind interfaces" \[ "ethernet4 4", "ethernet11 3", "ethernet24 3" ], "cable length" "cable 5m", "incast level" "medium", "class maps" \[ "roce lossless 5m medium 200g class map" ], "policy maps" \[ "roce lossless 5m medium 200g\@queue scheduler priority queue 6\@queue scheduler priority queue 7" ], "wred profiles" \[ "roce lossless 5m medium 200g ecn" ], "buffer profiles" \[ "roce lossless 5m medium 200g profile" ], "diffserv maps" \[ "roce lossless diffserv map" ] } ] } \<font color="#1d50a2">update the roce accept \</font> patch https //{switch ip}/rest/v1/running/roce/roce accept { "interfaces" \["ethernet1", "ethernet52"] } \<font color="#1d50a2">create a roce profile \</font> post https //{switch ip}/rest/v1/running/roce/roce profiles { "name" "roce lossless", "mode" "lossless" } \<font color="#1d50a2">tree diagrams\</font> + rw roce + rw roce accept \| + rw interfaces + rw roce profiles + rw roce profile \[name] + rw name + rw mode + rw publish via + rw bind interfaces + rw cable length? + rw incast level? + rw traffic model? + ro class maps + ro policy maps + ro wred profiles + ro buffer profiles + ro diffserv maps \<font color="#1d50a2">properties descriptions\</font> \<font color="#1d50a2">roce accept interfaces\</font> tree diagrams + rw roce + rw roce accept \| + rw interfaces table of properties name required type/range description roce m path only node, has no configurable value roce accept m (sub property of roce) path only node, has no configurable value interfaces o ethernet name (sub property of roce accept) accept or not roce configuration from specific port \<font color="#1d50a2">roce profiles\</font> tree diagrams + rw roce + rw roce profiles + rw roce profile \[name] + rw name + rw mode + rw publish via + rw bind interfaces + rw cable length? + rw incast level? + rw traffic model? + ro class maps + ro policy maps + ro wred profiles + ro buffer profiles + ro diffserv maps table of properties name required type/range description roce m path only node, has no configurable value roce profiles m (sub property of roce) path only node, has no configurable value easy roce top level container name m string (sub property of roce profiles) roce profile name; the name normally likes "roce xxx" and it would add three parts if "cable length", "incast level", "traffic model" in body for example, the name will be "roce lossless 5m low latency" if user configures those three parts in body mode m "lossless" "lossy" (sub property of roce profiles) trans mode for roce profile publish via o ethernet name (sub property of roce profiles) publish or not current roce configurations via specific ports bind interfaces o ethernet name (sub property of roce profiles) apply or not current roce configuration on specific ports cable length o "cable 5m" "cable 40m" "cable 100m" "cable 300m" (sub property of roce profiles) cable length on the link incast level o "low" "medium" "high" (sub property of roce profiles) traffic incast level on the link the incast phenomenon refers to a situation where multiple source nodes send data packets to a single destination node, causing these packets to converge on one or several intermediate nodes in the network this results in those nodes or links experiencing excessive traffic pressure we quantify three possible levels low, medium, and high as follows low => 1 1, medium => 3 1, and high => 10 1 these ratios are then used to calculate the ratio of uplink to downlink ports on the switch, which helps determine the pfc (priority based flow control) dynamic threshold (dynamic th) in cases where the actual convergence ratio does not match the quantified results, parameters should be selected based on the lower level for example, a convergence ratio of 2 1 corresponds to low, while a ratio of 8 1 would correspond to medium traffic model o "throughput" "latency" "balance" (sub property of roce profiles) traffic model on the link this parameter offers three options according to the standard configuration manual, for throughput sensitive services, it is recommended to set the ecn max th (maximum threshold for explicit congestion notification) to 10%; for latency sensitive services, it is suggested to set the ecn max th to 90% using the midpoint of these two values, (10% + 90%) / 2 = 50%, we quantify the traffic model into weights consequently, a weight of 0 2 is assigned for throughput sensitive services and a weight of 1 8 for latency sensitive services class maps ro class map name (sub property of roce profiles) class map that profile has bound policy maps ro policy map name (sub property of roce profiles) policy map that profile has bound wred profiles ro wred profile name (sub property of roce profiles) wred profile that profile has bound buffer profiles ro buffer profile name (sub property of roce profiles) buffer profile that profile has bound diffserv maps ro dscp tc map name dot1p tc map name (sub property of roce profiles) diffserv map that profile has bound \<font color="#1d50a2">qos roce accept from interface\</font> tree diagrams + x qos roce accept from interface \| + w input \| | + w interfaces \| + ro output \| + ro data? table of properties name required type/range description qos roce accept from interface m path only node, has no configurable value command "qos roce accept from interface" means platform could receive roce configuration from a part of ethernet ports on the platform interfaces m "all" ethernet name range, eg ethernet1 ethernet10 (sub property of input) accept roce configuration from specific port data ro \<anydata> (sub property of output) \<font color="#1d50a2">qos roce no accept from interface\</font> tree diagrams + x qos roce no accept from interface \| + w input \| | + w interfaces \| + ro output \| + ro data? table of properties name required type/range description qos roce no accept from interface m path only node, has no configurable value command "no qos roce accept from interface" means platform could not receive roce configuration from a part of ethernet ports on the platform any more interfaces m "all" ethernet name range, eg ethernet1 ethernet10 (sub property of input) accept roce configuration from specific port data ro \<anydata> (sub property of output) \<font color="#1d50a2">qos roce bind interface\</font> tree diagrams + x qos roce bind interface \| + w input \| | + w name \| | + w interfaces \| + ro output \| + ro data? table of properties name required type/range description qos roce bind interface m path only node, has no configurable value command "bind interface" means the roce profile under current view would be bound to a part of ethernet ports on the platform name m roce profile name (sub property of input) roce profile name; the name normally likes "roce xxx" interfaces m "all" ethernet name range, eg ethernet1 ethernet10 (sub property of input) apply current roce configuration on specific ports data ro \<anydata> (sub property of output) \<font color="#1d50a2">qos roce no bind interface\</font> tree diagrams + x qos roce no bind interface \| + w input \| | + w name \| | + w interfaces \| + ro output \| + ro data? table of properties name required type/range description qos roce no bind interface m path only node, has no configurable value command "bind interface all" means the roce profile under current view would not be bound to a part of ethernet ports on the platform any more name m roce profile name (sub property of input) roce profile name; the name normally likes "roce xxx" interfaces m "all" ethernet name range, eg ethernet1 ethernet10 (sub property of input) apply current roce configuration on specific ports data ro \<anydata> (sub property of output) \<font color="#1d50a2">qos roce publish via interface\</font> tree diagrams + x qos roce publish via interface \| + w input \| | + w name \| | + w interfaces \| + ro output \| + ro data? table of properties name required type/range description qos roce publish via interface m path only node, has no configurable value command "publish via interface" means platform would publish roce profile configuration to all the ethernet ports on the platform name m roce profile name (sub property of input) roce profile name; the name normally likes "roce xxx" interfaces m "all" ethernet name range, eg ethernet1 ethernet10 (sub property of input) publish or not current roce configurations via specific ports data ro \<anydata> (sub property of output) \<font color="#1d50a2">qos roce no publish via interface\</font> tree diagrams + x qos roce no publish via interface \| + w input \| | + w name \| | + w interfaces \| + ro output \| + ro data? table of properties name required type/range description qos roce no publish via interface m path only node, has no configurable value command "no publish via interface" means platform would not publish roce profile configuration to all the ethernet ports on the platform name m roce profile name (sub property of input) roce profile name; the name normally likes "roce xxx" interfaces m "all" ethernet name range, eg ethernet1 ethernet10 (sub property of input) apply current roce configuration on specific ports data ro \<anydata> (sub property of output) \<font color="#1d50a2">show qos roce all\</font> tree diagrams + x show qos roce all \| + ro output \| + ro roce profiles \| + ro roce profile \[name] \| + ro name \| + ro mode? \| + ro cable length? \| + ro policy maps \| + ro policy map \[name] \| + ro name \| + ro diffserv map type? \| + ro bind interfaces \| + ro congestion control \| | + ro congestion mode? \| | + ro enabled tc \| | + ro min threshold? \| | + ro max threshold? \| + ro priority flow control \| | + ro pfc tc \| | + ro tx status? \| | + ro rx status? \| + ro dscp tc mapping \| | + ro mapping \[dscp] \| | + ro dscp \| | + ro tc? \| + ro dot1 tc mapping \| | + ro mapping \[dot1p] \| | + ro dot1p \| | + ro tc? \| + ro queue schedulers \| | + ro queue scheduler \[queue id] \| | + ro queue id \| | + ro algorithm? \| | + ro weight? \| + ro pfc profiles \| + ro pfc profile \[name] \| + ro name \| + ro switch priority table of properties name required type/range description show qos roce all m path only node, has no configurable value show all the roce profiles configured on the platform roce profiles ro (sub property of output) path only node, has no configurable value name ro roce profile name (sub property of roce profiles) mode ro "lossless" "lossy" (sub property of roce profiles) cable length ro "cable 5m" "cable 40m" "cable 100m" "cable 300m" (sub property of roce profiles) policy maps ro (sub property of roce profiles) path only node, has no configurable value name ro "qos\ qos/policy maps/policy map/name" (sub property of policy maps) diffserv map type ro "dscp" "dot1p" (sub property of policy maps) bind interfaces ro ethernet name (sub property of policy maps) bind status is a boolean which means roce profile is binded to ports or not congestion control ro (sub property of policy maps) path only node, has no configurable value congestion mode ro "ecn" "drop" (sub property of congestion control) enabled tc ro 0 7 (sub property of congestion control) min threshold ro uint64 (sub property of congestion control) max threshold ro uint64 (sub property of congestion control) priority flow control ro (sub property of policy maps) path only node, has no configurable value pfc tc ro 0 7 (sub property of priority flow control) tx status ro "true" "false" (sub property of priority flow control) rx status ro "true" "false" (sub property of priority flow control) dscp tc mapping ro (sub property of policy maps) path only node, has no configurable value only applicable when diffserv map type = "dscp" mapping ro (sub property of dscp tc mapping) path only node, has no configurable value mapping relationship between dscp value and traffic class dscp ro 0 63 (sub property of mapping) the value of dscp tc ro 0 7 (sub property of mapping) the value of traffic class dot1 tc mapping ro (sub property of policy maps) path only node, has no configurable value only applicable when diffserv map type = "dot1p" mapping ro (sub property of dot1 tc mapping) path only node, has no configurable value mapping relationship between dot1p value and traffic class dot1p ro 0 7 (sub property of mapping) the value of dot1p tc ro 0 7 (sub property of mapping) the value of traffic class queue schedulers ro (sub property of policy maps) path only node, has no configurable value only applicable when mode = "lossless" queue id ro 0 7 (sub property of queue schedulers) algorithm ro "strict" "dwrr" (sub property of queue schedulers) weight ro 0 100 (sub property of queue schedulers) only applicable when algorithm = "dwrr" pfc profiles ro (sub property of policy maps) path only node, has no configurable value name ro "qos\ qos/profiles/buffer profiles/buffer profile/name" (sub property of pfc profiles) switch priority ro 0 7 (sub property of pfc profiles) \<font color="#1d50a2">show qos roce summary\</font> tree diagrams + x show qos roce summary \| + ro output \| + ro roce profiles \| + ro roce profile \[name] \| + ro name \| + ro mode? \| + ro bind status? table of properties name required type/range description show qos roce summary m path only node, has no configurable value show summary information of all the roce profiles configured on the platform roce profiles ro (sub property of output) path only node, has no configurable value name ro roce profile name (sub property of roce profiles) mode ro "lossless" "lossy" (sub property of roce profiles) bind status ro "true" "false" (sub property of roce profiles) bind status is a boolean which means roce profile is binded to ports or not \<font color="#1d50a2">show counters qos roce\</font> tree diagrams + x show counters qos roce \| + w input \| | + w interface name \| | + w queue id \| + ro output \| + ro roce states \| | + ro interface name? \| | + ro queue id? \| + ro pfc stats \| | + ro receive stats? \| | + ro transmit stats? \| | + ro pg stats \| | + ro total packets? \| | + ro total bytes? \| | + ro drop packets? \| | + ro curr occupancy? \| + ro ecn stats \| | + ro ecn counters? \| | + ro ecn buffer \| | + ro shared use watermark? \| | + ro total use watermark? \| | + ro total use count? \| + ro queue stats \| + ro counter packets? \| + ro counter bytes? \| + ro drop packets? \| + ro drop bytes? \| + ro counter rate packets? \| + ro counter rate bytes? \| + ro drop rate packets? \| + ro drop rate bytes? \| + ro occupancy bytes? \| + ro shared occupancy bytes? table of properties name required type/range description show counters qos roce m path only node, has no configurable value show qos roce counters interface name m ethernet name (sub property of input) interface name queue id m 0 7 (sub property of input) queue id roce states ro (sub property of output) path only node, has no configurable value interface name ro ethernet name (sub property of roce states) ethernet port name queue id ro 0 7 (sub property of roce states) specfic traffic queue of port pfc stats ro (sub property of output) path only node, has no configurable value receive stats ro uint64 (sub property of pfc stats) pause frame statistics that port has recevied transmit stats ro uint64 (sub property of pfc stats) pause frame statistics that port has transmitted pg stats ro (sub property of pfc stats) path only node, has no configurable value total packets ro uint64 (sub property of pg stats) the total number of packets received by this priority group since the last polling (in packets) total bytes ro uint64 (sub property of pg stats) counter/bytes the total number of bytes received by this priority group since the last polling (in bytes) drop packets ro uint64 (sub property of pg stats) the number of packets dropped by this priority group due to queue overflow or policies (such as red/pfcwd drop) since the last polling curr occupancy ro uint64 (sub property of pg stats) the current or peak occupancy level of the per pg buffer, in bytes ecn stats ro (sub property of output) path only node, has no configurable value ecn counters ro uint64 (sub property of ecn stats) statistics of packets which is marked by wred ecn function ecn buffer ro (sub property of ecn stats) path only node, has no configurable value shared use watermark ro uint64 (sub property of ecn buffer) the maximum number of bytes occupied by this priority group in the shared buffer pool since the last water level reset total use watermark ro uint64 (sub property of ecn buffer) the maximum number of bytes occupied by the priority group in its private or total buffer total use count ro uint64 (sub property of ecn buffer) the priority group accumulates (total) the number of bytes read or written from the dedicated buffer during the query cycle queue stats ro (sub property of output) path only node, has no configurable value counter packets ro uint64 (sub property of queue stats) the total number of successfully joined (or sent) messages in this queue since power on or reset counter bytes ro uint64 (sub property of queue stats) the corresponding cumulative number of bytes (including frame headers), which is the total number of bytes occupied by these messages drop packets ro uint64 (sub property of queue stats) the total number of messages that have been discarded in the queue due to congestion, policies (such as wred, pfc exceeding limits), and other reasons drop bytes ro uint64 (sub property of queue stats) the cumulative number of bytes corresponding to the discarded message counter rate packets ro 0 0 max (sub property of queue stats) the number of successfully queued messages per second counter rate bytes ro 0 0 max (sub property of queue stats) the number of bytes successfully logged in per second drop rate packets ro 0 0 max (sub property of queue stats) the number of messages discarded per second drop rate bytes ro 0 0 max (sub property of queue stats) the number of bytes discarded per second occupancy bytes ro uint64 (sub property of queue stats) indicates the size of the local buffer currently in use for the queue (in bytes), which is equivalent to the total size of unprocessed messages in the queue buffer occupancy can be considered as buffer occupancy the amount of data in the queue that has not yet left the queue shared occupancy bytes ro uint64 (sub property of queue stats) when the device uses a shared buffer pool, this field displays the number of bytes occupied by the queue in the shared pool the shared buffer mechanism allows multiple queues to dynamically request larger memory to cope with sudden traffic \<font color="#1d50a2">clear counters qos roce\</font> tree diagrams + x clear counters qos roce + ro output + ro data? table of properties name required type/range description clear counters qos roce m path only node, has no configurable value clear qos roce counters data ro \<anydata> (sub property of output)