DetNet Shaofu. Peng Internet-Draft ZTE Intended status: Standards Track Peng. Liu Expires: 22 December 2024 China Mobile Kashinath. Basu Oxford Brookes University Aihua. Liu ZTE Dong. Yang Beijing Jiaotong University Guoyu. Peng Beijing University of Posts and Telecommunications 20 June 2024 Timeslot Queueing and Forwarding (TQF) Control Plane draft-peng-detnet-tqf-controller-plane-00 Abstract To achive DetNet QoS in IP/MPLS network and meet the large scaling requirements, timeslot queueing and forwarding (TQF) mechanism for enhancing TAS is introduced. This document describes the controller plane function (CPF) for TQF mechanism. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 22 December 2024. Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. Peng, et al. Expires 22 December 2024 [Page 1] Internet-Draft TQF Control Plane June 2024 This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 2. TQF Path Calculation and Timeslot Resource Reservation . . . 3 2.1. Timeslot Resource Definition . . . . . . . . . . . . . . 4 2.2. Arrival Postion in the Orchestration Period of UNI . . . 6 2.3. Proccess of Each Reservation Sub-task . . . . . . . . . . 9 2.3.1. Resource Reservation on the Ingress Node . . . . . . 10 2.3.2. Resource Reservation on the Transit Node . . . . . . 12 2.3.3. Resource Reservation on the Egress Node . . . . . . . 13 3. Multiple Orchestration Periods . . . . . . . . . . . . . . . 13 4. Flow Aggregation and De-aggregation . . . . . . . . . . . . . 13 5. Provision Flow Identification Information . . . . . . . . . . 14 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 7. Security Considerations . . . . . . . . . . . . . . . . . . . 14 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 14 9.1. Normative References . . . . . . . . . . . . . . . . . . 14 9.2. Informative References . . . . . . . . . . . . . . . . . 15 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15 1. Introduction DetNet (Deterministic Networking) provides the ability to carry specified unicast or multicast data flows for real-time applications with extremely low packet loss rates and assured maximum end-to-end delivery latency. A description of the general background and concepts of DetNet can be found in [RFC8655]. In particular, [RFC8655] defines the Controller Plane Function (CPF), which is in charge of computing deterministic paths to be applied in the Network Plane. CPF refers to any device operating in the Controller Plane, whether it is a Path Computation Element (PCE) [RFC4655], a Network Management Entity (NME), or a distributed control protocol. To achive DetNet QoS in IP/MPLS network and meet the large scaling requirements, [I-D.peng-detnet-packet-timeslot-mechanism] introduces timeslot queueing and forwarding (TQF) mechanism for enhancing IEEE 802.1 TSN TAS [TAS] (e.g., avoiding time synchronization, timeslot Peng, et al. Expires 22 December 2024 [Page 2] Internet-Draft TQF Control Plane June 2024 based queue allocation rule). It needs to bring timeslot type of resources to layer-3 and construct timeslot resources on each link within the repeated gating cycle (also termed as Orchestration Period). By carefully interleaving flows in different timeslots in the entire network, TQF can improve flow scale. This document describes the controller plane function (CPF) for TQF mechanism. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 2. TQF Path Calculation and Timeslot Resource Reservation A centralized controller or the network entrance node may calculate a DetNet path which using TQF scheduling mechanism and can be abbreviated as a TQF path. A TQF path can provide bounded end-to-end latency, bounded end-to-end latency jitter, and consume certain burst and bandwidth resources. A single TQF path may carry multiple DetNet flows. The centralized controller, or network nodes involved (through RSVP- TE [RFC3209], can reserve corresponding timeslot resources along the TQF path. On each node, for a given incoming timeslot and the reserved outgoing timeslot, an evaluation of node residence delay can be obtained. If a path carries multiple DetNet flows, it may reserve timeslot resources for the aggregated DetNet flow, and may reserve the burst resources in multiple timeslots in the orchestration period at the same time. However, it would still be beneficial to distinguish between reservation sub-tasks corresponding to different DetNet flows in the combined reservation task. In this document, we refer to a reservation sub-task as an individual timeslot resource reservation action related to a DetNet flow. Note that one or more reservation sub-tasks for a specific DetNet flow may be derived based on its TSpec, and each reservation sub-task will allocate corresponding timeslot. The intermediate nodes do not maintain the state of DetNet flow and only reserve timeslot resources based on the reservation sub-tasks. Peng, et al. Expires 22 December 2024 [Page 3] Internet-Draft TQF Control Plane June 2024 During resource reservation, it is necessary to distinguish the requirements between low latency service and non-low latency service. For low latency service requirements, the physical offset between the reserved outgoing timeslot and the incoming timeslot is small; while for loose latency service requirements, this physical offset can be large. It is necessary to maintain the end-to-end total residence delay budget for each reservation sub-task. This is used to select outgoing timeslot at each node. The sum of residence delays caused by all nodes should not exceed the total residence delay budget. Multiple reservation sub-tasks may generate different incoming/ outgoing timeslot mapping relationships on the node. For example: * The timeslot mapping relationship created by the sub-task-1: <(incoming port a, incoming slot id 3), (outgoing port b, outgoing slot id 60)> * The timeslot mapping relationship created by the sub-task-2: <(incoming port a, incoming slot id 3), (outgoing port b, outgoing slot id 61)> Special care should be taken not to confuse the use of different mapping relationships for different DetNet flows. It is recommended, but not mandatory, to reserve timeslot resources on the outgoing port of each hop from the headend of the path to the endpoint, that is, first determine the timeslot reserved on the headend, then determine the timeslot reserved on the next hop , and so on. A DetNet flow is assumed to have a periodic arrival time (i.e., the time when the regulated packet arrived at the scheduler), and there is an ideal position relationship between the arrival time and the orchestration period of the UNI, so selecting the outgoing timeslot closed to the arrival time or within the expected offset range in the orchestration period of NNI can minimize the residency delay on the headend. However, sometimes it is necessary to get a larger residence delay on the headend and a smaller residence delay on other nodes to ensure successful path calculation. 2.1. Timeslot Resource Definition The timeslot resources of a link can be represented as the corresponding bit amounts of all timeslots included in an orchestration period. Basically, the link capability should contain the following information: Peng, et al. Expires 22 December 2024 [Page 4] Internet-Draft TQF Control Plane June 2024 * Timeslot Length (TL): Represents the length of the timeslot, in units of us. Generally, the length of each timeslot included in the orchestration period is the same. * Orchestration Period Length (OPL): Represents the length of the orchestration period, in units of us. The orchestration period contains N timeslots, numbered sequentially from 0 to N-1. That is, OPL = N*TL. * Scheduling Period Length (SPL): Represents the length of the scheduling period, in units of us. The scheduling period contains M timeslots, numbered sequentially from 0 to M-1. That is, SPL = M*TL.. Figure 1 shows the timeslot resource model of the link, with an orchestration period instance consisting of N timeslots numbered from 0 to N-1. The resource information of each timeslot includes the following attributes: * Timeslot ID: Indicates the NO. of the timeslot in the orchestration period instance. The NO. of the first timeslot is 0, and the NO. of the last timeslot is N-1. * Maximum Reservable Bursts (MRBur): Refers to the maximum amount of bit quota corresponding to this timeslot, with unit of bits. It is a configurable preset value that is related to the service rate (termed as C) and the timeslot length (termed as TL), and the Maximum Reservable Bursts should be set to a value not exceeding C*TL. Generally, the Maximum Reservable Bursts of each timeslot included in the orchestration period are all the same. * Unreserved Bursts (UBur): Refers to the amount of unreserved bits reservable corresponding to the timeslot, with unit of bits. Peng, et al. Expires 22 December 2024 [Page 5] Internet-Draft TQF Control Plane June 2024 #N-1 +---------------------------------------+ | Timeslot Length: TL(n-1) | | Maximum Reservable Bursts: MRBur(n-1) | | Unreserved Bursts: UBur(n-1) | +---------------------------------------+ ... ... ... ... ... ... #1 +---------------------------------------+ | Timeslot Length: TL(1) | | Maximum Reservable Bursts: MRBur(1) | | Unreserved Bursts: UBur(1) | +---------------------------------------+ #0 +---------------------------------------+ | Timeslot Length: TL(0) | | Maximum Reservable Bursts: MRBur(0) | | Unreserved Bursts: UBur(0) | +---------------------------------------+ -----------------------------------------------------------> Timeslot Resources of an TQF Instance of the Link Figure 1: Timeslot Resources Model The IGP/BGP extensions to advertise the link's capability and timeslot resource is defined in [I-D.peng-lsr-deterministic-traffic-engineering]. 2.2. Arrival Postion in the Orchestration Period of UNI On the network entrance node, a DetNet flow, after policing, will release sub-bursts to the network, with flow pattern that is evenly distributed within the service burst interval. Each regulated sub- burst will fall into the ideal incoming timeslot of UNI. Based on the ideal incoming timeslot, the corresponding ideal outgoing timeslot of NNI port can be reserved for the sub-burst. For example, if a DetNet flow distributes m sub-bursts during the orchestration period, the network entry should maintain m states for that flow: * * * ... ... Peng, et al. Expires 22 December 2024 [Page 6] Internet-Draft TQF Control Plane June 2024 * However, the packets arrived at the network entry are not always ideal, and the departure time from regulator may not be in a certain ideal incoming timeslot. Therefore, an important operation that needs to be performed by the network entry is to determine the ideal incoming timeslot i based on the actual departure time. This can first determine the actual incoming timeslot based on the actual departure time, and then select an ideal incoming timeslot that is closest to the actual incoming timeslot and not earlier than the actual incoming timeslot. Figure 2 shows, for some typical DetNet flows, the relationship between the service burst interval (SBI) and the orchestration period length (OPL) of UNI, as well as the possible timeslot resource reservation on NNI for these DetNet flows. |<--------------------- OPL ---------------------->| +----+----+----+----+----+----+----+----------+----+ | #0 | #1 | #2 | #3 | #4 | #5 | #6 | ... ... |#N-1| +----+----+----+----+----+----+----+----------+----+ +--+ Flow 1: | |b1| | +-----+--+-----------------------------------------+ |<------------------- SBI ------------------------>| +--+ +--+ Flow 2: | |b1| |b2| +------------+--+------------------------+--+------+ |<------------------- SBI ------------------------>| +------+ Flow 3: | | b1 | +---------------------------+------+---------------+ |<------------------- SBI ------------------------>| +--+ +--+ +--+ Flow 4: | |b1| | |b1| | |b1| | +----+--+--------+----+--+--------+----+--+--------+ |<----- SBI ---->|<----- SBI ---->|<----- SBI ---->| Figure 2: Relationship between SBI and OP Peng, et al. Expires 22 December 2024 [Page 7] Internet-Draft TQF Control Plane June 2024 As shown in the figure, the length of service burst intervals for flows 1, 2, 3 is equal to the length of orchestration period, while the length of the service burst interval for flow 4 is only 1/3 of the orchestration period. * Flow 1 generates a small single burst amounts within its burst interval, which may reserve timeslot 2 or other subsequent timeslot in the orchestration period; * Flow 2 generates two small discrete sub-bursts within its burst interval and also be shaped, which may reserve slots 4 and N-1 in the orchestration period for each sub-burst respectively; * Flow 3 generates a large single burst amount within its burst interval but not be really shaped (due to purchasing a larger burst resource and served by a larger bucket depth), which may also be split to multiple back-to-back sub-bursts and reserve multiple timeslots in the orchestration period, such as timeslots 8 and 9. * The length of the service burst interval for flow 4 is only 1/3 of the orchestration period. Hence, construct flow 4' with 3 occurrence of the flow 4 within an orchestration period. So flow 4' is similar to flow 2, generating a small amount of three separate sub-bursts within its burst interval. It may reserve timeslots 3, 7, and N-1 in the orchestration period. Each sub-burst corresponds to a reservation sub-task. For simplicity, each regulated sub-burst in the service burst interval always reserves timeslot resources according to the maximum sub-bust size. For a specific DetNet flow, to determine how many reservation sub- tasks are required, can be summarized as: * First, align the service burst interval with the orchestration period of UNI to ensure that the two are of equal length. If the service burst interval is only a fraction of the orchestration period, multiply it several times to obtain the expanded service burst interval to get a new flow'. * Check how many discrete sub-bursts will be generated during the orchestration period, and for each sub-burst: - If the proportion of the sub-burst size to the MRB of a single timeslot does not exceed a specific value, then the sub-burst corresponds to a reservation sub-task; Peng, et al. Expires 22 December 2024 [Page 8] Internet-Draft TQF Control Plane June 2024 - Otherwise, continue to split the sub-burst into multiple sub- sub-bursts, so that the proportion of each sub-sub-burst size to the MRB of a single timeslot does not exceed the specific value, and each sub-sub-burst corresponds to a reservation sub- task. 2.3. Proccess of Each Reservation Sub-task Each reservation sub-task contains a separate parameter set, which is used in the process of timeslot resource reservation. Note that this set may be a local information for the path compuation engine (e.g, a controller), or may signal between nodes (e.g, RSVP-TE). * Total Residence Budget: It is the sum of the residence delay allowed by the DetNet flow within all nodes in the path, which is equal to the end-to-end delay requirement of the DetNet flow minus the propagation delay of all links included in the path. * Node Residence Budget: It refers to the residence delay budget of the current node traversed during the process of reserving timeslot resources on each node along the path in sequence. A simple way is to divide the Total Residence Budget by the number of nodes included in the path to obtain the average residence delay budget as the Node Residence Budget for each node, or use a specified budget list to specify the residence delay budget for each node separately. * Accumulated Node Residence Budget: It refers to the accumulated residence delay budget of those nodes that have executed resource reservation. * Accumulated Node Residence Evaluation: It refers to the accumulated evaluation value of the residence delay of nodes that have executed resource reservation. The residence delay evaluation value of a node refers to the residence delay evaluation value calculated based on the delay formula (see below) when the node actually reserves a certain outgoing timeslot for the reservation sub-task. Generally, if a node is able to reserve the expected outgoing timeslot according to its residence delay budget, the residence delay evaluation value does not differ from the residence delay budget. However, in some cases, due to insufficient resources in the expected timeslot, resources have to be reserved in the timeslot adjacent to the expected timeslot, which can lead to a difference between the residence delay evaluation value and the budget value. Peng, et al. Expires 22 December 2024 [Page 9] Internet-Draft TQF Control Plane June 2024 * Accumulated Node Residence Deviation: It is equal to the Accumulated Node Residence Budget minus the Accumulated Node Residence Evaluation. * Node Residence Budget Adjustment: It is equal to the Node Residence Budget plus the Accumulated Node Residence Deviation. The usage for the above parameter set is: * For specific reservation sub-task, determine the Node Residence Budget for each node in the path, which can be taken from the average residence delay budget per node or the specified budget list. * From the headend to the endpoint, on each node's outgoing port in sequence, reserve outgoing timeslot resources based on the Node Residence Budget Adjustment, to let the residence delay evaluation value of the node obtained from the reserved outgoing timeslot be equal to or close to the Node Residence Budget Adjustment. - On the headend, the Accumulated Node Residence Deviation is the initial value of 0. Therefore, the Node Residence Budget Adjustment is equal to the Node Residence Budget. - On any other nodes, the Accumulated Node Residence Deviation is generally not 0. If the residence delay evaluation value of the node obtained from the reserved outgoing timeslot be equal to the Node Residence Budget Adjustment, it will cause the Accumulated Node Residence Deviation faced by the downstream node in the path to be 0 again. Note that the above parameter set is only an implementation choice and is not mandatory. There may be more intelligent path calculation methods available. 2.3.1. Resource Reservation on the Ingress Node On the headend H, as mentioned above, each sub-burst corresponds to an ideal incoming timeslot i of UNI port. After the intra-node forwarding delay (F), the end of the incoming timeslot i reaches the outgoing port, the timeslot currently in the sending state (i.e., the ongoing sending timeslot of NNI port) is j, and there is time T_ij left before the end of the timeslot j. The outgoing timeslot reserved for the sub-burst by the headend is offset by o (>=1) timeslots after timeslot j, which means the outgoing timeslot is z = (j+o)%N_nni, where N_nni is the number of timeslots in the orchestration period of NNI port. Peng, et al. Expires 22 December 2024 [Page 10] Internet-Draft TQF Control Plane June 2024 Note that o must be less than M. (where o is the offset and M is the number of timeslot in the scheduling period as mentioned in Section 2.1) Thus, on the headend H the residence delay evaluation value obtained from the reserved outgoing timeslot z is: Best Node Residence Evaluation = F + T_ij + (o-1)*L_nni Worst Node Residence Evaluation = F + L_uni + T_ij + o*L_nni Average Node Residence Evaluation = F + T_ij + (L_uni + (2o- 1)*L_nni)/2 where, L_uni is the timeslot length of UNI port, L_nni is the timeslot length of NNI port. The Best Node Residence Evaluation occurs when the sub-burst is at the end of the ideal incoming timeslot i, and sent at the head of outgoing timeslot z. The Worst Node Residence Evaluation occurs when the sub-burst is at the head of the ideal incoming timeslot i, and sent at the end of outgoing timeslot z. The delay jitter within the headend is (L_uni + L_nni). However, the jitter of the entire path is not the sum of the jitters of all nodes. Depending on the implementation, the above Best Node Residence Evaluation, Worst Node Residence Evaluation, or Average Node Residence Evaluation can be used to compare with the Node Residence Budget Adjustment, so that when selecting the appropriate outgoing timeslot z, the two are equal or nearly equal, and the corresponding Unreserved Burst resources of the outgoing timeslot z meet the burst demand of the sub-burst. However, this document suggests using the Average Node Residence Evaluation to compare with the Node Residence Budget Adjustment, because the characteristic of the forwarding behavior based on TQF is that adjacent nodes on the path will not simultaneously face the best or worst residency delay. Note that there is a runtime jitter (i.e., the resource reservation process on the control plane is not aware of it), as mentioned earlier, which depends on the deviation between the actual incoming timeslot i' and the ideal incoming timeslot i. Assuming that i = (i'+e)%N_uni, where e is the deviation, N_uni is the number of timeslots in the orchestration period of UNI port, then the additional runtime jitter is e*L_uni, that should be carried in the packet to eliminate jitter at the network egress. Peng, et al. Expires 22 December 2024 [Page 11] Internet-Draft TQF Control Plane June 2024 2.3.2. Resource Reservation on the Transit Node On the transit node V, there is a timeslot mapping relationship between the incoming timeslot i and the ongoing sending timeslot j of outgoing port, and there is time T_ij left before the end of the timeslot j. For a specific sub-task with incoming timeslot i, assuming the outgoing timeslot z is reserved for it, by o (>=1) timeslots after timeslot j, i.e., z = (j+o)%N_out, where N_out is the number of timeslots in the orchestration period of port_out. Note that o must be less than M. Thus, on the transit node V the residence delay evaluation value obtained from the reserved outgoing timeslot z is: Best Node Residence Evaluation = F + T_ij + (o-1)*L_out Worst Node Residence Evaluation = F + T_ij + L_in + o*L_out Average Node Residence Evaluation = F + T_ij + (L_in+(2o- 1)*L_out)/2 where, L_in and L_out is the length of incoming timeslot and outgoing timeslot respectively. The Best Node Residence Evaluation occurs when the packet is received at the end of incoming timeslot i and sent at the head of outgoing timeslot z; The Worst Node Residence Evaluation occurs when the packet is received at the head of incoming timeslot i and sent at the end of outgoing timeslot z. The delay jitter within the node is (L_in + L_out). However, the jitter of the entire path is not the sum of the jitters of all nodes. Depending on the implementation, the above Best Node Residence Evaluation, Worst Node Residence Evaluation, or Average Node Residence Evaluation can be used to compare with the Node Residence Budget Adjustment, so that when selecting the appropriate outgoing timeslot z, the two are equal or nearly equal, and the corresponding Unreserved Burst resources of the outgoing timeslot z meet the burst demand of the sub-burst. However, this document suggests using the Average Node Residence Evaluation to compare with the Node Residence Budget Adjustment, because the characteristic of the forwarding behavior based on TQF is that adjacent nodes on the path will not simultaneously face the best or worst residency delay. Peng, et al. Expires 22 December 2024 [Page 12] Internet-Draft TQF Control Plane June 2024 2.3.3. Resource Reservation on the Egress Node Generally, for the deterministic path carrying the DetNet flow, the flow needs to continue forwarding from the outgoing port of the egress node to the client side, and also faces the issues of queueing. However, the outgoing port facing the client side is not part of the deterministic path. If it is necessary to continue supporting TQF mechanism on that port, timeslot resources should be reserved on the higher-level DetNet path (an overlay path) using the above reservation method. In this case, the underlay DetNet path will serve as a virtual link of the overlay path, providing a deterministic delay performance. Therefore, for deterministic paths, the residence dalay evaluation value on the egress node is only contributed by the forwarding delay (F) including parsing, table lookup, internal fabric exchange, etc. 3. Multiple Orchestration Periods Multiple orchestration periods each with different length may be provided by the link. Interworking between different nodes is based on the same orchestration period. That means that the timeslot resource reservation along the path for a sub-task should be in the context of the specific orchestration period. 4. Flow Aggregation and De-aggregation Multiple DetNet flows may share the same timeslot resources on each link included in a certain path segment in the network. These flows are simply encapsulated with the same timeslot id during forwarding. TQF allows for different lengths of incoming and outgoing timeslots, which is useful for natural flow aggregation caused by the network topology (such as access, aggregation and backbone domains). For example, for the direction from the access domain to the aggregation domain, on the aggregation point, the incoming timeslot length may be larger than the outgoing timeslot length, and multiple incoming timeslots will be mapped to the same outgoing timeslot. Generally, the capacity of a short outgoing timeslot is still larger than that of a long incoming timeslot, otherwise, the aggregated incoming timeslots will be mapped to several consecutive outgoing timeslots. Switching from a long incoming timeslot to a short outgoing timeslot will accelerate packet forwarding. Peng, et al. Expires 22 December 2024 [Page 13] Internet-Draft TQF Control Plane June 2024 In the opposite direction, on the de-aggregation point, a single incoming timeslot may be mapped to multiple outgoing timeslots each for a specific outgoing port, if the capacity of the incoming timeslot is larger than that of the outgoing timeslot. 5. Provision Flow Identification Information TQF use outgoing timeslot id on each node to provide PHB treatment for flows in the network. A DetNet flow will obtain the outgoing timeslot id stack once the related path is calculated and setup. Essentially, the use of timeslot id is a function of the forwarding sub-layer An additional flow identification is still necessary for a DetNet flow that is sent on the TQF path, for the purpose of service sub- layer functions such as PREOF (Packet Replication, Elimination and Ordering Functions). The flow identification should also be determined for the flow once the related path is calculated and setup. This provision is a common operation and not unique to TQF. 6. IANA Considerations TBD. 7. Security Considerations TBD. 8. Acknowledgements TBD. 9. References 9.1. Normative References [I-D.peng-detnet-packet-timeslot-mechanism] Peng, S., Liu, P., Basu, K., Liu, A., Yang, D., and G. Peng, "Timeslot Queueing and Forwarding Mechanism", Work in Progress, Internet-Draft, draft-peng-detnet-packet- timeslot-mechanism-06, 4 March 2024, . Peng, et al. Expires 22 December 2024 [Page 14] Internet-Draft TQF Control Plane June 2024 [I-D.peng-lsr-deterministic-traffic-engineering] Peng, S., "IGP Extensions for Deterministic Traffic Engineering", Work in Progress, Internet-Draft, draft- peng-lsr-deterministic-traffic-engineering-01, 4 July 2023, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC 3209, DOI 10.17487/RFC3209, December 2001, . [RFC4655] Farrel, A., Vasseur, J.-P., and J. Ash, "A Path Computation Element (PCE)-Based Architecture", RFC 4655, DOI 10.17487/RFC4655, August 2006, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, "Deterministic Networking Architecture", RFC 8655, DOI 10.17487/RFC8655, October 2019, . 9.2. Informative References [TAS] "Time-Aware Shaper", 2015, . Authors' Addresses Shaofu Peng ZTE China Email: peng.shaofu@zte.com.cn Peng Liu China Mobile China Email: liupengyjy@chinamobile.com Peng, et al. Expires 22 December 2024 [Page 15] Internet-Draft TQF Control Plane June 2024 Kashinath Basu Oxford Brookes University United Kingdom Email: kbasu@brookes.ac.uk Aihua Liu ZTE China Email: liu.aihua@zte.com.cn Dong Yang Beijing Jiaotong University China Email: dyang@bjtu.edu.cn Guoyu Peng Beijing University of Posts and Telecommunications China Email: guoyupeng@bupt.edu.cn Peng, et al. Expires 22 December 2024 [Page 16]