3.1. Overview of GMPLS Control Plane

GMPLS separates the control plane and the data plane to support time-division, wavelength, and spatial switching, which are significant in transport networks. For the NE level control in GMPLS, each node runs a GMPLS control plane instance. Functionalities such as service provisioning, protection, and restoration can be performed via GMPLS communication among multiple NEs. At the same time, the GMPLS control plane instance can also collect information about node and link resources in the network to construct the network topology and compute routing paths for serving service requests.¶

Several protocols have been designed for the GMPLS control plane [RFC3945], including link management [RFC4204], signaling [RFC3471], and routing [RFC4202] protocols. The GMPLS control plane instances applying these protocols communicate with each other to exchange resource information and establish LSPs. In this way, GMPLS control plane instances in different nodes in the network have the same view of the network topology and provision services based on local policies.¶

3.2. Overview of Centralized Controller System

With the development of SDN technologies, a centralized controller architecture has been introduced to transport networks. One example architecture can be found in ACTN [RFC8453]. In such systems, a controller is aware of the network topology and is responsible for provisioning incoming service requests.¶

Multiple hierarchies of controllers are designed at different levels to implement different functions. This kind of architecture enables multi-vendor, multi-domain, and multi-technology control. For example, a higher-level controller coordinates several lower-level controllers controlling different domains for topology collection and service provisioning. Vendor-specific features can be abstracted between controllers, and a standard API (e.g., generated from RESTCONF [RFC8040] / YANG [RFC7950]) may be used.¶

3.3. GMPLS Control Interworking with a Centralized Controller System

Besides GMPLS and the interactions among the controller hierarchies, it is also necessary for the controllers to communicate with the network elements. Within each domain, GMPLS control can be applied to each NE. The bottom-level centralized controller can act as an NE to collect network information and initiate LSPs. Figure 1 shows an example of GMPLS interworking with centralized controllers (ACTN terminologies are used in the figure).¶

                        +-------------------+
                        |    Orchestrator   |
                        |       (MDSC)      |
                        +-------------------+
                          ^       ^       ^
                          |       |       |
            +-------------+       |       +--------------+
            |                     |RESTCONF/YANG modules |
            V                     V                      V
      +-------------+      +-------------+       +-------------+
      |Controller(N)|      |Controller(G)|       |Controller(G)|
      |    (PNC)    |      |    (PNC)    |       |    (PNC)    |
      +-------------+      +-------------+       +-------------+
           ^  ^                  ^  ^                  ^  ^
           |  |                  |  |                  |  |
    NETCONF|  |PCEP       NETCONF|  |PCEP       NETCONF|  |PCEP
     /YANG |  |            /YANG |  |            /YANG |  |
           V  V                  V  V                  V  V
       .----------.  Inter-  .----------.  Inter-  .----------.
      /            \ domain /            \ domain /            \
     |              | link |     LMP      | link |     LMP      |
     |              |======|   OSPF-TE    |======|   OSPF-TE    |
     |              |      |   RSVP-TE    |      |   RSVP-TE    |
      \            /        \            /        \            /
       `----------`          `----------`          `----------`
    Non-GMPLS domain 1      GMPLS domain 2        GMPLS domain 3

Controller(N):

A domain controller controlling a non-GMPLS domain¶

Controller(G):

A domain controller controlling a GMPLS domain¶

Figure 1 shows the scenario with two GMPLS domains and one non-GMPLS domain. This system supports the interworking among non-GMPLS domains, GMPLS domains, and the controller hierarchies.¶

For domain 1, the network elements were not enabled with GMPLS, so the control is purely from the controller, via Network Configuration Protocol (NETCONF) [RFC6241] / YANG and/or PCE Communication Protocol (PCEP) [RFC5440].¶

For domains 2 and 3:¶

• Each domain has the GMPLS control plane enabled at the physical network level. The Provisioning Network Controller (PNC) can exploit GMPLS capabilities implemented in the domain to listen to the IGP routing protocol messages (for example, OSPF Link State Advertisements (LSAs)) that the GMPLS control plane instances are disseminating into the network and thus learn the network topology. For path computation in the domain with the PNC implementing a PCE, Path Computation Clients (PCCs) (e.g., NEs, other controllers/PCEs) use PCEP to ask the PNC for a path and get replies. The Multi-Domain Service Coordinator (MDSC) communicates with PNCs using, for example, Representational State Transfer (REST) / RESTCONF based on YANG data models. As a PNC has learned its domain topology, it can report the topology to the MDSC. When a service arrives, the MDSC computes the path and coordinates PNCs to establish the corresponding LSP segment.¶
• Alternatively, the NETCONF protocol can be used to retrieve topology information utilizing the YANG module in [RFC8795] and the technology-specific YANG module augmentations required for the specific network technology. The PNC can retrieve topology information from any NE (the GMPLS control plane instance of each NE in the domain has the same topological view), construct the topology of the domain, and export an abstract view to the MDSC. Based on the topology retrieved from multiple PNCs, the MDSC can create a topology graph of the multi-domain network and can use it for path computation. To set up a service, the MDSC can exploit the YANG module in [YANG-TE] together with the technology-specific YANG module augmentations.¶

This document focuses on the interworking between GMPLS and the centralized controller system, including:¶

• the interworking between the GMPLS domains and the centralized controllers (including the orchestrator, if it exists) controlling the GMPLS domains and¶
• the interworking between a non-GMPLS domain (which is controlled by a centralized controller system) and a GMPLS domain, through the controller hierarchy architecture.¶

For convenience, this document uses the following terminologies for the controller and the orchestrator:¶

Controller(G):

A domain controller controlling a GMPLS domain (the Controller(G) of the GMPLS domains 2 and 3 in Figure 1)¶

Controller(N):

A domain controller controlling a non-GMPLS domain (the Controller(N) of the non-GMPLS domain 1 in Figure 1)¶

H-Controller(G):

A domain controller controlling the higher-layer GMPLS domain, in the context of multi-layer networks¶

L-Controller(G):

A domain controller controlling the lower-layer GMPLS domain, in the context of multi-layer networks¶

H-Controller(N):

A domain controller controlling the higher-layer non-GMPLS domain, in the context of multi-layer networks¶

L-Controller(N):

A domain controller controlling the lower-layer non-GMPLS domain, in the context of multi-layer networks¶

Orchestrator(MD):

An orchestrator used to orchestrate the multi-domain networks¶

Orchestrator(ML):

An orchestrator used to orchestrate the multi-layer networks¶

4.1. LMP

The Link Management Protocol (LMP) [RFC4204] runs between nodes and manages TE links. In addition to the setup and maintenance of control channels, LMP can be used to verify the data link connectivity and correlate the link properties.¶

5.1. OSPF-TE

OSPF-TE is introduced for TE networks in [RFC3630]. OSPF extensions have been defined in [RFC4203] to enable the capability of link state information for the GMPLS network. Based on this work, OSPF has been extended to support technology-specific routing. The routing protocols for the Optical Transport Network (OTN), Wavelength Switched Optical Network (WSON), and optical flexi-grid networks are defined in [RFC7138], [RFC7688], and [RFC8363], respectively.¶

5.2. IS-IS-TE

IS-IS-TE is introduced for TE networks in [RFC5305], is extended to support GMPLS routing functions [RFC5307], and has been updated [RFC7074] to support the latest GMPLS switching capability and Types fields.¶

5.3. NETCONF/RESTCONF

NETCONF [RFC6241] and RESTCONF [RFC8040] protocols are originally used for network configuration. These protocols can also utilize topology-related YANG modules, such as those in [RFC8345] and [RFC8795]. These protocols provide a powerful mechanism for the notification (in addition to the provisioning and monitoring) of topology changes to the client.¶

6.1. Controller-Based Path Computation

Once a controller learns the network topology, it can utilize the available resources to serve service requests by performing path computation. Due to abstraction, the controllers may not have sufficient information to compute the optimal path. In this case, the controller can interact with other controllers by sending, for example, YANG-based path computation requests [PATH-COMP] or PCEP to compute a set of potential optimal paths; and then, based on its constraints, policy, and specific knowledge (e.g., cost of access link), the controller can choose the more feasible path for end-to-end (E2E) service path setup.¶

Path computation is one of the key objectives in various types of controllers. In the given architecture, it is possible for different components that have the capability to compute the path.¶

6.2. Constraint-Based Path Computing in GMPLS Control

In GMPLS control, a routing path may be computed by the ingress node [RFC3473] based on the ingress node Traffic Engineering Database (TED). In this case, constraint-based path computation is performed according to the local policy of the ingress node.¶

6.3. Path Computation Element (PCE)

The PCE was first introduced in [RFC4655] as a functional component that offers services for computing paths within a network. In [RFC5440], path computation is achieved using the TED, which maintains a view of the link resources in the network. The introduction of the PCE has significantly improved the quality of network planning and offline computation. However, there is a potential risk that the computed path may be infeasible when there is a diversity requirement, as a stateless PCE lacks knowledge about previously computed paths.¶

To address this issue, a stateful PCE has been proposed in [RFC8231]. Besides the TED, an additional LSP Database (LSP-DB) is introduced to archive each LSP computed by the PCE. This way, the PCE can easily determine the relationship between the computing path and former computed paths. In this approach, the PCE provides computed paths to the PCC, and then the PCC decides which path is deployed and when it is to be established.¶

With PCE-initiated LSPs [RFC8281], the PCE can trigger the PCC to perform setup, maintenance, and teardown of the PCE-initiated LSP under the stateful PCE model. This would allow a dynamic network that is centrally controlled and deployed.¶

In a centralized controller system, the PCE can be implemented within the centralized controller. The centralized controller then calculates paths based on its local policies. Alternatively, the PCE can be located outside of the centralized controller. In this scenario, the centralized controller functions as a PCC and sends a path computation request to the PCE using the PCEP. A reference architecture for this can be found in [RFC7491].¶

7.1. RSVP-TE

RSVP-TE is introduced in [RFC3209] and extended to support GMPLS signaling in [RFC3473]. Several label formats are defined for a generalized label request, a generalized label, a suggested label, and label sets. Based on [RFC3473], RSVP-TE has been extended to support technology-specific signaling. The RSVP-TE extensions for the OTN, WSON, and optical flexi-grid network are defined in [RFC7139], [RFC7689], and [RFC7792], respectively.¶

8.1. Topology Collection and Synchronization

Topology information is necessary on both network elements and controllers. The topology on a network element is usually raw information, while the topology used by the controller can be either raw, reduced, or abstracted. Three different abstraction methods have been described in [RFC8453], and different controllers can select the corresponding method depending on the application.¶

When there are changes in the network topology, the impacted network elements need to report changes to all the other network elements, together with the controller, to sync up the topology information. The inter-NE synchronization can be achieved via protocols mentioned in Sections 4 and 5. The topology synchronization between NEs and controllers can either be achieved by routing protocols OSPF-TE/PCEP-LS in [PCEP-LS] or NETCONF protocol notifications with a YANG module.¶

8.2. Multi-Domain Service Provisioning

Service provisioning can be deployed based on the topology information on controllers and network elements. Many methods have been specified for single-domain service provisioning, such as the PCEP and RSVP-TE methods.¶

Multi-domain service provisioning would require coordination among the controller hierarchies. Given the service request, the end-to-end delivery procedure may include interactions at any level (i.e., interface) in the hierarchy of the controllers (e.g., MPI and SBI for ACTN). The computation for a cross-domain path is usually completed by controllers who have a global view of the topologies. Then the configuration is decomposed into lower-level controllers to configure the network elements to set up the path.¶

A combination of centralized and distributed protocols may be necessary to interact between network elements and controllers. Several methods can be used to create the inter-domain path:¶

1)

With an end-to-end RSVP-TE session:¶

In this method, all the domains need to support the RSVP-TE protocol and thus need to be GMPLS domains. The Controller(G) of the source domain triggers the source node to create the end-to-end RSVP-TE session; and the assignment and distribution of the labels on the inter-domain links are done by the border nodes of each domain, using RSVP-TE protocol. Therefore, this method requires the interworking of RSVP-TE protocols between different domains.¶

There are two possible methods:¶

1.1)

One single end-to-end RSVP-TE session:¶

In this method, an end-to-end RSVP-TE session from the source node to the destination node will be used to create the inter-domain path. A typical example would be the PCE initiation scenario, in which a PCE message (PCInitiate) is sent from the Controller(G) to the source node, triggering an RSVP procedure along the path. Similarly, the interaction between the controller and the source node of the source domain can be achieved by using the NETCONF protocol with corresponding YANG modules, and then it can be completed by running RSVP among the network elements.¶

1.2)

LSP Stitching:¶

The LSP stitching method defined in [RFC5150] can also create the E2E LSP. That is, when the source node receives an end-to-end path creation request (e.g., using PCEP or NETCONF protocol), the source node starts an end-to-end RSVP-TE session along the endpoints of each LSP segment (S-LSP) (refers to S-LSP in [RFC5150]) of each domain, to assign the labels on the inter-domain links between each pair of neighbor S-LSPs and to stitch the end-to-end LSP to each S-LSP. See Figure 2 as an example.¶

Note that the S-LSP in each domain can be either created by its Controller(G) in advance or created dynamically triggered by the end-to-end RSVP-TE session.¶

                  +------------------------+
                  |    Orchestrator(MD)    |
                  +-----------+------------+
                              |
 +---------------+     +------V-------+     +---------------+
 | Controller(G) |     | Controller(G)|     | Controller(G) |
 +-------+-------+     +------+-------+     +-------+-------+
         |                    |                     |
+--------V--------+   +-------V--------+   +--------V--------+
|Client           |   |                |   |           Client|
|Signal   Domain 1|   |    Domain 2    |   |Domain 3   Signal|
|  |              |   |                |   |              |  |
|+-+-+            |   |                |   |            +-+-+|
|| | |  +--+  +--+|   |+--+  +--+  +--+|   |+--+  +--+  | | ||
|| | |  |  |  |  ||   ||  |  |  |  |  ||   ||  |  |  |  | | ||
|| ******************************************************** ||
||   |  |  |  |  ||   ||  |  |  |  |  ||   ||  |  |  |  |   ||
|+---+  +--+  +--+|   |+--+  +--+  +--+|   |+--+  +--+  +---+|
+-----------------+   +----------------+   +-----------------+
 |   .           .     .              .     .           .   |
 |   .<-S-LSP 1->.     .<- S-LSP 2 -->.     .<-S-LSP 3->.   |
 |               .     .              .     .               |
 |-------------->.---->.------------->.---->.-------------->|
 |<--------------.<----.<-------------.<----.<--------------|
 |       End-to-end RSVP-TE session for LSP stitching       |

2)

Without an end-to-end RSVP-TE session:¶

In this method, each domain can be a GMPLS domain or a non-GMPLS domain. Each controller (which may be a Controller(G) or a Controller(N)) is responsible for creating the path segment within its domain. The border node does not need to communicate with other border nodes in other domains for the distribution of labels on inter-domain links, so an end-to-end RSVP-TE session through multiple domains is not required, and the interworking of the RSVP-TE protocol between different domains is not needed.¶

Note that path segments in the source domain and the destination domain are "asymmetrical" segments, because the configuration of client signal mapping into the server-layer tunnel is needed at only one end of the segment, while configuration of the server-layer cross-connect is needed at the other end of the segment. See the example in Figure 3.¶

                  +------------------------+
                  |    Orchestrator(MD)    |
                  +-----------+------------+
                              |
 +---------------+     +------V-------+     +---------------+
 |  Controller   |     |  Controller  |     |  Controller   |
 +-------+-------+     +------+-------+     +-------+-------+
         |                    |                     |
+--------V--------+   +-------V--------+   +--------V--------+
|Client   Domain 1|   |    Domain 2    |   | Domain 3  Client|
|Signal           |   |                |   |           Signal|
|  |  Server layer|   |                |   |              |  |
|  |     tunnel   |   |                |   |              |  |
|+-+-+       ^    |   |                |   |            +-+-+|
|| | |  +--+ |+--+|   |+--+  +--+  +--+|   |+--+  +--+  | | ||
|| | |  |  | ||  ||   ||  |  |  |  |  ||   ||  |  |  |  | | ||
|| ******************************************************** ||
||   |  |  |  |  || . ||  |  |  |  |  || . ||  |  |  |  |   ||
|+---+  +--+  +--+| . |+--+  +--+  +--+| . |+--+  +--+  +---+|
+-----------------+ . +----------------+ . +-----------------+
 .                  .                    .                  .
 .<-Path Segment 1->.<--Path Segment 2-->.<-Path Segment 3->.

The PCEP / GMPLS protocols should support the creation of such asymmetrical segments.¶

Note also that mechanisms to assign the labels in the inter-domain links also need to be considered. There are two possible methods:¶

2.1)

Inter-domain labels assigned by NEs:¶

The concept of a stitching label that allows stitching local path segments was introduced in [RFC5150] and [SPCE-ID], in order to form the inter-domain path crossing several different domains. It also describes the Backward Recursive PCE-Based Computation (BRPC) [RFC5441] and Hierarchical PCE (H-PCE) [RFC8685] PCInitiate procedure, i.e., the ingress node of each downstream domain assigns the stitching label for the inter-domain link between the downstream domain and its upstream neighbor domain; and this stitching label will be passed to the upstream neighbor domain by PCE protocol, which will be used for the path segment creation in the upstream neighbor domain.¶

2.2)

Inter-domain labels assigned by the controller:¶

If the resources of inter-domain links are managed by the Orchestrator(MD), each domain controller can provide to the Orchestrator(MD) the list of available labels (e.g., time slots, if the OTN is the scenario) using the IETF Topology YANG module and a related technology-specific extension. Once the Orchestrator(MD) has computed the E2E path, RSVP-TE or PCEP can be used in the different domains to set up the related segment tunnel consisting of label inter-domain information; for example, for PCEP, the label Explicit Route Object (ERO) can be included in the PCInitiate message to indicate the inter-domain labels so that each border node of each domain can configure the correct cross-connect within itself.¶

8.3. Multi-Layer Service Provisioning

GMPLS can interwork with centralized controller systems in multi-layer networks.¶

+----------------+
|Orchestrator(ML)|
+------+--+------+
       |  |                            Higher-layer Network
       |  |                           .--------------------.
       |  |                          /                      \
       |  |   +--------------+      |   +--+   Link   +--+   |
       |  +-->| H-Controller +----->|   |  |**********|  |   |
       |      +--------------+      |   +--+          +--+   |
       |                             \    .            .    /
       |                               `--.------------.---`
       |                                  .            .
       |                              .---.------------.---.
       |                             /    .            .    \
       |      +--------------+      |   +--+   +--+   +--+   |
       +----->| L-Controller +----->|   | ============== |   |
              +--------------+      |   +--+   +--+   +--+   |
                                     \         H-LSP        /
                                       `-------------------`
                                        Lower-layer Network

An example with two layers of network is shown in Figure 4. In this example, the GMPLS control plane is enabled in at least one layer network (otherwise, it is out of the scope of this document) and interworks with the controller of its domain (H-Controller and L-Controller, respectively). The Orchestrator(ML) is used to coordinate the control of the multi-layer network.¶

8.4. Recovery

The GMPLS recovery functions are described in [RFC4426]. Span protection and end-to-end protection and restoration are discussed with different protection schemes and message exchange requirements. Related RSVP-TE extensions to support end-to-end recovery are described in [RFC4872]. The extensions in [RFC4872] include protection, restoration, preemption, and rerouting mechanisms for an end-to-end LSP. Besides end-to-end recovery, a GMPLS segment recovery mechanism is defined in [RFC4873], which also intends to be compatible with Fast Reroute (FRR) (see [RFC4090], which defines RSVP-TE extensions for the FRR mechanism, and [RFC8271], which describes the updates of the GMPLS RSVP-TE protocol for FRR of GMPLS TE-LSPs).¶

8.5. Controller Reliability

The reliability of the controller is crucial due to its important role in the network. It is essential that if the controller is shut down or disconnected from the network, all currently provisioned services in the network continue to function and carry traffic. In addition, protection switching to pre-established paths should also work. It is desirable to have protection mechanisms, such as redundancy, to maintain full operational control even if one instance of the controller fails. This can be achieved through controller backup or functionality backup. There are several controller backup or federation mechanisms in the literature. It is also more reliable to have function backup in the network element to guarantee performance in the network.¶