By Ken Fridley
Utilities depend on reliable, secure and fast networks to monitor, analyze, control and protect their systems. As the density of intelligent sensors increase, so does the complexity and scale of communications networks required to support their latency and bandwidth requirements. To address these demands, utilities are migrating to Ethernet based networks using virtual local area network (VLAN) or Multiprotocol Label Switching (MPLS) protocols to improve efficiency and reduce costs. Millions of time-division multiplexing (TDM) based circuits, however, persist on the edges of the network and are destined to stay deployed for the foreseeable future. The old adage, “if it’s not broken, don’t fix it,” applies in some cases. Because of this hybrid mix of communication technologies, the need for multifunctional test solutions that are designed for novice, yet tech savvy, technicians is recommended.
Ethernet Migration Challenges
Because of lower cost accompanied by higher available bandwidth compared to TDM and Synchronous Optical NETwork (SONET) while maintaining reliability of legacy carrier circuits, Ethernet is the preferred protocol for new network deployments. When provisioning new metro and core circuits, most operators agree it is generally acceptable to perform out-of-service testing using RFC-2544 or Bit Error Rate (BERT) tests methods. With these methods the entire data pipe is filled to 100 percent capacity with test traffic for a certain period of time, normally 15 minutes to 24 hours, and a performance report is created. If no errors are identified, the circuit is then placed into service.
Challenges such as accurate network timing, ability to bypass bandwidth limits or security policy gaps can occur at the substation level if improper configuration of network elements occurs. These problems can be difficult to detect with BERT or RFC-2544 because gigabit interfaces are common on most switches and routers but substations rarely require the full bandwidth available. This can be a security risk because network attackers can flood a network with data from a single location. To counter this risk, traffic policing on the switch or router is configured, which “clamps” total traffic allowed onto the backhaul network, preventing flooding. As additional sensors are installed, however, total bandwidth increases through the network. If traffic policies are not adjusted, data will be dropped, which results in lost frames or sensor alarms.
Quality of Service Management
A second challenge in migration to Ethernet networks is maintaining acceptable latency on point-to-point links carrying supervisory or VoIP traffic. Latency in excess of 10us may cause false failover alarms to be reported on high speed protection links whereas latency in excess of 125us will impact voice quality. Quality of service (QoS) or VLAN tagging must be also be properly configured to ensure high priority traffic is given precedence over best effort traffic when commingled on network backhaul links. If misconfigured, congested networks may give preferential treatment to low priority traffic while dropping high value sensor data. If VLANs are improperly assigned, the same issue may occur.
Detailed Service Testing
If the goal is to identify advanced policy issues, testing beyond what simple bit error rate or RFC-2544 benchmarking tests provide is required. It is recommended that operators use the ITU-T Y.1564 service assurance test methodology. Y.1564 emulates multiple traffic flows, each with varying bandwidth, latency and QoS requirements—then verifies each service is receiving its defined service level. User defined service flow profiles such as high priority sensor traffic, normal priority voice/video, and low priority data are defined in the test system prior to initiation. When the test is started, each service is verified individually for proper configuration to ensure bandwidth, VLAN, MPLS and QoS settings are correct. Services not passing the policy configuration test can quickly be isolated and policies updated on the switch or router. Once configuration is verified, all services are transmitted simultaneously to emulate live traffic on the network. This verifies traffic flows are able to traverse the network as designed. Other benefits of using Y.1564 vs. RFC-2544 are multiple destinations can be selected for each service and testing is often completed in a little as 15 minutes. Y.1564 can be performed while the circuit is in service.
TDM Migration Challenges
One of the primary drivers for migration away from legacy TDM services is product obsolescence. The cost of maintaining their network and offering wireless private network alternatives to their customers have driven telecom service providers to discontinue TDM services. Utilities are also facing increasing pressure to discontinue the use of TDM circuits as aging remote terminal units (RTUs) and substation Plesiosynchronous Digital Hierarchy (PDH) / Synchronous Digital Hierarchy (SDH) / SONET multiplexers reach end of life.
Millions of TDM based data circuits, however, persist on utility networks and are destined to stay deployed into the foreseeable future. As older RTUs become obsolete or TDM services are terminated, some utilities have chosen to packetize existing 2-wire FXS, 4-wire E&M and low speed RS-232 protection links using Ethernet pseudo-wires in place of performing a full forklift upgrade. The result is hybrid systems that mostly function well together but can be problematic to diagnose when the network is modified without understanding the ramifications of doing so.
In preparing to transition onto Ethernet networks, it is often assumed modern digital sensors are being connected to the network and input data is being directly collected in digital form that is transported in a well behaved fashion. When legacy 2-wire or 4-wire analog circuits are connected using media converters, analog to digital conversion using pulse coded modulation (PCM) is first required. Analog signals are sampled and converted to digital format—but so is extraneous Gaussian noise. If noise exceeds the maximum sample input level supported by the converter, the receiver may become saturated or, worse yet, the signal may be companded by the codec and transmitted as distortion. The far side circuit may interpret this as a drop in signal or existence of an invalid signal. In this case, how is a technician to know where the trouble is actually occurring? Is it on the analog input, the PCM translation, or the transport network itself? The solution is to place a transmission (TIMS) test set on either end of the line that can detect dropouts—gains and phase hits will help determine where the issue is occurring.
Workforce Migration Challenges
As network complexity and capacity increases, the need for highly skilled technicians who understand both legacy and packet based networks will be essential to maintaining operations. The issue is that 60 percent of the workforce will soon be eligible for retirement. As this workforce retires, knowledge regarding hybrid or legacy systems will retire along with them. This means aging legacy systems requiring maintenance will be performed by new technicians with little exposure in how to maintain them. These new technicians, however, are tech savvy and comfortable using tools that leverage the tablets and smart phones they use on a daily basis. By providing access to visual or audible training on the test device when they encounter an unfamiliar task, learning curves are reduced.
When selecting test tools, it is important to specify solutions that support Ethernet, legacy transport and analog transmission test interfaces commonly found in the network. They should be designed with the technician in mind who requires multi-functionality in their kit and should provide a familiar user experience that leverages technology technicians are comfortable with. By following these recommendations, utilities will be ready to assure yesterday, today and tomorrow’s networks.
About the author:
Ken Fridley, Ethernet/Transport Product Manager for Greenlee Communications, has 25 years of experience in the telecommunications industry working in line, management and technical support roles. He has presented extensively on communications topics at trade shows around the world and has authored several articles for various trade magazines. For more information, please visit http://www.greenleecommunications.com or email firstname.lastname@example.org.