Why Does My System Fail When Each Component Works?

by Regular Contributor on ‎10-02-2009 12:25 PM - last edited on ‎10-14-2009 09:11 AM by Trusted Contributor

Chad Dozier, Caterpillar Electric Power Division

A few years ago I was involved with a project for a new hospital. During the initial commissioning, testing showed that the ten second power restoration (NFPA-110 regulation) was not being met. The bottom line was this: separately the components met their requirement, but when combined, the system failed. Let me walk you through it.

Site Layout
High level, the site consisted of:
  • Two 12,470V utility feeds with switchgear
  • Generator paralleling switchgear
  • Two 2250 kW, 480V generator sets
  • Transfer switches to transfer power between utility switchgear and paralleling switchgear

The sequence of operations was:
  • Transfer switch detects loss of utility, and after a one second delay, it signals the generator paralleling switchgear to start.
  • The generator paralleling switchgear then starts the generator sets.
  • As the first generator set reaches rated speed and voltage, the generator circuit breaker (located in the paralleling switchgear) is closed.
  • The transfer switch detects generator power and transfers load.

The Problem
Total time from lights out to lights on was eleven seconds. There was a big scramble by the various suppliers involved to prove their equipment was not at fault. And every one of the suppliers was right – not one piece of equipment in the system was operating outside of the expected parameters. Still, the system was failing to meet the requirement. Weeks were lost troubleshooting functional components to come to this conclusion.

The Investigation
It was not until the site was looked at as a complete system that the problem was identified. We supplied data acquisition equipment for the site, and the local Cat dealer connected it to multiple points in the system. This allowed for real time monitoring and logging of exactly what was happening with the equipment. The focus was on the control signals – how long did each of these steps take:
  • ATS to detect loss of utility
  • Generator set to start
  • Switchgear to detect proper voltage and close the breaker
  • ATS to transfer to the emergency source – powering the load

It was only after this data was collected and analyzed that the root cause was determined.

The Findings
The switchgear was not initiating the closure of the generator circuit breaker until after the generator set reached rated speed and voltage. Typically load is applied to the generator set when it reaches 90% rated speed and voltage. This added a full second to the overall time. This was further compounded by the fact that that it took between one and a half to two seconds after the generator circuit breaker was closed for the transfer switch to transfer load to the generator sets.

The Solution
The switchgear settings needed to be adjusted so that the circuit breaker closed when the genset reached 50% rated speed and voltage, improving the time to provide power to the transfer switch by nearly two seconds. The installation realized an overall power restoration time decrease of one and a half seconds – putting the site within the ten second power restoration requirement.

While problems on site can often be attributed to a single piece of equipment, it should not automatically be assumed that any single component is at fault. Taking a system level approach to troubleshooting can save valuable time and resources. In the above example, weeks were spent troubleshooting the problem. Once the data acquisition equipment was brought in and the data collected, it took less than a day of analysis to discover the root cause and identify solutions.

This is just one example I have personally been involved in. Have you experienced a similar situation? Have you used different techniques to analyze and troubleshoot installations? Can you offer any tips to others out there? Post your comment below.
Message Edited by woodag on 10-14-2009 09:11 AM

by New member learnee
on ‎10-14-2009 09:47 AM

If you tested your system while utility power was present it sounds to me that your transfer system uses a dwell instead of a phase monitor and control circuit in order to insure that the power supplied does not conflict with the residual energy in the building citrcuits.

   "it took between one and a half to two seconds after the generator circuit breaker was closed for the

     transfer switch to transfer load to the generator sets."

The transfer time can be reduced to ~ 100 milliseconds if a phase monitor & control circuit is used.

And I believe a static transfer switch can transfer in less than 40 milliseconds.

by New member mittap
on ‎10-20-2009 09:53 AM
Is it safe and reliable to close circuit breaker when genset reaches only 50% rated speed and voltage?
by Regular Contributor
on ‎10-21-2009 01:55 PM

To address some of the comments I have received on my post, perhaps I should clarify my statement:

     "....the circuit breaker closed when the genset reached 50% rated speed and voltage, improving the time to provide power to the transfer switch by nearly two seconds."


Based on the site design for this specific installation, the generator circuit breaker did not directly feed the load, but fed a trasfer switch.  Closing the generator circuit breaker at 50% rated speed and voltage did not apply load to the generator. Instead, it applied power to the emergency side of the transfer switch. The ATS then transfered to the generator set after about a one and a half to two second delay - causing load to be applied to the generator set between 90%-100% rated speed and voltage. 

by Visitor sd
on ‎10-22-2009 11:29 PM
Surely, a holistic approach always gives better solution. In this regard it may also be checked whether the residual voltage of the load is small as it is in the case of Slow Bus Transfer. Then there may be no need to check whether source and load voltages are in phase unison.
on ‎11-02-2009 02:16 AM

Dear Sir,


Based on above clear site layout details, I would like to propose my opinion on this issue,


This Delay to supply the electric power to the loads might be related to the distance and the impedance of the power cables. If we can assume the power system (Generator sets, Switchgears, ATS's, Power Cables) by an equation, so through this equation, we will be able to define transite time, I propose to use the MATLAB Software,


OR, the Start up of the gensets should be as fast as possible in order to minimize the starting delay (Maximum 5 s),


I hope my comment could be useful and up to your satisfaction,


Best regards, 

by Visitor dsandberg
on ‎11-02-2009 09:45 AM
Also keep in mind that timing can be effected when doing a test (hot to hot) vs pull the plug. When using the test switch, some manufacturers incorporate a delay so that bumping the test switch will not inadvertently start the engine of transfer the load. Also, the acceptable levels need to be carefully considered depending on the sensitivity of the connected equipment. While lowering the acceptable limit in this case solves the immediate problem, it may well create another.
on ‎10-21-2010 05:42 PM

One thing Chad's example alludes to but doesn't address is all the parties involved had the primary goal of exoneration versus solution.  This blame game eats time and materials for all parties involved with the owner often left dealing with the cost of delayed opening and unnecessary "repairs".  In my years in this industry, whether it's lights not on in 10 seconds or a generator not playing nice in the sandbox with a UPS, there seems to be two ultimate constants: 1. The generator is malfunctioning, and 2. Everyone else runs leaving the generator supplier to solve the problem.  By no means is this universal, but ask a few generator suppliers and I can assure you accuracy of my assessment is in the 95th or higher percentile from their point of view.


The solution is a single source of responsibility.  If the generator sets, paralleling gear, automatic transfer switches, and uninterruptible power supply are all purchased from a single (reputable) source, there is no room for the blame game.  Our customers like the term "one throat to choke".  Pick your euphemism, but the result is predictable.  A reliable supplier partner has a single motivaton - solving your problem.

on ‎12-09-2010 10:06 PM

Keith: like your mantra of "one throat to choke", and in situations where it is possible to purchase an entire system from a single supplier it is generally the best course of action.


Totally agree that a systems-based approach is the best way to troubleshoot issues like these.


Cat Rental Power - <a href="http://www.catrentalpower.com.au" title="Generator Suppliers">Generator Suppliers</a>

by Travestis
on ‎10-10-2016 08:54 AM

Vey article ! :D

Help us grow the Caterpillar Community: Invite a Friend

Meet Our Bloggers