Specific comments:

1. The report fails to address the effects of decision-dependent uncertainty on its dataset. The consequence of this oversight is that the dataset is invalid as of the date of publication.

2. The report claims the reported data can be used for model validation but fails to explain how the data can validate a model (nuclear plant risk model) in light of the fact that the dataset lacks the necessary correlate (that is, core damage or radiation release.)

3. The data should be made useful by adding information related to the efficacy of protection. For example by correlating protective system unavailability to the fire data, the statistics would be useful to investigators.

Summary Explanation of Comments

Effective risk management requires understanding and addressing the root cause of events that can lead to catastrophe; a useful fire database would explain the root cause for fires, possibly categorizing similar causes The data in the report should at a minimum be recast to indicate events where availability of protective system equipment is affected (reduced).

The DRAFT NUREG in a few places suggests the data can be used to produce “more realistic modeling of fire risk”. This claim is misleading for many reasons but particularly when used in a setting of dynamic risk management. Models of fire hazard (for example, probabilistic models) cannot be verified using data as proposed in the DRAFT NUREG; for the model to be verified, data for fire terminating in catastrophe (for example, core damage or radiation release) would be needed. If it were possible to have such data then a model ([pra]{acronym-label=”pra” acronym-form=”singular+short”} or some statistical method) could be validated; otherwise, in the absent ouch data, there is no way to validate “nearness” to reality. In point of fact, lack of evidence may be an indication that, while occurrence of fire is certainly to be avoided, fire is of less concern compared to other protective system breakdowns that have actually led to catastrophe.

Dynamic risk management in learning organizations will cause observed fire root causes to be eliminated or significantly mitigated as they occur; that is, the intent of risk management in learning organizations is change the future based on observations. This effect, changing operation, maintenance, or design based on root causes in response to fire events can be characterized as decision-dependent uncertainty.

Details on Explanation of Comments

1. The fire statistics developed in the DRAFT NUREG are not helpful to investigators interested in avoiding catastrophe (core damage or more importantly, radiation release) because (at a minimum) they are disconnected from protective equipment availability. Only when sufficient protective system equipment are unavailable will catastrophe follow; a relevant example would be the sustained loss of electrical power in Generation II light water reactors due to fire. The NUREG should be rewritten to produce statistics on data related to protective system equipment unavailability due to fire. Such data would be in the form of event occurrence (date, time, where) and consequential protective system equipment unavailability (list of equipment affected, duration of unavailability); these data would be useful to reliability engineering investigators.

2. When using empirical data as a predictor of future performance, as proposed in the DRAFT NUREG, there are at least two important cautions (enumerated below) that must be observed. Nowhere in the DRAFT NUREG is it explained how these cautions are being considered.

   1. The data must be meaningful to the observation of interest - Lacking statistical data on catastrophe following fire or equipment unavailability is the only statistic with relevance to reliability. Note that the DRAFT NUREG has no information about protective system unavailability as part of the statistics developed. If the data are not meaningful in this regard, the statistics will not be helpful to investigators who intend to reduce the probability of protective system equipment unavailability due to fire.

      Statistics for any complicated process with risk for harm are obtained to understand frequency of harmful events. For example, data on pedestrian crossing deaths at an automobile intersection are collected by noting when a death occurs. At some level of loss of life, actions are taken to reduce the risk (traffic deaths) using, for example traffic signals, overhead walkways, and other means; the data are not collected at a level that checks for frequency of human error on braking the automobile, driver attentiveness, condition of the automobile, likely speed of approach, weather conditions, etc. Only the frequency of pedestrian deaths is required to bring about risk management action.

   2. The end use of the statistics is (rationally should be) to improve the availability of critical protective system equipment or decrease the occurrence of fires affecting critical protective equipment availability (that is, risk reduction). A direct consequence of actions taken to avoid future occurrence (depending on the efficacy of the actions) invalidates the data used to “predict” the future. Prediction of future performance is the domain of decision-dependent uncertainty; decision-dependent uncertainty is avoided out of necessity by investigators using historical datasets by being careful to preserve prior data validity (ensure the future is predicted by the past). For example, hedge fund managers use historical data to develop high frequency trade algorithms and many are very successful. However, unlike risk management in commercial nuclear power where workers intend to change the future, the clever hedge fund manager is very careful to ensure her trading volume doesn’t affect the past data: she must avoid the temptation to become “too greedy”.

3. The prior comments are not entirely naïve; the reviewer understands the end use of the DRAFT NUREG statistics is [pra]{acronym-label=”pra” acronym-form=”singular+short”}. A critical point is [pra]{acronym-label=”pra” acronym-form=”singular+short”} may or may not contain (have as a basic event) the most interesting aspect of a particular fire event that bears on risk management – the failure mode (or mechanism) which is the root cause.

   For example, dust buildup in a high voltage enclosure has been shown (this has happened) to result in switchgear fires; a particular [pra]{acronym-label=”pra” acronym-form=”singular+short”} is very likely to have an initiating event of fire in such an enclosure as a basic event. Depending on the enclosure, the [pra]{acronym-label=”pra” acronym-form=”singular+short”} may include a sequence that contains the event as a cause for protective equipment unavailability. The advantage of [pra]{acronym-label=”pra” acronym-form=”singular+short”} over other forms of reliability analysis is that it generally goes further than FMEA for example by attempting to comprehensively include all scenarios that could be envisioned from the event.

   In contradiction to what is suggested in the DRAFT NUREG, supplying more detailed statistics to a [pra]{acronym-label=”pra” acronym-form=”singular+short”} in pursuit of quantification will not help plant workers or regulators develop new strategies, different inspections, or tests that would reduce the probability of future catastrophe (this point is closely related to Hansson’s “Tuxedo Fallacy”).¹ Unless this reviewer is missing a more subtle point, it seems the objective of the work documented in the DRAFT NUREG is (should be) intended to help risk management. Such an objective can only come from engineering analysis of the event, correctly understanding the root cause, and developing strategies that would reduce protective system equipment unavailability (in this case from fire). Once actions are taken, the prior data and concomitant statistics presented in the DRAFT NUREG are invalidated.

Discussion

The commercial nuclear power industry is experiencing difficulty competing in the current electrical market leading to the extent that several plants are forced into early retirement; and therefore plant operators (especially “merchants” in deregulated markets) are looking for maximum benefit for resources expended.

This reviewer notes that developing and maintaining a “Fire [pra]{acronym-label=”pra” acronym-form=”singular+short”}” costs plants millions of dollars.² Such large costs would be justified if they led to, or could be shown to lead to, significant improvements to safety. The comments made in the previous are to say that expenditures that would add complexity to PRA will not improve the efficacy of protection against core damage or radiation release. As stated above, based on basic risk management practice, this reviewer claims it can not be shown that spending resources that would apply the statistics developed in the DRAFT NUREG (that is, add complexity to a [pra]{acronym-label=”pra” acronym-form=”singular+short”}) will change the risk of core damage or radiation release in a nuclear power power plant; although the assessed values (numbers) may change, it would have no effect on safety.

This reviewer recommends a more well-founded statistical analysis be conducted that at least gives estimates of the current state of affairs in terms of fires on protective system equipment availability. Of course any fire should be prevented if for no other reason than worker protection; root cause analysis and risk management against recurrence (addressing the root cause) is the best path to prevention.

1. Hansson, Sven Ove. “From the casino to the jungle.” Synthese 168, no. 3 (2009): 423-432. ↩

2. https://www.nrc.gov/docs/ML1300/ML13004A391.pdf, accessed June 19, 2019. ↩