Condition assessments require judgment. A typical practice is to disaggregate the evaluation of each asset into a handful of attributes (weighted or unweighted) that describe the asset’s condition. The brainless course of action is to weight each attribute by its relative importance and then do some questionable math to aggregate all of the attributes into a final condition score. But is this the best way to perform a condition assessment?
Observations
Basic attributes involve the five primary exteroceptive senses – general sight/appearance, sound, smell, touch, and taste – while others involve more specific observations (such as frayed wiring, burn marks on electrical panels, leaking lubricants) and non-invasive field testing (such as vibration analysis, acoustics, and thermography).
Modes of Failure
For each asset class, a unique set of failure modes exist. Unique criteria of field observation and associated measurements (like vibration analysis) are required. This requires both time and money to perform and collect properly.
Questionable Math
Two types of questionable math arise in a condition assessment.
Math, specifically multiplication, should be performed on real numbers with continuous scales. Observations using a 1 to 5 scale or 1 to 10 scale are performed with ordinal numbers.
The data is mixed. One form of missed numbers is aggregated scales that use ordinal numbers from observation and real numbers from testing devices. Another form of mixed numbers is binary (1 is yes and 5 is no) versus graded (full range of 1 to 5).
The math gets questionable when we mindlessly combine mixed scales. The reason we disaggregate and weight the attributes is to avoid bias. Ironically, we are baking in a different form of bias when we do questionable math.
Additional Considerations
Some additional considerations are important when establishing field condition assessments and using the associated data for analysis. These are six to think about:
1. Relying on Visual Data
The degree to which to rely on visual data, especially in mechanical and electrical systems. This can also be a valid concern in piping systems, especially in pressure pipe systems where test data is typically costly, and external subsurface data can be misleading for internal pipe conditions/integrity.
2. Field Measurement versus Visuals
The degree and weighting of non-invasive field data collection techniques in comparison to visual observations.
3. Hot or Not?
The need to have the equipment in operation (or not in operation) when the field condition assessment is performed and whether to perform some of the work in off-hours. If observations are performed when the equipment is not in its desired operating state, should those results be weighted differently?
4. Safety
What weighting and priority are given to health and safety considerations? An example comes from DOT bridge weighting systems where an unsafe condition results in a bridge being taken immediately out of service (even the most poorly rated bridge structures do not pose an imminent danger to the workers or public health & safety).
5. One Measure or Many
For repairable mechanical systems, the degree to which one measurement (such as bearing vibration) should weight the overall condition score.
6. Random Failures
For some electrical assets, many fail randomly regardless of the assessment process, the degree to which any field measurement or visual has relevance over actual performance history or manufacturer life data analysis.
Weight of Evidence, Not Weight of Attributes
The term weight of evidence is used here to describe the whole set of evidence presented on an asset’s condition, such that the overall weight of the evidence can be said to favor scoring an asset a 2 versus 5.
In practice, look at the raw score of each attribute and simply score the asset by which the number appears the most frequently. In a spreadsheet, simply use the "mode" function.
Another option is practice is to identify which observations matter most and use this as a tipping point. In other words, if there are three attributes with a score of 3 and two attributes considered more important with scores of 2, then call it a 2.
The weight of evidence approach also drives the evaluator to more closely assess whether an attribute outside an acceptable range should override the entire weighting process.
Example Using Weight of Evidence
The maximum deviation in any one attribute should override the aggregate weighted score. A classic example is from the US Army Corps of Engineers, where a weighted attribute system is used to evaluate the overall condition of a sheet piling system until such point as the maximum level of a single attribute – in this case, the separation of adjacent sheet piles - overrides all other attributes that are being weighted. The separation attribute overrides all other scores, making the condition "critical."
Moving Forward
To weight or not to weight? That is the question when it comes to disaggregating attributes and determining an asset condition score.
Disaggregating an evaluation into multiple attributes is a good way to de-bias individual judgment. The trouble comes when we do questionable math to further show the objectivity of the process. Avoid weighting attributes and then doing questionable (and deceiving) math when doing condition assessments. Use the weight of evidence approach instead. After all, it is all judgment.
JD Solomon Inc provides solutions at the nexus of the facilities, infrastructure, and the environment. Contact us for more information on our asset management ASAP approach, condition assessments, renewal & replacement forecasting, preventative maintenance programs, and reliability assessments. Sign-up for monthly updates on how we are applying reliability and risk concepts to natural systems.
Comments