Using a Rapid Assessment Tool to Improve Water Quality Testing Results

First published in Water e-Journal Vol 6 No 1 2021.

Altogether Group have developed a Rapid Assessment Tool (RAT) to assess risk and provide guidance in monitoring water quality.

DOWNLOAD THE PAPER

Abstract

The role of monitoring is fundamental to the successful implementation of the risk framework in the contemporary water utility. Meaningful data interpretation from that monitoring can identify and implement timely risk management to protect public health. 

Altogether Group, an Australian multi-utility operating along the east coast, identified a need to improve the timely understanding of the materiality and reliability of exceedances reported by its laboratory provider, to guide an appropriate incident management response.  

Development of the Rapid Assessment Tool (the RAT) fits into the existing incident management framework and is activated upon receipt of out-of-specification water quality results from the laboratory. The RAT involves a risk-based investigation component (location of sample, significance of parameter, critical control point performance), overall risk category assignment (low, medium or high risk), and direction for further actions (to fit within the incident management framework). 

Validation via team review and past results demonstrated that the tool is reproducible, reliable, easy to use and met design objectives. Benefits include access to a systematic tool for consistency and increased confidence in decision making, leading to a reduction in public health risk by ensuring incidents are managed promptly and appropriately and corrective actions fit the risk profile.   

The RAT has been successfully used following out-of-specification water quality results. Altogether will continue to use and refine the RAT during incident management. 

Introduction

Altogether Group (Altogether) is an Australian sustainable utility provider with a multi-service offering incorporating drinking water, recycled water, wastewater and stormwater harvesting as well as energy services. The utility’s decentralised services integrate local treatment, production, reuse and storage solutions. Altogether operates under the Water Industry Competition Act 2006 (NSW), through a series of operating licences, for a number of water schemes within NSW, for sewage, drinking and recycled water services.  

Altogether is also guided by the governing risk-based ‘guideline suite’ for water quality management within the National Water Quality Management Strategy family of guidelines, including a water quality management system for different water industry products, e.g.: 

• The Framework for Management of Drinking Water Quality (the Framework) within the Australian Drinking Water Guidelines (ADWG) and 
• The Framework for Management of Recycled Water Quality and Use within the Australian Guidelines for Water Recycling (AGWR). 

Altogether is committed to producing quality and safe products for its customers and, in pursuit of better services to customers, identified a need to improve its ability to: 

• Rapidly assess water quality monitoring results.
• Improve the interpretation of the materiality of the result. 
• Improve the notification and reporting of the result (internally and externally). 
• Improve risk communication. 
• Improve out of specification water quality responses. 

To do this a risk-based approach was considered the most appropriate. Contemporary water and enterprise risk management frameworks are underpinned by the understanding and management of risks (e.g. Davison, 2011). 

Globally, water quality risk assessment tools have been utilised to ensure safe public health outcomes and have parameters selected based on the population that is most vulnerable. An example of a tool is RADWQ, where the selection of indicators and parameters for inclusion is based on prioritising those that will have the greatest impact on the health status of the entire population, especially the most vulnerable groups (e.g. children, the poor, pregnant women; WHO 2012).  

In Australia, the ADWG and AGWR provide overarching principles and guidance on the management of risks and water quality incidents. What these guidelines do not include is a specific water quality and system assessment tool to facilitate understanding of the materiality of monitoring and other surveillance findings. In essence, and regardless of the standard or framework used, good risk assessment and management (Figure 1) is underpinned by the continuous application of the following: 

• System understanding, including products and services, stakeholders, the operating environment and system characteristics 
• Identification and assessment of risk (within the context of the community or circumstance), including contextualised risk objectives and tolerability 
• Management of the risk 
• Monitoring of the risk and the mitigation measures. 

Figure 1: Risk understanding, assessment, management and monitoring (Davison, 2020). 

The role that monitoring plays within the risk framework is fundamental to the successful implementation of the framework and successful management of risk overall. Different types of monitoring are required with each having its own purpose relative to where it fits within an overarching source to endpoint water product supply chain, management system and operating context (Box 1).  

·       Baseline monitoring – where are we right now?

·       Validation – will it work?

·       Operational monitoring – is it working now?

·       Verification monitoring – did it work?

·       Customer request monitoring – did our customers think we got it right?

·       Investigative monitoring – can we make it work better?

Box 1: Example monitoring types and their purpose (Davison, 2020). 

Organisations need to understand the ‘what, why, where, when and how’ of monitoring, regardless of type, to understand and manage risk. Fundamental to all these monitoring types is not just the collection of data, but the interpretation of the data, i.e. turning the numbers and observations from the monitoring into information with which to better understand and manage risk. 

Contemporary water utilities are increasingly complex organisations, with a growing number of products and services and associated data collection locations, data streams and databases. To manage this situation, tools are required to facilitate the interpretation of water quality results. Never is data interpretation more important than in identifying and putting in place timely risk management to ensure that public health is protected. The Framework covers incident and emergency management at Element 6 (Table 1), with these obligations also often being imposed on utilities either directly through legislation or through operating licence requirements or water supply agreements. 

The Framework requires water suppliers to define incidents in conjunction with relevant agencies. It is important to define incidents as this helps to ensure that: 

• There are no surprises for any party in managing incidents and emergencies.
• There is no lost time in understanding whether something is or is not a material public health risk.
• There is no lost time in understanding whether a notification to the regulator is required. 

For some events, notification may be formalised, e.g. through reporting manuals. For other events, a requirement to notify may be uncertain. Monitoring results help a business to decide if something is: 

• ‘Green’ – considered business as usual operating conditions. 
• ‘Orange’ – indicative of a process trending out of control but situation is still manageable. 
• ‘Red’ – indicative of a process or product being no longer fit for purpose. 

It is therefore important for utilities to understand the public health materiality of their monitoring results, and what they need to do with that information i.e. is the result (or combination of results) indicative of normal operating conditions or is it indicating that the system is or may not be fit for purpose, and therefore requires the incident management process to be implemented (with the appropriate corrections and notification involved)? 

Component	Action
C 6.1: Communication	A 6.1.1: Define communication protocols with the involvement of relevant agencies and prepare a contact list of key people, agencies and businesses.  A 6.1.2: Develop a public and media communications strategy
C 6.2: Incident and emergency response protocols	A 6.2.1: Define potential incidents and emergencies and document procedures and response plans with the involvement of relevant agencies   A 6.2.2: Train employees and regularly test emergency response plans  A 6.2.3: Investigate any incidents or emergencies and revise protocols as necessary

Table 1: Element 6 - Management of incidents and emergencies (Source: ADWG, NHMRC/NRMMC, 2011). 

To facilitate the above, and to support the earlier identified goals of Altogether to improve its incident assessment and response, the Rapid Assessment Tool (the RAT) was developed. In this paper, we describe the development of the RAT, and the benefits it has achieved for improved water quality and public health risk management at Altogether.  

Methods: Convening the expert panel

Several outcomes were important in developing the RAT: 

• It had to fit within and support existing incident and management architecture. 
• It had to be consistent with organisational risk assessment protocols. 
• It had to be evidence-based. 
• It had to be robust. 
• The outcomes had to be credible. 
• It had to be easy to use. 

The following steps describe how the tool was designed, to meet the above objectives. 

Fit within Existing Architecture 

Altogether has a licence obligation which requires it to immediately notify any incident in the conduct of its licensed activities that threatens, or could threaten, water quality, public health or safety to relevant authorities. Altogether achieves this obligation through development and implementation of an incident and emergency management framework including a number of key documents: 

• Overarching Incident Management Plan
• Water Operations Incident Management Procedure.
• Incident Notification Protocol with NSW Health.

The framework is also supported by various out of specification policies and procedures, depending on the water type. Each scheme also has its own Incident Management Manual which contains an emergency contact list of key people, agencies and businesses which may need to be contacted in the event of an incident or emergency. The incident and emergency management framework has been developed in consultation with NSW Health and with reference to NSW Health guidance materials.¹ 

The team designed the RAT to fit within, and support Altogether’s existing incident and emergency management framework. Principles for the design were based on: 

• A seamless fit into the Altogether water quality management systems and incident management documentation.
• Practicality of use on receipt of an out of specification water quality result.

Design of the RAT 

To help inform Altogether on whether a monitoring result was ‘out of specification’ and the public health materiality of the result, the project team used the following questions (Table 2) to help drive development of the RAT: 

1. Where was the sample taken?
2. What was the level of significance of the parameter being monitored?
3. Did the result occur in conjunction with any other result?
4. What should be done with the result? 

Reason	Example / Further Information
1. Location of sample
Depending on the location of the sample, an out of specification water quality result from that sample will provide more or less time to act, and can also provide insight into the cause of the contamination, i.e. likely treatment or network causes.	The location of a finding, within the source to end point supply chain, is material from a public health risk management perspective and was the first parameter to be considered in the RAT (see Figure 2).
2. Level of Significance
The significance of a water quality finding depends on the nature of the parameter being monitored, the monitoring type (i.e. frequency and intent) and the levels. For key water quality parameters, and for the purposes of the RAT, the evidence base for the significance of the finding was determined from industry literature (such as guideline values), discussion with industry colleagues, regulators, experts, Altogether operators and overall water quality team experience. The evidence base for the significance of each water quality parameter was documented e.g. ADWG or AGWR value.	As an example for treatment, if spores of sulphite-reducing clostridia (SSRC) are found at the outlet of a filter, this is indicative of protozoal filter breakthrough risk (where the SSRC are used as a surrogate for cryptosporidial oocysts).
3. Coincidence of Result
Altogether has designed its source to endpoint supply chains to meet the ADWG/AGWR multiple barrier approach i.e. if one barrier fails, there is another in place as insurance, to help protect public health. Therefore in designing the RAT, the team needed to understand the significance of system performance overall, and not just consider the result in isolation.	Key questions which helped to feed into development of the RAT included:  Were the critical control points performing as planned?  Were there any other activities occurring in the system at the time of the finding?  What were the results from other parameters in the same location?  What were the results elsewhere in the system, including upstream of the finding?  Was there anything to suggest network or other asset (e.g. storage tank) integrity had been compromised?  Were there any recent customer complaints in the local area?  Was there a source water change?  Were there any other operational considerations including cross connections, pressure fluctuations, or standpipe usage?
4. Response to Finding
Individual Question Outcome:  The response to finding was determined based on assessment, outcome of the risk category and actions required.  Each question in the RAT was scored on a risk-based traffic light system of green (low risk), orange (medium risk), and red (high risk). The criteria for each risk category was based on best industry practice, regulator focus, target operating parameters, historical data and previous operating experience.	Examples:  High risk: out of specification result at the network point of use (the water had potentially already reached customers).   Medium risk: out of specification result at entry point of water to the network (more opportunity to make correction prior to the water reaching customers).   Low risk: out of specification result for an aesthetic guideline limit (quality not safety parameter).
Overall Category Outcome:  To guide the next steps, an overall risk outcome category was decided, based on the number of outcomes in the low, medium, and high risk categories.	Overall high risk category outcome: any findings of high risk or more than two findings of medium risk.   Overall medium risk category outcome: two or less findings in the medium category.  Overall low risk category outcomes: all low risk findings.
Follow up actions:  Vary slightly based on whether the out of specification detection was microbial or chemical.	Microbial out of specification result: for a detection of E. coli, the first action regardless of risk level is to contact the laboratory and organise confirmation of the E. coli using a second method. Then, if the overall risk outcome is low risk, the follow up actions are resampling, with follow up sampling results guiding further actions. A medium or high risk result prompts resampling plus implementation of the scheme-specific out of specification procedure.  Physical / chemical out of specification result: resampling with follow up sampling results guiding further actions e.g.  Low risk result: investigation based on the scheme-specific out of specification work instruction  High risk result: consultation with the relevant public health unit regarding rectification, potential alternative supply, public warnings, investigation and sampling.  Follow up actions were developed with regard to the NSW Health guidance NSW Health Response Protocol: For the Management of Physical and Chemical Quality and Managing pathogen risks in drinking water: Response protocol for water utilities and public health units

Table 2: Development of the RAT. 

Figure 2. Conceptual source to endpoint supply chain for drinking water (DW) and recycled water (RW).  

Microsoft Excel was used to build the tool based on the development steps noted in Table 2, and contains three parts: 

1. Investigation (questions and manually entered responses).
2. Outcome of risk category (automated via a logic statement which collates and assesses the overall responses and assigns a risk category).
3. Actions to take dependent on determination of risk category.

Results

Using the RAT 

The questions developed to determine the significance of a microbial finding are provided in Table 3. Answers from an out of specification microbial water quality result, likely an erroneous laboratory E. coli result are shown. A similar table was developed for physical/chemical findings (not shown). Responses to the questions and other evidence as appropriate and input into columns within the RAT (not shown here due to space limitations, however for ease of reading example fields are highlighted). 

The outcome of the automated risk category assessment is shown in Table 4. As the answers generated in Step 1 resulted in two entries in the medium risk category, the tool calculates the overall risk rating as ‘Medium’ (Table 4). The resulting suggested actions dependent on the risk category classification are shown in Table 5. For a ‘Medium’ risk rating, this is to follow the scheme specific out of specification work instruction, which includes zone wide resampling and possible operational changes at the local water centre.   

Table 3. Step 1 of the RAT (investigation questions and risk categories – microbial finding).

 

Table 4: Step 2 of the RAT (example outcome of the risk category assessment). 

Table 5: Step 3 of the RAT (actions to take based on risk category assessment). 

Validation of the RAT 

Validation of the RAT was undertaken through a two-step process: 

Team review:

• Internal review: Water Operations Management, the Process Engineer and the Water Risk and Compliance positions,  
• External review: water quality risk management experts.

The review of past incidents: 

• Previous incidents were used to measure the outcome achieved by the RAT against the incident outcomes and actions taken at the time.  

Validation Results 

Table 6 further shows the results of the out of specification water quality result shown in Table 3, that was used to test the tool. At the time of the finding, the Water Quality Team suspected sample contamination as the cause of the finding, however, there was no recordable or formal way for Altogether to justify this position to the NSW Health Public Health Unit or regulator or to justify the operational response. Information surrounding the out of specification finding was entered into the RAT (e.g. Table 3). Answers are summarised in Table 6, highlighting the two answers that fit the ‘medium risk’ category, resulting in an overall outcome of ‘medium risk’. The resulting actions of resampling and referring to the scheme specific out of specification work instructions supports the actions undertaken by Altogether at the time and shows an outcome that passed a ‘sanity check’ and provided evidence and improved records. This outcome demonstrates that the tool is reproducible and reliable, and met the team’s design objectives. 

Result	Justification
Low Risk   (12 answers)	·        Corresponding chlorine residual of >0.5 mg/L (actual result 1.6 mg/L)   ·        Turbidity of <2 NTU (actual result <1 NTU),   ·        pH between 7.5 – 8.0 (actual result 7.75)  ·        Coliform <1 CFU/100mL, (actual result <1 CFU/ 100 mL)   ·        Indication of sufficient chlorine residual for secondary disinfection.  ·        No water quality results to indicate comprised disinfection or contamination.   ·        No other activity in the network indicating operational/ network integrity issues.  ·        Storage tank inspection indicated no breaches.
Medium Risk   (2 answers)	·        Temperature 23 – 30 C (actual result 23.5 C)  ·        Sample was from the point of supply indicating:  o    there was still time to rectify issues if present and   o    the potential contamination had not reached the customer point of use.

Table 6: Outcomes of applying the RAT to a likely erroneous historical E. coli laboratory finding. 

Discussion

The RAT has been successfully utilised following receipt of out of specification water quality results. In real time, the RAT fits seamlessly into the incident management framework and has been a valuable tool in allowing Altogether to assess the materiality of the finding, in consultation with public health units. 

Following out of specification water quality results from routine verification monitoring, as a first step the Altogether team inputs all relevant data into the RAT tool to gauge the level of risk and guide downstream actions.  

Development and subsequent use of the RAT has provided Altogether with significant benefits. Most importantly, public health risk is reduced by ensuring that corrective actions fit the risk profile, and that incidents are managed appropriately and in a timely fashion. 

Confidence in decision-making regarding further actions to undertake has increased as the onus is not on an individual to make and justify a decision, rather use of a validated tool. Having this systematic and reliable tool assists with making assessments and actions consistent over different incidents and different incident managers, thereby increasing regulator and customer confidence through an easily repeatable process. 

This also results in time and cost savings by streamlining the decision-making process and minimising the chances of unnecessary and potentially costly operational adjustments.  

Risk assessment descriptors and incident and emergency management documentation are more easily translated into one easy to use tool. This results in improved staff understanding of incident classification and response for the current and future incident. Having a formal process for recording results and justification of actions and not just keeping the information ‘in your head’ or reliant on any individual further provides documented evidence of actions improved incident management governance for actions taken. The records also provide valuable information for incident debriefs and can be used for training purposes, further increasing efficiency in incident management.  

As the tool is used more it will be further refined and improved, by using ongoing results and outcomes from incidents and out of specification water quality results. 

Conclusion

Altogether has been able to develop an evidence-based rapid and reliable assessment tool that can be used upon receipt of out of specification water quality results to assess the materiality of the result and to guide further actions. The tool provides a mechanism to achieve the balance of cost, risk and performance to create sustainable value (ISO55000, 2014). That is, ensuring Altogether can achieve its commitment to provide safe, high quality drinking water and recycled water, without unnecessary expense and action.  

Validation of the tool was successful, and it has since been used following out of specification water quality results. Altogether will continue to use and refine the RAT through user experience during out of specification results events. This tool could be used by other water utilities to help guide their own risk assessment and guide downstream actions.  

Acknowledgements

The authors would like to thank Duncan Middleton of Seqwater for helpful discussions on how Seqwater assess the public health materiality of water quality results. 

About the authors

Annette Davison | Principal Risk Analyst and founder of Risk Edge® Pty Ltd and R&D Manager and co-founder of D2K Information Pty Ltd. Annette is known as The Water Risk Doctor® and is a specialist in water cycle risk management, regulation and policy. Annette is the Australian Water Association’s 2021 Australian Water Professional of the Year. Annette has more than 30 years experience in environmental management, integrated water cycle management, risk assessment and paralegal work across the private, multinational, academic and state government sectors. Annette is a recognised global leader in risk assessment for water products and environmental management, validated through multiple national and international awards, invited contributions to conferences and seminars, and substantial publication list (including invited journal articles, book chapters and books). Annette is a qualified Lead Auditor.

Audrey Killeen | Audrey has a BSc in Agricultural and Biological Engineering and eight years experience in the industry across operational and process oversight, management system development and implementation, operator training, business improvement initiatives, risk assessment, onsite commissioning/optimisation and process design. She has developed and implemented risk-based quality management systems as per the Framework in the Australian Drinking Water Guidelines for numerous clients across Australia.  She is currently working as the Water Operations Manager for a private water and energy utility called Altogether Group. She manages a team looking after the operation, process optimisation and water quality systems for 8 water schemes which supply drinking water, wastewater and recycled water services under the Water Industry Competition Act.

Sarah Schroeder | Sarah has a PhD in environmental microbiology and BBus/Applied Science (Honours). She has over 15 years experience in the water industry, encompassing water and wastewater research, more specifically biological phosphorus removal, lecturing in microbiology/ public health, and recycled water/ drinking water regulation. More recently Sarah has specialised in utility water quality operations including recycled water and drinking water quality management systems, risk management, data analysis and interpretation, network water quality and optimisation, and response to customer concerns. She is currently the Water Quality Systems Manager at Altogether Group. Sarah has been an active member of AWA including Qld YWP mentoring co-ordinator and president, and recipient of the 2013 Qld YWP of the Year.

Sanjay Kumarasignham | Sanjay Kumarasingham is an environmental engineer with specialisation in water and wastewater treatment. He has held several positions within the water utility sector. He is recognised for his technical and stakeholder management skills. His passion is in valuing water – which includes treatment, beneficial reuse and energy generation. Skills includes project management, operational management, process engineering and strategic planning. Roles held includes Plant operator, Shift Engineer, Process Engineer, Plant Manager, Area Manager, Technical Manager, Environmental Manager and Operations Manager. Wastewater expertise includes project upgrades including design reviews, operating, commissioning, performance testing, take over testing and defects rectification in preliminary, primary, secondary and tertiary unit processes spanning over 30 plants. Collaborative work includes crown research institutes and universities.  

References

Davison, A.D. 2011, ‘Enterprise Risk Management. Risk appetite and risk tolerance: how robust are yours?’ Water Journal vol. 38, no, 5, pp. 65-68. 

Davison, A. 2020, ‘The Application of ISO 31000 to Drinking Water Quality Risk Management: A Practical Approach.’ Risk Edge® Pty Ltd. 

International Organization for Standardization (2014) Asset management – Overview, principles and terminology. (ISO standard no. 55000:2014).  

National Health and Medical Research Council, National Resource Management Ministerial Council 2011, Australian Drinking Water Guidelines Paper 6 National Water Quality Management Strategy. 

World Health Organization (WHO) 2012, Rapid Assessment of Drinking Water Quality: a handbook for implementation. World Health Organization.