Energy trading and risk management (ETRM) software is that category of software applications, architectures and tools that support the business processes associated with trading energy commodities (crude oil, refined products, natural gas and electric power, etc.) ETRM systems give you better control of your business, provide a clearer view of the supply chain and cost-base, but are also some of the most tightly integrated and functionality-rich applications within a business and IT landscape. Hence developing a risk-informed software testing strategy early in the project lifecycle will improve delivered solution quality without inflating implementation costs.
Purpose & Objective Of Testing
The primary objective of software testing is to provide an objective evaluation of the quality of a particular component or components being reviewed, and to surface examples of non-performance – called defects. The process of testing can be broadly considered to be seeking to validate and verify that the ETRM solution and associated integration:
- Meets the needs of the business and IT stakeholders.
- Consistently performs as expected.
- Is ready for productive deployment.
To evaluate against this high-level set of standards several different types of testing can be utilized (e.g., installation testing, regression testing). Most best practice guides for software testing target the process of assessing custom code so many of the recommendations need not necessarily be applied to a commercial off-the-shelf solution like an ETRM. Part of what companies pay for when they purchase third-party software is a foundational level of product QA performed before it’s ever turned over to the customer for use.
A Risk-Based Approach To A Testing Strategy
A quick glance at a software QA textbook, a testing whitepaper or even Wikipedia reveals an almost dizzying array of testing approaches, levels, and types. Spending time studying them in more detail might convince the reader that all serve an important purpose and should be incorporated into any IT project to cover all the bases, but that simply wouldn’t be affordable. Quality can be built into the integrated solution in many ways from the outset of the project, and the testing strategy should clearly define which methods will be employed to do so. Conversely, it should specify which will not…scope creep can take many forms and testing is often one of them. How, then, can it be decided which testing activities should be included?
At its core, and in this context, software QA is about reducing the risk that, after going live, the solution fails in a way that has a significant adverse effect on the business. Every organization’s appetite for risk is different. Below are some key areas of potential failure to consider:
- Direct impact to billing and/or cashflow.
- Lack of visibility to active commodity price risk.
- Inability to manage counterparty/credit risk.
- Failure of a key financial control.
- Gap in regulatory compliance.
There are obviously other risks that’ll be important to consider for certain organizations.In fact, a company’s enterprise risk matrix and the project’s risk register can both be examined to identify others that should be contemplated. Once these risks have been enumerated, they can be used to inform the “what” and the “how” for the testing strategy. Reviewing the project scope document(s) and the ETRM solution architecture will aid in firming up units and assemblages that need to be tested.
The testing strategy should define the tools and methods to be utilized to verify that the system responds correctly. In addition, it’ll outline what needs to be accomplished, how to achieve it and when each type of test will be performed. The testing strategy shouldn’t be considered in a vacuum and must be aligned with other major project activities such as communication, training,conversion and cut over, technical change control, etc. as part of an integrated delivery approach. The resulting strategy can then serve as a framework for preparing a detailed set of test plans and appropriately incorporate them into the overall project schedule.
Traditional Testing Approaches, Levels & Types
Once project leadership has settled on “what” needs to be tested the next step is to map out and plan for the “how”. While there are several different approaches that can be taken when testing integrated ETRM solutions, a few can be reasonably set aside as they’re focused on more technical aspects of software quality that don’t necessarily address any of the risks discussed previously.
Myriad types of testing exist, each tailored to achieve the verification or validation objectives of an ETRM software testing program in a slightly different way focusing on quality attributes of an integrated solution. Not all these types of testing will be appropriate for, or will be utilized as part of, the test program for an ETRM project. Also, these types of testing procedures span all four primary testing levels (defined below).
One static testing approach that’s very common to ETRM system implementations takes the form of walk-throughs, often referred to as scenario modeling.This approach couples well with dynamic testing approaches that leverage predefined test cases applied against technical objects like interfaces to round out a more complete software QA program.
Defect Management
Defects are a normal part of software testing and there are a number of tools that can make logging, tracking, and reporting on them more straightforward for the team and project leadership.However, clear definitions of what defects are (and aren’t) combined with an easy-to-follow scheme for prioritization sometimes take a back seat to tools and process but can positively contribute to testingoutcomes.A defect couldbe defined as an identified software issue that results in a failure of system performance as compared to the documented, approved system configuration blueprint or functional specification.
One relatively simple way to agree on a rubric for defect prioritization is to align it with the IT department’s existing incident severities since this is something with which most stakeholders will already be familiar. In any event, these levels should serve the purpose of unambiguously determining which issues the project can go live without resolving, which must be remediated before moving to the next level of testing (i.e., entrance/exit criteria) and so on.
As more organizations move towards Agile implementation frameworks and DevOps models, the number of deployments per year will increase while the overall complexity of integrated solutions remain. With an increase in deployments and changes to applications, having a library of documented business scenario tests and the underlying datasets will allow these tests to be leveraged continuously and the quality payback for investing in these tests even more beneficial. When tests can be automated, execution time can be reduced significantly without sacrificing quality for each incremental release.
Conclusion
ETRM systems are some of the most tightly integrated and functionality-rich applications within a business and IT landscape. Developing a risk-informed testing strategy and investing in the documentation and automation of tests will improve the overall quality of the solution and reduce project delivery risk. In the long run, this will also allow for a more stable platform and lower costs to sustain and improve the solution while remaining responsive to continuously evolving demands of trading and risk management.
(Kent Landrum is Managing Director in Opportune LLP’s Process & Technology practice who leads the firm’s Downstream Sector and Steven Bradford is a Managing Director in Opportune LLP’s Process & Technology practice and the views expressed in this article are their own)
