2016 Audit Quality Indicators Pilot Project Interim Report Summary In 2016, CPAB launched an Audit Quality Indicators (AQI) Pilot Project (Pilot) with six Canadian audit committees. Our observations to date suggest to us that AQIs could have significant potential to positively impact audit quality. We encourage audit committees, management and audit firms to continue to explore how AQIs can be integrated into the audit process. For details on how to participate in our 2017 Pilot, please see page 3.
THE BENEFITS Previous literature about AQIs identified a number of benefits including more efficient and effective interactions between the audit committee and the auditor, and assistance in auditor evaluation. CPAB’s Pilot has highlighted two more: 1. 2.
A better understanding of management, audit committee and external auditor expectations of their roles in the audit and responsibilities related to audit quality, particularly as a result of discussions about how to evaluate AQIs and what benchmarks should be set. An improvement in the knowledge of, and engagement in, the audit and audit quality by audit committee members who don’t have an audit background.
These benefits enhance audit quality in two ways: 1. 2.
An enriched understanding of expectations among stakeholders improves coordination and cooperation in the execution of an audit. A more informed and engaged audit committee results in improved oversight over the external audit process.
Background The intent of CPAB’s Pilot was to obtain practical, Canadian feedback regarding the usefulness of AQIs. We are part way through the Pilot. All participants have selected their AQIs and most have received an interim report on some indicators, as applicable. In November 2016, CPAB held a Roundtable with participants to obtain their initial observations. The Roundtable included audit committee chairs, management and lead audit partners from participating organizations. This document provides a summary of learnings to date on the Pilot and includes highlights of the Roundtable discussion. For a more detailed report, please see 2016 Audit Quality Indicators Pilot Project: Interim Report
THE CHALLENGES
2016 AQI Pilot Participants
The most significant challenge identified by participants was the development of evaluation criteria for AQIs. Others include the fact that key contributing factors to audit quality, such as how an audit firm and engagement team are able to deal with the unexpected, are not easily quantifiable; reporting may require changes in systems, processes and attitudes at the audit firm level which takes time; and context and background is important in interpreting AQIs which is a skill audit committees will have to build.
Company
Audit Committee Chair
Air Canada
Chris Clark
Intact Financial Corporation
Eileen Mercier
Magna International Inc.
Lawrence Worrall
Royal Bank of Canada
David Denison
Sun Life Financial Inc.
William Anderson
Telus Corporation
William MacKinnon
Increased use of, and experimentation with, AQIs could be helpful in building capacity to develop solutions to these challenges. CPAB EXCHANGE
2016 AQI Pilot Project: Interim Report Summary
1
KEY INSIGHTS Project management and audit quality
Definitions and evaluation
Some participants developed indicators related to tracking project management milestones or phases of the audit. These indicators typically held the audit firm accountable for meeting certain milestones by a specific date. Benefits resulting from the inclusion of such an AQI included increased focus by the audit firm on project management, and improved coordination among management, the audit firm, and the audit committee throughout the audit process.
Participants identified the development of evaluative criteria as their most significant challenge. Possibilities raised in discussion included comparisons to prior year(s), a budget, an objective target, a benchmark, or to an industry standard. Most participants evaluated AQIs by comparing to budget or to a pre-agreed upon target. Participants agreed that historical trending would be useful in evaluating a current year AQI in the future.
Management’s role in audit quality Some participants developed AQIs to measure management’s role in audit quality. This provided a broader perspective to the audit committee highlighting the contribution of management to the successful completion of a quality audit.
As a result of including a management AQI in their AQI profile, one audit committee discovered that over 40 per cent of management’s deliverables were provided an average of five days late to the auditor. This insight resulted in actions from management that positively impacted audit quality.
SPECIFIC AUDIT QUALITY INDICATORS Participants identified a number of uses for AQIs and their selection of specific indicators was driven by what they wanted to get out of the AQI process. The list below indicates some of the objectives participants had in mind when selecting their AQIs: • General oversight of the audit, including project management and monitoring of key audit risks. • External auditor evaluation in terms of both audit quality and client service. • Monitoring and managing the added value provided by the auditor.
CPAB EXCHANGE
2016 AQI Pilot Project: Interim Report Summary
Diversity in perceived usefulness Roundtable participants rated each of the indicators on a five point scale (5 being extremely useful and 1 being not useful at all), regardless of whether they had selected the indicator themselves. Most indicators received a mixed reaction, with participants rating the indicator from mildly useful to extremely useful. This illustrates the diverse perspectives of participants and their differing needs and goals.
Additionally, in determining which AQIs to select, participants were encouraged to consider a number of factors, including: • The unique nature of their business. • Specific areas of significant audit risk. • Areas of the audit over which they would like more transparency or information. • Factors that are most meaningful to them when evaluating their external auditor. • Themes noted in the reports of external audit inspections including CPAB’s latest Public Report on Inspections. • AQIs that their external auditor is already tracking or considering for their own internal management purposes. • Ability of their external auditor to provide them with the information.
2
The chosen objective(s), along with consideration of the factors listed above, influenced the selection of the specific AQIs. As a result, there was significant variation in the types of AQIs selected. Figure 1 outlines some of the AQIs selected.
Example AQIs selected by Pilot participants
Figure 1
Category
Example AQIs
Audit Execution
• • • • •
Firm Level Metrics
• Audit firm quality review results (internal and/or external) • Training and professional development • Independence
Management Indicators
• •
Engagement Team
• Staffing leverage • Experience of engagement team • Workload • Use of specialists • Engagement team turnover
Client Service Indicators
• Survey of audit committee members • Sharing of thought leadership / ideas / best practices with management and/or the audit committee
Audit hours by phase Audit hours by risk Amount of audit work performed by centralized service centres Achievement of significant milestones in the audit Frequency and areas of challenge
Achievement of timing of agreed upon deliverables from management to the auditor Timely reporting of internal control deficiencies identified by management to the audit committee and/or external auditor
WHAT’S NEXT? AQIs can be a useful tool for audit committees to broaden and deepen the dialogue around audit quality. CPAB supports increased awareness, discussion, and collaboration around AQIs to develop preferred practices and inform the path forward on this issue.
To help advance this effort, CPAB will continue the AQI Pilot Project in 2017 and we encourage Canadian reporting issuers to participate. If you are interested, please contact your audit firm or Lindsay Colley, Director, CPAB at
[email protected].
Learn More Visit us at www.cpab-ccrc.ca and join our mailing list. Follow us on Twitter — @CPAB-CCRC This publication is not, and should not be construed as, legal, accounting, auditing or any other type of professional advice or service. Subject to CPAB’s Copyright, this publication may be shared in whole, without further permission from CPAB, provided no changes or modifications have been made and CPAB is identified as the source. © CANADIAN PUBLIC ACCOUNTABILITY BOARD, 2017. ALL RIGHTS RESERVED www.cpab-ccrc.ca / Email:
[email protected]
CPAB EXCHANGE
2016 AQI Pilot Project: Interim Report Summary
3
