ATO Practice Statement Law Administration

PS LA 2001/11

SUBJECT: Technical Decisions - Quality Reviews
PURPOSE: To advise the processes to be followed in conducting periodic technical quality reviews of written interpretative decisions, and reporting the results

This version is no longer current. Please follow this link to view the current version.

  • This document has changed over time. View its history.

FOI status: may be released
This Practice Statement is issued under the authority of the Commissioner and must be read in conjunction with Law Administration Practice Statement PS LA 1998/1. It must be followed by Tax officers unless doing so creates unintended consequences. Where this occurs officers must follow their Business Line's escalation process.
TABLE OF CONTENTS
STATEMENT
1
When Technical Quality Reviews are conducted
Major steps in the Technical Quality Review process
EXPLANATION
5
Technical Quality Reviews
Why the Tax Office conducts Technical Quality Reviews
Features of a Technical Quality Review
Methodology
Review panel
Moderation process
Community representative
Decision selection
Populations
All finalised decisions
Code of Settlement Practice Agreements
Penalty decisions
Documentation
Sample size
Content of Technical Quality Reports
TQR Report template
Conformance with mandatory corporate work practices and procedures
TQR Database
Certification - Technical Quality Review report
Post Technical Quality Review actions
Clients
Tax Office staff
Attachment A
Attachment B

STATEMENT

1. Each business line is required to report on the quality of its written interpretative decision making to the Corporate Assurer, Technical Quality, in Law and Practice (L&P).

2. To do this each line conducts a Technical Quality Review (TQR).

When Technical Quality Reviews are conducted

3. TQRs are conducted biannually, for decisions finalised during the periods 1 August to 31 January and 1 February to 31 July. The TQR reports are to be with the Law Infrastructure Branch (LIB) of L&P by 2 April and 2 October respectively.

Major steps in the Technical Quality Review process

4. The following steps are undertaken when conducting a TQR:

LIB will:

Identify the decision populations for review (paragraphs 21 - 29)
Select a valid random sample of decisions from each population (paragraphs 31 - 34)
Prepare the Corporate Assurer's report on technical quality.

Business lines will:

Form the review panel (paragraphs 11 - 15)
Review and rate each decision using the Judgment Model (paragraphs 9 - 10)
Use a moderation process where members of a review panel differ on the rating of a particular decision (paragraphs 16 - 17)
Assess each decision for compliance with a number of processes and procedures (paragraph 37)
Tabulate the results, complete the reporting template (paragraphs 35 - 36), attach the community representative's report (paragraphs 19 & 41), and input the results to the TQR Database (paragraph 39)
Obtain sign off for business line TQR report by an SES officer, and forward the report to LIB (paragraph 40)
Provide constructive timely feedback to every decision-maker whose decision is examined and to other relevant staff (paragraphs 43 - 45).

EXPLANATION

Technical Quality Reviews

5. A TQR is an examination of a random sample of written decisions (eg private rulings) provided to our clients on the interpretation and application of the laws administered by the Commissioner. It is also an examination of some internal Tax Office decisions such as ATO Interpretative Decisions (ATO IDs) to assess their quality.

Why the Tax Office conducts Technical Quality Reviews

6. To comply with the tax laws the community needs to understand them. One way to gain this understanding is through written advice provided by the Tax Office. We aim to provide advice that is accurate, consistent, relevant and clearly explained.

7. We need to know how well we are doing this. TQR is one means of assessing our performance.

8. TQR is also necessary to:

Identify systemic issues that may need to be addressed to continue to improve the quality of decisions, eg Information Technology systems, work practices and staff skilling needs
Provide input into the determination of remuneration for Tax Office staff, and
Assess the level of compliance with mandatory work practices and procedures.

Features of a Technical Quality Review

Methodology

9. Each business line is to use the Judgment Model in conducting TQRs (a link to this document is available in the Other References section at the conclusion of this practice statement).

10. Each decision, other than Code of Settlement Practice Agreements (Settlements), is to be rated 'A', 'B', 'C' or Fail, (ie 'D' or 'E') in accordance with the Judgment Model. Decisions selected for review are to be rated as a 'Fail' if any of the following apply:

Failure to understand clients' questions
Failure to accurately identify and/or address all issues
Incorrect decision
Insufficient evidence (eg lack of documentation) in the decision report to arrive at the correct decision
Insufficient evidence from the taxpayer to support assumptions made, or
Incorrect application of the penalty provisions.

A modified Judgment Model is used in reviewing Settlements. Results are rated either as 'Pass' or 'Fail'.

Review panel

11. A review panel is to consist of at least four members, comprising:

one person external to the Tax Office (known as the community representative)
one LIB representative (if none available from LIB then one person from another business line with TQR experience can be used as a substitute) (paragraph 12), and
two people from the business line undertaking the review. There is an expectation that the business line's senior technical staff will actively participate in the review process.

12. A LIB representative will participate in each business line's TQR process. This person's role is to contribute to the corporate consistency of the process, provide process guidance, facilitate moderation where necessary, and to provide a conduit for feedback to enable further improvement of the process. If a LIB person is unavailable a person from another business line must participate.

13. A business line may decide to have more than one panel operating concurrently to complete a review. If so, the additional panel(s) may consist entirely of people from the business line undertaking the review. However, the LIB and community representatives must rotate between the panels.

14. Members of review panels are to possess appropriate technical expertise and security clearance relevant to the decisions under review. However, where a panel's expertise does not cover a particular matter, or it cannot reach a majority view on a rating for a decision, the panel is to seek specialist advice, for example, from a Centre of Expertise.

15. All reviewers are to have a sound understanding of the use of the Judgment Model. Business lines are to ensure continuity in membership of the review panels by including some members who participated in the previous review.

Moderation process

16. If a panel cannot reach agreement on a rating, the decision is to be moderated by members of another panel from that particular business line. The moderation process enables a final rating to be determined.

17. Where the moderation process has been used, the number of decisions moderated and the outcomes must be included in the business line TQR report.

Community representative

18. At least one member of the review panel is to be from outside the Tax Office. The inclusion of external representation adds to the credibility and integrity of the results. This is enhanced through rotation of external members. Accordingly, a community representative must not serve on more than two consecutive review panels within the same business line.

19. Each community representative is to prepare a report, which is to be attached to the business line TQR report.

20. The community representative's report is to:

include the representative's name and qualifications/expertise
comment on the accuracy of the results including areas of disagreement
suggest areas for improvement, and processes to achieve improvements
make observations on the TQR process, including whether it was sufficiently independent from the original decision making process.

These requirements are to be reflected in the contract between the community representative and the Tax Office.

Decision Selection

Populations

21. LIB, with the assistance of the business lines, is to determine the following decision populations for each business line for the review period:

All finalised decisions
Settlements, and
Penalty decisions.

All finalised decisions

22. The types of decisions to be included in the 'All finalised' population for each business line are: all written interpretative decisions such as private rulings, administratively binding advice, Class Rulings, Product Rulings, disputes, litigation, interpretative decisions arising from active compliance activities and other interpretative advice.

23. For Centres of Expertise, all published ATO IDs are also to be included in the 'All finalised decisions' population.

Code of Settlement Practice Agreements

24. The decisions to be included in the Settlements population are all agreements which have been finalised in accordance with the Code of Settlement Practice (a link to this document is available in the Other References section at the conclusion of this practice statement).

25. In exceptional circumstances it may be necessary to exclude a settlement agreement from TQR examination (eg the settlement contains highly sensitive material).

26. Any such exclusion is to be approved, in writing, by the relevant National Program Manager. The approved statement of reasons for exclusion is to be attached to the business line report and tabled as part of the TQR Corporate Assurer's report.

27. A settlement agreement will not be excluded merely because it concerns a High Wealth Individual or a Restricted Access Client (refer paragraph 14).

Penalty decisions

28. A sample of penalty decisions is to be selected for review. This gives effect to recommendations outlined in the Australian National Audit Office report No.31 Administration of Tax Penalties. LIB is to forward the results of the penalty decisions to Compliance Penalty and Interest Team for review and follow up action.

29. The population is to consist of all decisions finalised, where penalties should have been considered. For example, the following are included in the population: audits where adjustments have been made, objections against decisions where penalties have been applied, decisions which apply or remit Failure to Lodge Penalty, Tax Shortfall Penalty, Shortfall Interest Charge and the General Interest Charge.

Documentation

30. For each decision the review panel is to have access to all documentation necessary to review the decision. This is vital for an accurate assessment to occur under the Judgment Model.

Sample size

31. LIB, with the assistance of the business line, selects a valid random sample of decisions from each of the populations outlined above.

32. The number of decisions to be reviewed for each population is determined by reference to Attachment A.

33. The sample is to be chosen using either a computer based random number generator or a book of random numbers.

34. A decision is not to be rejected as part of the sample simply because documentation is incomplete. These decisions are to be reviewed even though they may fail.

Content of Technical Quality Reports

TQR report template

35. Each business line is to complete a TQR report template and send it to LIB by the due date (refer to paragraph 3).

36. The TQR report is to include:

The 'A' and 'pass' rating percentages for the 'All Finalised' decision sample
The 'A' and 'pass' rating percentages for the 'Penalty' decisions sample
A comparison of these percentages with those awarded in the previous period, and commentary on trends overall
An explanation where there is a variation of 5 or more percentage points from the previous period's results
The main reasons for the 'Fail' ratings (ie 'D' and 'E' ratings)
The moderation process used where there was disagreement on a particular rating including the number of decisions moderated and the final outcomes
Proposed mitigation strategies to address areas of concern
The effectiveness of previous mitigation strategies where undertaken
An explanation of any systemic issues affecting the quality of interpretative decision making
Commentary on the LIB and community representative reports, and
Where applicable, separate commentary on, and ratings for, Code of Settlement Practice Agreements.

Conformance with mandatory corporate work practices and procedures

37. Each business line's TQR report must include a report on conformance with the list of Law Administration Practice Statements at Attachment B.

TQR Database

38. LIB is to populate the TQR database with the samples to be reviewed by the sixth working day after the end of the review period.

39. Each business line is to update the TQR database with the results of their work items reviewed within two months of the TQR period ending (ie by 31 March and 30 September).

Certification - Technical Quality Review report

40. The business line TQR report is to be signed by the Senior Executive Service officer responsible for management of interpretative advice within the business line. The officer is to state whether, in their judgment, there has been an increase or decrease in the quality of written decisions when compared to the previous reporting period.

41. The report of the community representative must be attached to the business line TQR report.

Post Technical Quality Review actions

Clients

42. During a TQR, interpretative errors may be detected in advice provided to clients. Where the tax laws permit, action is to be taken to rectify such errors.

Tax Office staff

43. The results of the TQR need to be promulgated widely in each business line, (eg Connect or Intranet business line site), and must include details of mitigation strategies to rectify any deficiencies identified.

44. Constructive timely feedback is to be provided to every decision-maker whose decision is examined and to other relevant staff, including authorising officers. To ensure that the feedback has current effect, it is to be provided within six weeks of finalisation of the relevant business line report. To acknowledge and reinforce proficient and excellent work, feedback is to be given to those decision-makers whose cases are rated 'Pass' or 'A' as well as to those whose cases are rated 'fail'. A random sample of cases is to be spot checked by corporate assurers subsequent to each TQR process to enable assurance that feedback has been given. Individual feedback must include:

Rating and TQR panel's comments, including the reasons for the rating in terms of the Judgment Model
Results of conformance checks
Any particular corrective action required.

45. Business lines must provide an undertaking in their TQR report that all necessary feedback will be provided by 14 May and 13 November.

46. The Integrated Quality Framework (IQF) will incrementally replace the TQR process as the Tax Office's corporate technical quality improvement and assurance process. Implementation of the IQF began in 2008; and completion during 2009 is expected. To reduce duplication of effort during the transition period: for business lines and products in respect of which the IQF has been implemented, individual decisions will be subject to the IQF only. Stipulated TQR sample sizes will be varied to account for business lines and products covered by the IQF.

ATTACHMENT A

SAMPLE SIZE FOR POPULATIONS
No of Decisions Sample No of Decisions Sample No of Decisions Sample
1 1 44-45 31 156-162 61
2 2 46-47 32 163-169 62
3 3 48-49 33 170-177 63
4 4 50-52 34 178-185 64
5 5 53-54 35 186-194 65
6 6 55-57 36 195-203 66
7 7 58-59 37 204-213 67
8 8 60-62 38 214-223 68
9 9 63-64 39 224-234 69
10-11 10 65-67 40 235-246 70
12 11 68-70 41 247-259 71
13 12 71-73 42 260-273 72
14 13 74-76 43* 274-288 73
15-16 14 77-80 44 289-305 74
17 15 81-83 45 306-322 75
18-19 16 84-86 46 323-342 76
20 17 87-90 47 343-363 77
21-22 18 91-94 48 364-386 78
23 19 95-98 49 387-412 79
24-25 20 99-102 50 413-441 80
26 21 103-106 51 442-473 81
27-28 22 107-111 52 474-510 82
29-30 23 112-115 53 511-551 83
31 24 116-120 54 552-599 84
32-33 25 121-125 55 600-654 85
34-35 26 126-131 56 655-718 86
36-37 27 132-136 57 719-794 87
38-39 28 137-142 58 795-886 88
40-41 29 143-148 59 887-999 89
42-43 30 149-155 60 =>1000** 97

* All Business lines will use this table, however, given the number and complexity of issues and the voluminous documentation often involved with a Large Business & International decision, Large Business & International will have a sample size with an upper limit of 43.

** Once the target population reaches 1000 clients, the incremental increase in sample size per large increase in target population is negligible and approximates the sample size for a large or unknown population (ie 97 decisions).

The above table produces a result where we can be 95% confident that there is a margin of error of 10% or less (for Large Business & International, when the population is greater than 76, a 95% confidence level up to a 15% margin of error results).

ATTACHMENT B

LIST OF CONFORMANCE CHECK QUESTIONS
1.
1. Have the requirements of the following Law Administration Practice Statements been met for each decision reviewed?
PS LA 2000/9 Remission of penalties under the new tax system
PS LA 2001/8 ATO Interpretative Decisions
PS LA 2002/8 Administration of penalties under the new tax system
PS LA 2002/13 Authorisation of written binding advice
PS LA 2002/16 Mandatory use of Information Technology systems for interpretative work - inclusion in performance agreements
PS LA 2003/3 Precedential ATO View
PS LA 2003/9 The Online Resource Centre for Law Administration (ORCLA)
PS LA 2004/4 Referral of issues to Centres of Expertise for the creation of precedential ATO view
PS LA 2004/5 Administration of shortfall penalties under the new tax system
PS LA 2006/2 Administration of shortfall penalty for false or misleading statement
PS LA 2006/8 Remission of shortfall interest charge and general interest charge for shortfall periods
PS LA 2008/3 Provision of advice by the Australian Taxation Office
PS LA 2008/4 Publication of written binding advice
PS LA 2008/5 Written binding advice - requests for further information, notification of assumptions and intended use of information from sources other than the applicant
2.
2. Have the requirements of the Code of Settlement Practice been met for each settlement agreement reviewed?
3.
3. Have the correct settlement amounts been entered into the ATO Settlement Register?

Date of Issue: 31 August 2001

Date of Effect: 31 August 2001

File 98/2810-9;
98/2870-2;
98/9287-7;
99/7541-1;
T2000/00213;
T2000/011869;
T2000/018353;
T2001/6955;
T2001/14475

Related Practice Statements:
PS LA 2000/9
PS LA 2001/8
PS LA 2002/8
PS LA 2002/13
PS LA 2002/16
PS LA 2003/3
PS LA 2003/9
PS LA 2004/4
PS LA 2004/5
PS LA 2006/2
PS LA 2006/8
PS LA 2008/3
PS LA 2008/4
PS LA 2008/5

Other References:
Code of Settlement Practice
ATO TQR Judgment Model (internal link only)

Subject References:
ATO Interpretative Decisions
Code of Settlement Practice
Review panel
Sampling
Judgment Model
Technical Quality Reviews
Written interpretative decisions

Authorised by:
Damien Browne
Assistant Commissioner
Law Infrastructure Branch

Amendment history 28 February 2008
Update references to PS LA 2008/3, 2008/4 and 2008/5; update references to Law Infrastructure Branch

8 January 2009
Paragraph 46 inserted

PS LA 2001/11 history
  Date: Version:
You are here 31 August 2001 Original statement
  13 August 2009 Archived