This is Part 2 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure you meet regulatory requirements and industry expectations.

In the previous article, we discussed the following regarding the validation of assessment resources:

• Explanation of assessment validation
• Typical benchmarks used during the validation processes
• Stages of validation (before, during and after the assessment judgements)

In this month’s article, we will explore the regulatory requirements around assessment validation.

Regulatory requirements for conducting validation

According to the Standards of Registered Training Organisations (RTOs) 2015, you are required to implement a quality review process (Clauses 1.8, 1.9, 1.10 and 1.11).

Clause 1.8a requires that the RTO’s assessment systems comply with the assessment requirements of the relevant training packages or VET accredited courses.  

Clause 1.8b requires RTOs to ensure that the evidence gathered is valid (one of the Rules of Evidence) and that assessment processes and outcomes are valid (one of the Principles of Assessment). 

These requirements must be met and demonstrated in all assessment policies, procedures, materials and tools of the RTO. Clause 1.8 primarily relates to the development (or purchase) of the RTO’s assessment resources.

Assessment validation has been strengthened in the Standards for RTOs and the requirement is to:

• Develop and implement a comprehensive plan for ongoing systematic validation of assessment that includes all training products on the RTO’s scope of delivery (Clause 1.9)
• Validate the assessment practices and judgements for each training product at least once every five years with at least 50% of products to be validated within the first three years of each five-year cycle (Clause 1.10)
• Ensure that validation is conducted by one or more suitably qualified persons, who are not directly involved in the delivery and/or assessment of the training product being validated. (Clause 1.11).
• These clauses relate primarily to the actual delivery and outcomes of the RTO’s assessment systems, including the performance of the RTO’s assessors.

Assessment system

Documents required for conducting an effective validation session, in the RTO’s assessment system, includes but is not limited to:

Validation related documents:

• Validation plan
• Validation schedule
• Validation record or validation form
• Validation register
• Validation report form
• Continuous improvement form
• Continuous improvement register
• Pre-assessment validation documents

Assessment resources:

• Unit assessment pack/student pack
• Trainer assessment pack/assessor pack
• Mapping document
• Assessment evidence according to a sample size

Other documents:

• Training and assessment strategy
• Feedback forms
• Unit of competency
• Companion volume/implementation guide
• AQF framework
• ACSF framework

You will be required to evaluate if the assessment resources meet:

• Training package requirements (application, elements and performance criteria, foundation skills, performance evidence, knowledge evidence, assessment conditions)
• Principles of assessment; fairness, flexibility, validity and reliability
• Rules of evidence; valid, sufficient, authentic and current
• The appropriate level of difficulties (AQF Level)
• Provide sufficient and clear instructions
• Record any appropriate adjustments

Who conducts validation?

Validation is a collaborative process. The team must hold collectively:

• Vocational competencies and current industry skills relevant to the assessment being validated
• Current knowledge and skills in vocational teaching and learning
• The TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor).
• Validators can be employees of your RTO, or you can seek external validators.

The trainer and assessor who delivered/assessed the training product being validated:

• Can participate in the validation process as part of a team
• Cannot conduct the validation on his/her own
• Cannot determine the validation outcome for any assessment judgements they made
• Cannot be the lead validator in the assessment team.

It is important to keep the records of all validation activities and validators as auditors might ask for it during audit activities and for managing continuous improvement processes at an RTO.

How is validation different from moderation?

Moderation is a quality control process aimed at bringing assessment judgements into alignment.

Moderation is generally conducted before the finalisation of student results as it ensures the same decisions are applied to all assessment results within the same unit of competency.

The requirement in the Standards to undertake validation of assessment judgements does not affect your RTO’s ability to undertake moderation activities, or any other process aimed at increasing the quality of assessment.

(ASQA, 2018)

(To be continued in the next newsletter)

This is Part 2 continuing from the previous newsletter. As discussed in Part 1 of this article, there are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources, therefore, meet the following criteria:

• The context and conditions of assessment. For example, an assessment tool could be developed to cater for a particular language, literacy and numeracy requirements, the learner’s workplace experience or other learner needs that require reasonable adjustment.
• The context of the assessment may also take into account assessments already completed, and the competencies demonstrated in these assessments. By looking at the context, you can consider the conditions under which evidence for assessment must be gathered.
• All activities are conducted adequately using the required:
  • equipment or material requirements
  • contingencies
  • specifications
  • physical conditions
  • relationships with team members and supervisors
  • relationships with clients/customers
  • timeframes for completion.
• Assessment methods or tasks are suitable to the requirements of the units of competency and students are assessed on the tasks and activities according to the requirements of the training package.
• The language used is simple English
• The evidence required to make a decision of competency is clearly outlined
• The types of activities and tasks student need to perform are clearly outlined
• The level of performance required for each assessment activity is clearly outlined
• Adequate exposure to workplace conditions, including appropriate simulated environments
• Sufficient knowledge-based assessment tasks and activities such as written questions and case studies etc.
• Sufficient practical based assessment tasks and activities such as projects, role plays, workplace tasks and observations etc.
• Assessment resources are error-free and free from any grammar, copyright or plagiarism issues

It is a wise decision to get your training and assessment strategies and resources validated by independent industry experts to get honest feedback and an unbiased opinion.

In this article, we will discuss different phases of validation processes that you should be following in your RTO to ensure you meet regulatory requirements and industry expectations.

Validation of RTO assessment resources

You must validate all your assessment resources to ensure they meet the principles of assessment, rules of evidence, training package requirements, regulatory guidelines and Industry expectations.

Explanation of assessment validation:

Validation is a process of checking that the assessment tools, methods, judgements, evidence and processes to ensure that the training product meets:

• ​Principles of Assessment – i.e. valid, reliable, flexible and fair
• Rules of Evidence – i.e. valid, authentic, current and sufficient
• The judgment made by the trainer/assessor is benchmarked with colleagues or industry experts
• There is sufficient evidence to support the judgment of the trainer/assessor
• Whether the requirements of the Training Package or accredited course have been met.

Typical benchmarks used during the validation process include:

• National training package which are developed by Skills Service Organisations (SSOs)/ Industry Reference Committees (IRCs) and can be found on the training.gov.au website.
• Units of competency which consist of competency standards and need to be unpacked so that those validating the assessments can compare the actual competency against the tools being validated.
• Industry standards and consultation will vary, and these standards form the basis of the skills and knowledge required to perform work roles.
• AQF Guidelines and Framework
• Information provided to candidates, assessors and third parties
• Legislation relevant to the assessment such as privacy, health and safety, anti-discrimination, copyright law and so on.

Validation occurs through different stages:

Stage 1: Validation before assessment judgements i.e. pre-validation of assessment resources

Validation before assessment judgements are made; look at the design of the assessment activities, if it meets the training package requirements, how the instructions for tasks or questions are presented and the benchmarks against the learner performance. This is where the mapping is undertaken. You review the assessment tool templates in detail to ensure they are compliant and meet regulatory standards and Industry requirements.

Stage 2: Validation during assessment

Validation during assessment is looking at the actual benchmarking answers or performance the learner has provided, and making a judgement with another assessor, either together or separately. This often is coordinated within assessors who undertake training and assessment of the same vocational area. This process was also known as moderation and always remember prevention is always better than the cure, therefore, any issues identified at stage 1 or stage 2 should be eliminated as soon as possible and gaps should be filled with gap-analysis to ensure your organisation is bullet-proof.

The requirement in the Standards to undertake validation of assessment judgements (post validation) does not prohibit your RTO from undertaking moderation activities, or any other process aimed at increasing the quality of assessment.

Stage 3: Validation post assessment (i.e. post validation)

Validation post assessment concentrates on the learners’ performance and their responses to questions, the actual assessment decision that was made, the task and processes that align to the assessment, any feedback from students, and the reporting processes.

The purpose of this post-assessment validation is to verify the validity and consistency of assessment decisions to bring assessment judgements and standards into alignment.

It is a process that ensures the same standards are applied to all assessment results within the same Unit(s) of Competency. It is an active process in the sense that adjustments to assessor judgements are made to overcome differences in the difficulty of the tool and/or the severity of judgements. It aims to ensure assessors have a common understanding of the unit requirements

It involves checking that your assessment tools have produced valid, reliable, sufficient, current and authentic evidence, enabling RTO to make reasonable judgements that the training package requirements have been met.

(To be continued in the upcoming blogs) 

It is important to look into ASQAs 2017 report that shows:

• Around 72% of RTOs FAIL audit on Assessment
• Approximately 50% of those FAIL to be able to rectify their assessment tools on resubmission under the OLD audit mode

According to the new audit model:

• There may be NO opportunity to rectify critical non-compliances
• Initial registration clients with critical non-compliances are unlikely to get an opportunity to rectify and potentially would be unable to reapply
• Registered RTOs risk sanctions, conditions, or even worse cancellations for critical non-compliances on the first audit

There are a number of assessment-related issues that may affect your audit outcome. You should ensure your assessment resources meet the following criteria:

• Assessment resources have sufficient and clear information regarding what, when, how, where, why for your assessment template and all assessment tasks and activities. 
• Assessment resources have robust benchmarking and/or trainers guide. 
• Assessment resources are allowing the trainer/assessor to assess the skills and knowledge of students through different assessment tasks over a period of timeto ensure “consistency” and “sufficiency” 
• Each and every question and assessment task has very clear guidelines around what is expected from the students in terms of both “quantity” and “quality” 
• You have customised the off-the-shelf resources according to your RTO needs and requirements and not using them “as-it-is”
• Your assessment resources are written by Industry experts with subject matter experts and are “Industry-relevant” and “current”
• Your assessment resources address all requirements of the training packaging rules
• Your assessment resources have detailed and valid performance checklists/observation checklists for assessing and observing the students before, during and after any skill assessment activity or workplace task 
• Your trainers and assessors gather sufficient, valid evidence for competency assessment
• Your organisation offers appropriate simulated environments for conducting assessments 
• The authenticity of assessment, particularly in distance and online delivery is established and maintained