The RADAR tool for Enablers – 1. Leadership (1c. Leaders engage with customers, partners and representatives of society) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
192
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
Contents
Page
A3.3
Strategic direction performance levels (SDPL) – evidence 2
123
A3.4
Strategic direction performance levels (SDPL) – evidence 3
126
Appendix 4
Self evaluation - Statutory compliance outcomes
132
Appendix 5
Benchmarking
137
Appendix 6
Stakeholder satisfaction levels
138
A6.1
Stakeholder satisfaction levels - The Awarding Organisation
138
A6.2
Stakeholder satisfaction levels - Quality Management
144
A6.3
Stakeholder satisfaction levels – Qualification Provision
145
A6.4
Stakeholder satisfaction levels – Awarding Services
149
A6.5
Stakeholder satisfaction levels – Awarding Systems
155
Appendix 7
Recognised centre satisfaction levels on external verification activity
159
Appendix 8
Learner satisfaction levels
163
Appendix 9
1st4sport employee satisfaction levels
165
Appendix 10
External verifier satisfaction levels on training and CPD provision
169
Appendix 11
Qualification specific performance statistics
170
Appendix 12
RADAR scoring matrix outcomes
189
Product ref. SAR 2013 - Executive Summary
III
The RADAR tool for Enablers – 1. Leadership (1d. Leaders reinforce a culture of excellence with the organisation‘s people ) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
193
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 1. Leadership (1e. Leaders ensure that the organisation is flexible and manages change effectively) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
194
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 2. Strategy (2a. Strategy is based on understanding the needs and expectations both stakeholders and the external environment ecological sustainability) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
195
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 2. Strategy (2b. Strategy is based on understanding internal performance and capabilities) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
196
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 2. Strategy (2c. Strategy and supporting policies are developed, reviewed and updated to ensure economic, societal and ecological sustainability) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
197
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
Improvement plan As a result of the self-assessment process several strengths and areas for improvement were identified in our effort to achieve our business aims and continuous development. The identification of areas for improvement led to the development of an improvement plan following the EFQM structure. To make the implementation of the improvement plan as straightforward as possible, a detailed schedule is produced to enable teams and individuals to work towards the completion of each action. Significant delays in Ofqual publishing of the annual market report and subsequent comparison to our performance data, which is key to the conduct of self-assessment, has considerably affected the implementation of our improvement plan schedule. Due to these limitations, the final version of the report this year is not published until early October; consequently allowing only three months of implementation, which is insufficient. Hence, a two year implementation timetable has been set for the achievement of the plan. This in turn, makes the implementation and related timescales more realistic in ensuring actions based on the priority level are full met. Key points of the improvement plan for 2013-2014 are summarised below, representing our high priority actions to ensure:
increased leaders’ knowledge related to the EFQM criteria/quality standards and to promote the culture of excellence. design and deployment of a web based mechanism to ensure methodical management of change, to improve communication issues and to provide a systematic reporting of decisions across the organisational structure. review and reallocation of human resources across the qualification development team to enable business and provision planning to be successfully implemented and improve our ability to respond to high volume qualification opportunities. addressing of areas of employee dissatisfaction and conduct of employee needs analysis related to their role to create an inspirational/creative working environment to improve organisational culture. detailed evaluation of our technology portfolio and resultant future recommendations and a business case for any developments with the aim of ensuring improved services to stakehold ers and continuous growth. redevelopment of the external verifier role to include the four fundamental aspects of compliance, communication, customer care and consumer growth. deployment of an effective mechanism to audit continuous monitoring of quality standards and regulations across our operations.
Methodical implementation of our improvement plan and strategic direction in addition to the continuous review of industry trends will help maintain a sustainable advantage and further improve competitive positioning within the industry. We endeavour to respond to the demands of the education industry, are committed to achieving maximum stakeholder satisfaction levels and ultimately aspire to continue to meet our mission.
Product ref. SAR 2013 - Executive Summary
IX
The RADAR tool for Enablers – 2. Strategy (2d. Strategy and supporting policies are communicated and deployed through plans, processes and objectives) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
198
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 3. People (3a. Peoples‘ plans support the organisations strategy) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
199
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 3. People (3b. Peoples‘ knowledge and capabilities are developed ) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
200
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 3. People (3c. People are aligned, involved and empowered) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
201
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
1.5
The self-assessment process
The process of self-assessment is continuous in order to obtain improved results on an annual basis. It is considered as a holistic perspective, which delivers greater results and full potential. To support this, self-assessment is undertaken in line with a defined process to ensure the effectiveness and validity of results, as indicated in the process diagram.
The self-assessment process
Data generated
Establish and communicate plan for selfassessment
Plan produced and communicated
Quality Manager
Liaise with relevant personnel for the collation of performance measurement da ta
Data collected
Quality Manager
Evaluate and assess performance against EFQM criteria
Evaluation and assessment conducted
Quality Manager
Organise representation of performance results Quality Manager
Performance results organised
Produce annual self-assessment report using the established structure
Report produced
Quality Manager
Forward selfassessment report to Senior Management Team for feedback
Feedback obtained
Quality Manager
Quality Manager
Diagram 1.1 Self-assessment process
Product ref. SAR 2013
Review feedback and amend report (where required)
4
Feedback reviewed
Publish selfassessment report
Quality Manager
Self-assessment undertaken, outcomes published
The RADAR tool for Enablers – 3. People (3d. People communicate effectively throughout the organisation) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinemen Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
202
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 3. People (3e. People are rewarded, recognised and cared for) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
203
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 4. Partnerships and Resources (4a. Partners and suppliers are managed for sustainable benefit) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
204
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 4. Partnerships and Resources (4b. Finances are managed to secure sustained success) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
205
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 4. Partnerships and Resources (4c. Buildings, equipment, materials and natural resources are managed in a sustainable way) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
206
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
EFQM enablers criteria and sub-criteria (1st4sport quality standards) 1
Leadership
1a
Leaders develop the mission, vision, values, and ethics and act as role models
1b
Leaders define, monitor, review and drive the improvement of the organisation’s management system and performance
1c
Leaders engage with customers, partners and representatives of society
1d
Leaders reinforce a culture of excellence with the organisation’s people
1e
Leaders ensure that the organisation is flexible and manages change effectively
2
Strategy
2a
Strategy is based on understanding the needs and expectations both stakeholders and the external environment ecological sustainability
2b
Strategy is based on understanding internal performance and capabilities
2c
Strategy and supporting policies are developed, reviewed and updated to ensure economic, societal and ecological sustainability
2d
Strategy and supporting policies are communicated and deployed through plans, processes and objectives
3
People
3a
Peoples’ plans support the organisations strategy
3b
Peoples’ knowledge and capabilities are developed
3c
People are aligned, involved and empowered
3d
People communicate effectively throughout the organisation
3e
People are rewarded, recognised and cared for
4
Partnerships and Resources
4a
Partners and suppliers are managed for sustainable benefit
4b
Finances are managed to secure sustained success
4c
Buildings, equipment, materials and natural resources are managed in a sustainable way
4d
Technology is managed to support the delivery of strategy
4e
Information and knowledge are managed to support effective decision making and to build the organisational capability
5
Processes, Products and Services
5a
Processes are designed and managed to optimise stakeholder value
5b
Products and services are developed to create optimum value for customers
5c
Products and services are effectively promoted and marketed
5d
Products and services are produced, delivered and managed
5e
Customer relationships are managed and enhanced
Product ref. SAR 2013
10
The RADAR tool for Enablers – 4. Partnerships and Resources (4d. Technology is managed to support the delivery of strategy) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
207
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 4. Partnerships and Resources (4e. Information and knowledge are managed to support effective decision making and to build the organisational capability) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
208
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 5. Processes, Products and Services (5a. Processes are designed and managed to optimise stakeholder value) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
209
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 5. Processes, Products and Services (5b. Products and services are developed to create optimum value for customers) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
210
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 5. Processes, Products and Services (5c. Products and services are effectively promoted and marketed) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
211
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
2.3.1
Selected approaches and rationale
Critical factors with regards to the potential success of each approach to self-assessment via the EFQM model were considered prior to any decision-making. This was primarily based on the resource availability/requirements and the accuracy of the self-assessment approach. Successful review regarding these criteria can assist the identification of the deliverables of each approach. In terms of resource requirements, factors that may define the benefits of the approach have been analysed and are identified as below:
time restrictions availability/compatibility of information systems and data finance and related costs training of allocated personnel corporate culture and current quality management/maturity level.
Selected approaches must be unique to meet the specific requirements within the several boundaries of the conduct of self-assessment. Therefore, dependent upon the maturity and effort level of the organisation there are respective approaches highlighted within table 2.3 that may be appropriate.
Mature organisation
Developing organisation
Starting on excellence journey
appropriate questionnaire
questionnaire and workshop matrix and workshop elementary questionnaire standard matrix
Low effort
pro forma supported by peer variation pro forma facilitated workshop standard questionnaire
Medium effort
award simulation
pilot award simulation pro forma and workshop very detailed questionnaire or tailored matrix
High effort
Table 2.3 The maturity effort chart (Centre for Integral Excellence, 2005) The benefits of using the pro forma, workshop and award simulation approach are the accuracy in score and clear identification of the strengths and areas for improvement. However, the accuracy lies in training and active involvement of appropriate personnel. These approaches are used by mature organisations and those making a high effort in reaching the award application score. As we are now a developing organisation, after implementing the EFQM for the last six years, a decision was made to expand our ‘high effort’ and use the workshop as an additional approach to our a tailored matrix.
Product ref. SAR 2013
16
The RADAR tool for Enablers – 5. Processes, Products and Services (5d. Products and services are produced, delivered and managed) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
212
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Enablers – 5. Processes, Products and Services (5e. Customer relationships are managed and enhanced) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
213
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Results – 6. Customer Results (6a. Perceptions) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
Product ref. SAR 2013
5
10
15
214
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Results – 6. Customer Results (6b. Performance Indicators) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
Product ref. SAR 2013
5
10
15
215
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Results – 7. People Results (7a. Perceptions) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
Product ref. SAR 2013
5
10
15
216
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
2.6.1
Regulators annual market reports review
The information presented on both regulators’ reports and reviews is taken into consideration in establishing our future plan ning and related actions to maintain our competitive positioning in both markets. Comparisons of the presented benchmarks in the industry are made against our performance to identify strengths and areas for improvement. The Ofqual Annual Qualifications Market Report (September 2013) provides an overview of the qualifications market in England, a single reference point and factual information about participants in the regulated qualifications market. Not only does it provide a picture of the qualifications market based on demand and supply, it also establishes a benchmark against which comparisons across awarding organisations can be made and key projections for the future in line with the sector reform. Similarly, the SQA Accreditation Annual Review (2012/2013) focuses on the qualification market trends in Scotland with an overview of the market size based on the number of qualifications and achievements and awarding bodies’ compliance levels in line with SQA’s auditing activity. Outcomes from the review of the qualifications market report are presented in Appendix 1. 2.6.2
1st4sport performance statistics and qualification specific performance statistics
Key 1st4sport performance and qualification specific performance statistics were collected via standard IT based performance measurement methods over a period of time. Substantial developments and enhancements in the performance measurement tools improved data collection and subsequent comparisons of our performance over four years to determine sustainability. This was particularly beneficial as it ensures that trends are identified and validated using available evidence and accurate data, especially when the results are based on factual data rather than perceptions. 1st4sport performance data includes results related to the supply of our qualifications and related demand as well as learner participation trends. The majority of statistics present results from January 2012 to December 2012, except a few cases where data was only available for the financial year and as such it is clearly indicated. 1st4sport performance statistics indicate achievement rates against set targets and overall strategy, providing an insight into progress made. Hence, the representation of those results is critical as it shows our trends over the last year, where data is available. Key trends in the demand for the last six years and detailed statistics on qualification demand were obtained, however comparisons per qualification were only made between 2011-2012 for the purpose of this report. It must be noted that enhancements made in our IT systems resulted in a slight variation in figures for specific areas due to altered data collection methods in previous years. Although minor, where these discrepancies occurred clarification is provided to explain the variation in the statistics. Outcomes from the review of the 1st4sport statistics are presented in Appendix 2. Detailed performance of our qualifications for 2012 is presented in Appendix 11.
Product ref. SAR 2013
22
The RADAR tool for Results – 7. People Results (7b. Performance Indicators) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
Product ref. SAR 2013
5
10
15
217
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Results – 8. Society Results (8a. Perceptions) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
Product ref. SAR 2013
5
10
15
218
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Results – 8. Society Results (8b. Performance Indicators) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
Product ref. SAR 2013
5
10
15
219
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Results – 9. Business results (9a. Business Outcomes) Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
Product ref. SAR 2013
5
10
15
220
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The highest percentage rate (19%) and total count of responses (309) was recorded in this year’s survey. The percentage rate received against the potential responses based on the number of the population contacted is not particular high, however considering the fact that completion of the survey is optional, a positive trend is evident. One of the reasons for this could be the increased familiarity with IT systems, making it more likely for people to respond to online surveys. The responses obtained from the survey contribute significantly in the measurement of our performance and help to determine the rationale for variations in results, with the respective use of qualitative and quantitative results. Outcomes from the stakeholder satisfaction survey are presented in Appendix 6. 2.6.7
Recognised centre satisfaction questionnaire
The Centre Feedback Form is an important tool used on a regular basis as a standard performance measurement mechanism to determine recognised centre satisfaction levels. Centre feedback is collected via Athena, which makes the submission and subsequent analysis easier and quicker. Although centre feedback is collected via the annual self-assessment survey the feedback form in Athena is an effective tool that is used systematically to collate feedback. Making the completion of the Centre Feedback Form optional for recognised centres has had a major impact on the number of responses received in 2012, the lowest in the last 5 years. However the quality and significance of the feedback has somewhat improved. Key changes to our IT systems, business processes and subsequently the use and content of the Centre Feedback Form have been made to ensure that it is no longer used to collate feedback solely on the conduct of the verification activity but also co vers the following types of feedback:
EV performance verification outcomes 1st4sport services access arrangements qualification(s) general feedback.
Recognised centres confirm their satisfaction levels by selecting the following rate:
Very satisfied – expectations exceeded Satisfied – expectations met Not satisfied – Expectations not met
The form consists of two sections ; the feedback section completed by the centre and 1st4sport’s response section. Qualitative and quantitative data is generated and results are reviewed on a regular basis and used to produce descriptive statistics. This enables comparisons to be made annually to identify trends and areas where improvement has been achieved. Recognised centre satisfaction levels on external verification activity are presented in Appendix 7 .
Product ref. SAR 2013
27
Contents 3
Page Results
33
3.1
Revised weighting between EFQM criteria
33
3.2
RADAR scoring outcomes
35
3.3
EFQM self-analysis matrix responses
40
3.4
EFQM self-analysis matrix score
44
Discussion
46
4.1
Analysis and representation of the EFQM scoring results
46
4.2
Improvement plan 2012
55
4.3
Limitations of the self-assessment process and future recommendations (reflective practice)
59
4.4
Conclusion
61
Education industry research and key trends (Ofqual Report and SQA Accreditation Annual Review)
62
Ofqual Annual Qualification Market Report
62
A1.1
Demand for qualifications
64
A1.2
Supply of regulated qualifications
73
AII
SQA Accreditation Annual Review
93
Appendix 2
1st4sport performance statistics
99
Appendix 3
Strategic direction achievement rates
114
A3.1
Strategic direction achievement results
114
A3.2
Strategic direction performance levels (SDPL) – evidence 1
117
4
Appendix 1 AI
Product ref. SAR 2013 - Executive Summary
II
I
Self-assessment Report - Executive Summary
Rationale 1st4sport Qualifications is an awarding organisation, recognised and regulated by the Office of the Qualifications and Examinations Regulator (Ofqual) and the Scottish Qualifications Authority (SQA), created with the aim of offering vocational qualifications in the active leisure, learning and well-being sector. We work in partnership with a variety of organisations including Governing Bodies of Sport (GBS and other organisations in the development of our qualifications. As an Ofqual and SQA recognised awarding organisation, we are required to guarantee thorough monitoring and reporting to the regulators our compliance and quality levels. This is achieved via an established Performance Management System (PMS), which ensures that our compliance is evaluated and our performance is monitored, managed and improved across all operations. The primary goal of the PMS is to effectively evaluate the implementation of our quality standards, established in line with the revised European Foundation for Quality Management (EFQM) Excellence Model. In monitoring the implementation of our quality standards, the effectiveness of our mission and our strategic direction (strategic objectives, targets, policy, procedure and process) are all reviewed and our compliance with the regulations is managed. Self-assessment is the integral component of the PMS, an effective technique, which contributes to our systematic continuous improvement and indicates the current level of our performance.
The self-assessment process and methodology Our quality standards are established in line with the revised EFQM model (figure I). To ensure consistent, valid and relevant performance results, which relate directly to the implementation of the quality standards, the EFQM model is also used to undertake selfassessment, ensuring it fits our corporate culture, needs and encourages organisational learning. The conduct of the self-assessment is based on:
The Fundamental Concepts of Excellence The EFQM Excellence Model RADAR logic.
The self-assessment report is published annually and encompasses outcomes from the self assessment process. The structure of the selfassessment report is based on the EFQM model structure, which facilitates an integrated approach for organisational excellence. As a result of the self-assessment process our strengths and several areas for improvement have been identified. All data was evaluated via the RADAR assessment and management tool. This information was triangulated against the self-analysis matrix to provide an indication of our global performance levels with an emphasis on strategic achievement rate and deployment of quality standards to ensure compliance with regulations.
Product ref. SAR 2013 - Executive Summary
IV
Over the last few years a significant investment has been made in developing an effective technology portfolio in support of our awarding operations which enables effective auditing and the production of valid statistical data. The increased validity of results has enabled accurate conclusions to be drawn. However, in some cases data represents individuals perceptions that may vary and may deviate from the pragmatic state. Other times data provides only an indication rather than an indisputable fact. Hence, conclusions and identification of trends has been carefully undertaken to avoid misinterpretations and misuse of data.
Figure I. The EFQM Excellence Model (2012)
Product ref. SAR 2013 - Executive Summary
V
Key Outcomes One of the most critical areas of the self-assessment process is the review and comparison of our performance with the industry trends, as this is essential in establishing our future planning and related actions to maintain our competitive positioning. A review of the information presented in the regulators’ annual reports was undertaken to support this aim. The Ofqual Annual Qualifications Market Report (September, 2013) was used to obtain an overview of the qualifications market in England, based on demand and supply in the regulated qualifications market. The Ofqual report also helps to establish a benchmark against which comparisons across awarding organisations can be made and key projections for the future, in line with the sector reform. Similarly, the SQA Accreditation Annual Review (2012/2013) was used to obtain an overview of the qualification market size and trends in Scotland, based on the number of qualifications and learner achievements. Outcomes of the comparisons of industry benchmarks contributed into the identification of strengths and areas for improvement. A summary of the key outcomes is provided in Table I. below.
Summary of key outcomes in 2012 The industry trends (supply and demand in England)
The industry trends
(supply and demand in Scotland)
a total of 176 awarding organisations were recognised to deliver regulated qualifications, five fewer than in the previous year, representing the first decrease since 2005, which is mainly due to Ofqual’s new recognition process. of the 19,000 ‘other’ qualifications on the Register over 11,000 qualifications recorded at least one achievement. there was an overall 4% increase to over 16 million learner achievements, with 5.5 million GCSE achievements, 2.1 million GCE achievements, and 8.7 million achievements in ‘o ther’ qualifications. over the five years, there was an increase of 13% in the number of achievements in regulated qualifications and 37% increase in the number of ‘other’ achievements. qualifications designed for the QCF accounted for over half of the qualifications with achievements, with Edexcel registering onequarter of the achievements and seven awarding organisations accounting for 63% of achievements. the preparation for life and work sector subject area continued to provide the highest number of ’other’ achievements with an 11% increase (a third of a million) over the previous year. This sector represented one- third of all ‘other’ achievements with a total of 38 awarding organisations registering an achievement. achievements are concentrated in relatively few qualifications, 50% of all achievements in ‘other’ qualifications being attributed to 212 qualifications. level 2 qualifications account for nearly half of all ‘other’ achievements. a total of 36 awarding bodies were approved by SQA Accreditation up until March 2013. Out of the 28 organisations that contacted SQA Accreditation to explore the possibility of becoming approved ten decided not to progress. Two awarding bodies, at their request, had SQA-approved awarding body status withdrawn due to lack of qualification uptake. at the end of 2012/13, SQA Accreditation had a total of 940 qualifications accredited, which is a decrease of 3% on prior year (694 SVQs, 196 Other qualifications, 50 Regulatory and Licensing qualifications). due to low or zero uptake of the qualification 97 accredited SVQs were withdrawn by awarding bodies during 2012/13 the highest volume of accredited qualifications are SVQ at levels 2 and 3 there were 196 ‘other’ qualifications accredited, which is an increase of 29% on prior year based on existing awarding bodies aspirations to expand their provision in ‘other’ qualifications in Scotland both learner registrations and certifications show an 8% increase (43,444 and 32,397 respectively). However, uptake is still not at the level seen in 2010/11 or 2009/10 and this may be due to some awarding bodies moving away from SVQs and opting for accreditation of alternative competence-based qualifications.
Product ref. SAR 2013 - Executive Summary
VI
Summary of key outcomes in 2012 Our market share
Despite the significant increase in the supply of the qualifications due to the QCF and increased competition, we sustained a competitive advantage confirmed by the number of qualifications achieved by our learners. Amongst the 530,748 recorded learner achievements for qualifications offered by 47 different awarding organisations we have managed to become the third largest awarding organisation in the leisure, travel and tourism sector in terms of number of learner achievements. In addition to this, the 1st4sport Level 1 Award in Coaching Football (QCF) is included in the list of the top 20 high volume qualifications funded by Adult Learner Responsive provision with a to tal of 14,800 learner achievements.
Supply of our qualifications
Our portfolio of qualifications has continued to expand. With an increase of 8% in the number of accredited qualifications since last year the total number of our accredited qualifications is now at 277. This number includes all pathways, disciplines and add-on modules which are being counted as separate qualifications to ensure consistency with our IT systems’ functionality and the associated audit trail being maintained.
Demand for our qualifications
Trends in the demand for our qualifications suggest sustainable growth. There was an increase (15%) in the number of organisations applying to become a 1st4sport recognised centre compared to last year. The qualification approval demand levels for 2012 have returned to normal levels following the extensive demand due to the QCF migration and as expected, there is a decrease (26%) in the number of qualification approval applications (405). The total number of events (5,252) authorised to be delivered by 1st4sport recognised centres was the largest to date. A minor decline (1%) in both the number of learner registrations (72,835) and certifications (64,069) was noted this year.
Our score against the EFQM model
Evaluation against the nine criteria (and 32 sub criteria) of the revised EFQM model shows that there has been an increase in the average score in the majority of the nine criteria with slight variance in the individual sub-criteria due to the nature of the evaluation methods; the scoring from the self-analysis matrix uses perceptual data whilst the RADAR results are based on evaluation of factual evidence. It is of great importance that the total score achieved is the highest in the last four years (670 out of 1000). The highest score was attained in the business results and the lowest scores in the strategy and people criteria.
Our strategic direction achievement
Extensive use of performance indicators confirmed a positive strategic achievement rate, which also appears to be the highest in the last four years (94% of strategic targets achieved). Our performance is being sustained and further improved across the majority of results. In addition, action was taken to ensure that identified targets are appropriate. Strategic achievements have led to the expansion of our awarding statuses and qualification provision to different types and market areas.
Our plan of provision and awarding status
Our provision has been developed strategically and implemented methodically showing a continuous growth over the years. The highest achievement rate against the effective implementation of our annual plan of provisions is noted this year (92% an 82% target). Furthermore, we successfully maintained our awarding recognition/approval statuses (Ofqual and SQA) in addition to our QCF accreditation and NQF Functional Skills status, ensuring the provision of our qualifications meets and exceeds end-users needs.
Our partnerships and support systems
Both the number of partnerships and number of qualifications developed with nationally recognised bodies/technical experts showed a continued growth, which has been for years our key strength. Existing technical development partnerships were effectively maintained and new ones established, which supports our aspirations to expand in new and traditional markets. Currently 83% of our qualifications are developed in partnership with governing bodies of sport and other organisations. Our target of establishing three new qualification partnerships annually is fully met. Our approach to the development of partnerships and management of support systems are integrated into our strategy and systematically implemented. Innovative IT systems have been successfully upgraded and enhanced in support of our awarding function and improved performance levels.
Product ref. SAR 2013 - Executive Summary
VII
Summary of key outcomes in 2012 Our compliance levels with regulations
Self-evaluation and continuous monitoring of our compliance with regulations is fundamental in order to maintain our awarding statuses. Compliance levels were satisfactory and improved since last year in specific areas where action plans were established. Additional systems/approaches will be developed in 2013 to generate sufficient evidence in particular areas.
Our corporate values and culture of excellence implementation
We aim to ensure we always operate in line with our corporate values, within a culture of excellence. Self-assessment outcomes confirm our efforts to reinforce a culture of excellence via the systematic implementation of our quality standards, developed and revised in line with the EFQM Excellence model. Survey results show that 99% o f our stakeholders confirm that we operate in a fair and equitable manner, and we deliver professional services. Furthermore, 98% perceive that we promote teamwork, we are open to learn and we preserved a personal touch. 95% of stakeholders feel that we try to maintain a sense of humour and have fun.
Satisfaction levels of our stakeholders
Customer loyalty, satisfaction levels and demand for our qualifications are maintained at high levels. Our annual stakeholder and employees satisfaction surveys, learner feedback and performance statistics showed that: 92% of stakeholders confirm that we improve each year in meeting needs/expectations and provide value-added service 100% of stakeholders confirm that we provide qualifications that are of value to each learner's career/role and therefore contribute to society 100% of stakeholders confirm that we develop, deliver and award qualifications that meet the needs of end users (learners, centres, employers) 98% of learners (the highest percentage to date) feel that each qualification adds value to their career/role pathway the number of learners with multiple 1st4sport qualifications is continuously growing; 44,898 of our learners have obtained more than one of our qualifications in 2012. The largest number of 1st4sport Qualifications obtained by a learner is 12. 90% of employees are satisfied with their current job at 1st4sport; most common areas of dissatisfaction are the limited career progression opportunities, ineffective communication across the organisation, inadequate organisational culture/working environment and poor quality of buildings/facilities.
(Stakeholders: recognised centres, learners, partners, external verifiers, 1st4sport employees)
Our external verification levels
According to feedback received from centres external verification activity is considered to be extremely beneficial and valuable due to the constructive feedback provided. Effective external verification procedures were systematically implemented and monitored with the support of our IT systems ( Athena), which also enabled prompt submission of external verification outcomes to centres. For the first time verification target (courses/cohorts being verified at a ratio of 1:6/1:75 - minimum of one course where less than six delivered) is not only met but also exceeded (78%).
The number of incidents that have been effectively managed has not increased significantly in 2012 yet it is the highest number (68) identified and handled in the last six years. Our designated qualified staff and the redesign of our incident and investigation management processes supported by adequate web-based audit mechanisms ensured all incidents were effectively and promptly managed to mitigate any potential risk and maintain compliance levels with regulations. Table I. Key Performance Outcomes Our management of incidents
Product ref. SAR 2013 - Executive Summary
VIII
Conclusion Self-assessment is the fundamental tool by which we gain an insight of our current performance and stakeholder satisfaction levels, industry trends and our compliance levels. Detailed information on our current market position, business results and qualification provision enable us to plan effectively and act strategically in response to the radical changes. Analysis of the self-assessment outcomes provided an understanding of the cause and effect relationship between our enablers (current approaches/systems/arrangements/ processes etc.) and results to help us identify strengths that we need to mai ntain and areas for improvements for the next year. On a global scale, our self-assessment outcomes were very positive, accompanied by continuous upward trends and considerable increase in a number of areas where improvement has been achieved, such as overall performance and provision of service. Despite the significant increase in the supply of the qualifications due to the QCF and the increased competition, we sustained a competitive advantage confirmed by the number of qualifications achieved by our learners. In spite of the introduction of rigorous and stringent regulations by Ofqual, the increased number of risk based SQA audits and qualification withdrawals noted in the Scottish market, we have successfully maintained our awarding statuses and market share with both regulators. Continuous and radical industry change, heightened regulatory expectations and SSC rationalisation required key developments and effective strategic planning in our effort to secure our sustainability. Key achievements which enabled us to maintain our competitive position in the market and increased stakeholder satisfaction confirm our continuous aspirations for improvement and our agility in responding to ever changing industry needs. Our total quality management approach and all interrelated management systems contributed towards maintaining our awarding function through the implementation of our quality standards and the systematic strategic planning process. This in turn, enables us to provide the active leisure and learning industry with a quality-assured and cost-effective qualification awarding service. Regardless of the limitations and given the availability of resources, the self-assessment has been a critical process in evaluating our performance towards compliance with the statutory regulations, overall strategic direction, quality standards and consequently provided an insight into the level of quality. Therefore, it is essential that self-assessment outcomes are communicated and actions are implemented as part of the improvement plan. It is anticipated that maintenance of our strengths and the deployment of the improvement plan will enable us to maintain sustainable competitive advantage and continue to work towards performance excellence in this challenging climate ensuring end-users needs are met. Katerina Doutsiou Quality Manager
Product ref. SAR 2013 - Executive Summary
X
1
Introduction
1.1
Overview
1st4sport Qualifications is an awarding organisation, recognised and regulated by the Office of the Qualifications and Examinations Regulator (Ofqual) and the Scottish Qualifications Authority (SQA). We aim to offer vocational qualifications in the active leisure, learning and well-being and also the l earning and skills sectors. We work in partnership with a variety of organisations, including Governing Bodies of Sport (GBS) and other organisations, to develop qualifications across the following areas:
coaching leadership spectator safety leisure operations and management education and training sports volunteering first aid and injury management functional skills
physical education and school sport using sport to tackle youth crime employee rights and responsibilities exercise and fitness multi-skills development the outdoors sports performance.
1st4sport Qualifications is a brand of Coachwise Ltd, the trading arm of the National Coaching Foundation (known as sports coach UK), the UK-registered charity leading the development of coaches and the coaching system in the UK. Any proceeds go directly back to sports coach UK, to help them develop and advance sport in the UK.
1st4sport Qualifications Mission Statement To provide the active leisure and learning industry with a quality-assured and cost-effective qualification awarding service. As an Ofqual and SQA recognised awarding organisation, we are required to guarantee thorough monitoring and reporting to the regulators regarding compliance and quality levels. This is achieved via an established Performance Management System (PMS), which ensures that our compliance is evaluated and our performance is monitored, managed and improved across all operations. The primary goal of the PMS is to effectively evaluate the implementation of our quality standards derived from the revised European Foundation for Quality Management (EFQM) Excellence Model (2012). In monitoring the implementation of our quality standards, the effectiveness of our mission, strategic direction (strategic objectives, targets, policy, procedure and process) are reviewed and our compliance with regulations is managed. Self-assessment is the integral component of the PMS, an effective technique, which contributes to systematic continuous improvement and indicates the current level of our performance. The fundamental purpose of the self-assessment is to support quality improvement and measure progress against our mission, strategic direction and quality standards.
Product ref. SAR 2013
1
Self-assessment includes a rigorous self-evaluation of our performance against the regulatory requirements to determine compliance levels using the evaluation principles of the EFQM model. To ensure consistent, valid and relevant performance results, which relate directly to the implementation of the quality standards, the EFQM model is also used to undertake self-assessment. On this basis, the self-assessment process involves the use of the EFQM model on which to base the evaluation and diagnostics, with further deployment to enhance our overall performance. Furthermore, the systematic application of the EFQM model contributes significantly towards becoming Recognised for Excellence and ultimately achieving the EFQM Excellence Award. This document aims to report on the outcomes of the self-assessment identifying strengths, areas for improvement and an associated improvement plan structured in line with the nine categories for assessment and 32 sub-criteria contained within the EFQM model.
1.2
1st4sport Total Quality Management Approach
To ensure the continuous development and maintenance of our awarding function within a culture of excellence, we have established The 1st4sport Total Quality Management Approach. This approach is based upon the systematic deployment of the excellence model, which involves implementation of the EFQM criteria set as our quality standards. To facilitate effective implementation of the approach, the quality standards are integrated within our strategic planning and common process framework, providing our organisation with a sustainable advantage in the active leisure, learning and well-being sector and the learning and skills sector. The common process framework is established in line with our quality standards and strategic direction, which is deployed at an operational level. Relevant statutory regulation is encompassed within the processes and related arrangements to ensure compliance with regulator expectation. Control, a process mapping application, has been identified as the fundamental tool for developing and sharing our business processes at an organisational and operational level, hence creating the 1st4sport Common Process Framework . It is our objective to maintain and monitor our awarding statuses via established processes, which are managed by Control (process mapping application) and monitored via the deployment of the f ollowing interrelated systems:
Quality Assurance System ( Athena) Performance Management System Risk Management System.
It is our aspiration that this integrated approach will enable the quality standards, strategic direction and our common process framework to be effectively implemented and rigorously monitored. It is our philosophy to deploy our processes and provide related professional services to our customers through motivated people, who are committed to our mission and the core values which underpin our work. Our core values are also based on the fundamental concepts of the EFQM model. We endeavour to achieve customer/stakeholder satisfaction and ensure continuous improvement through fundamental total quality management principles in the provision of our qualifications and related services. In conducting self-assessment we aim to evaluate and report on levels of performance related to our compliance with regulations and quality improvements. Therefore, for the purpose of this report the focus will remain with that of the performance management system.
Product ref. SAR 2013
2
1.3
1st4sport Performance Management System (PMS)
The rationale for the development of our performance management system is resultant of the tools required in the monitoring of our quality standards with an emphasis on the strategic direction and compliance with statutory regulations. To generate valid data and understand the cause and effect relationship of the results; appropriate performance measurement tools are utilised in the methodical evaluation and management of our performance. The performance mechanisms established are systematically monitored and improved to ensure that the data generated is valid, accurate and enables the identification of trends and comparisons of our performance over the years. This in turn, enables a rigorous performance analysis to determine whether targets have been achieved and also the reasons why performance may not be sustained in some areas. Performance measurement outcomes are monitored and assessed on a regular basis, presented in the form of the annual selfassessment report, which contributes to effective decision -making and future strategic planning. Performance measurement methods are primarily used to gain an insight on the current status concerned with the effectiveness and consistency of the total quality management approach. As the PMS contributes to the continuous quality improvements via the effective review of key areas and compliance with the regulations it also leads to the identification of future developments. Results from the performance measurement tools serve to maintain and increase the level of quality, thus continuously improving the services provided. This is how we endeavour to achieve performance excellence and sustainable development over the years.
1.4
Self-assessment
Self-assessment as the integral component of the performance management system indicates the current level of our performance and contributes to systematic continuous improvement. It provides vital information in relation to the progress made, strengths and weaknesses and the simultaneous identification of causes and areas for improvement. The process itself increases organisational learning, staff awareness and establishes the basis for appropriate decision –making. To this end, self-assessment as an effective performance management technique enables us to:
maintain and manage our quality standards and awarding function measure our process effectiveness and business performance (financial and non-financial) provide the regulatory authorities with relevant information on an operational and compliance level obtain feedback to establish stakeholder satisfaction levels conduct benchmarking and make relative comparisons develop quality-oriented processes seek opportunities in achieving improvement and leading the improvement process implement a regular strategic and planning process to enable continuous improvement and increase the overall performance and quality level.
The core focus of the self-assessment is ultimately to support quality improvement and measure progress against our mission and effectiveness of our strategy.
Product ref. SAR 2013
3
1.6
The EFQM Excellence Model as a quality management and self-assessment model
Amongst the numerous management tools and techniques commonly used, the EFQM Excellence Model was selected as the most appropriate model as it provides a holistic view of the organisation and can be used to determine how different methods fit together and complement each other. The model is therefore used as a quality management model, representing an overarching framework based on our needs and awarding function for developing sustainable excellence, but also as a self-assessment tool. The great benefit that can be obtained from the effective use of the model, which leads to sustainable success, is based on the three integrated components:
The Fundamental Concepts of Excellence The EFQM Excellence Model RADAR logic
Use of these three integrated components enables organisations of all sizes and from all sectors to compare themselves with the attributes, qualities and achievements of sustainable organisations and also ensures that management practices form a coherent system that is continually improved and delivers the intended strategy for the organisation. 1.6.1
The Fundamental Concepts of Excellence
The Fundamental Concepts of Excellence as shown in Figure 1.1 are the underlying principles and form the essential foundation for achieving sustainable excellence. Used as the basis to describe the attributes of an excellent organisational culture they also serve as a common language for senior management.
Figure 1.1 The Fundamental Concepts of Excellence Product ref. SAR 2013
5
1.6.2
The EFQM Excellence Model
As a quality management model, the EFQM indicates the areas an organisation should focus on in order to improve performance and competitive positioning. The model represents a holistic view of quality concepts, which are required to be developed and linked to our processes to accomplish positive results. The EFQM model consists of nine assessment criteria and 32 sub-criteria, which comprise our quality standards.
Figure 1.2 The EFQM Excellence Model (2012) Our decision to implement the EFQM as a quality management model for our quality standards was based upon its capacity to enhance performance and to provide consistent, valid and relevant performance results. On this basis, it was deemed most appropriate that it is also applied as our self-assessment model. The EFQM model allows managers/leaders to understand the cause and effect relationships between the outputs and results achieved.
Product ref. SAR 2013
6
Execution of the EFQM as a self assessment model:
promotes sharing and dissemination of information on successfully deployed strategies and derived benefits facilitates in achieving quality improvements and can be used for benchmarking activity against other awarding bodies stimulates systematic self-assessment against the designed criteria and the market industry provides consensus of our strengths and areas for improvement enables measurement of our progress, key developments and respective growth.
1.6.3
The RADAR logic
The third component, the RADAR logic as shown in Figure 1.3, is a dynamic assessment framework and powerful management tool that provides the backbone to support an organisation. Due to the particular design and structure it enables the identification of weaknesses and provides a structured approach to assess the organisational performance. The RADAR also supports the scoring mechanism behind the EFQM Excellence Award and other recognition/assessment schemes and can help to lead change and manage improvement projects.
Figure 1.3 The RADAR logic Product ref. SAR 2013
7
1.7
The self-assessment report
To ensure full review of our operations in line with the EFQM, comparisons are made across the full range of our performance measurement outcomes via the RADAR assessment and management tool and the matrix score. The combination of the RADAR and the matrix approach used in the conduct of self assessment enables the consolidation and alignment of the results from our measurement tools used on an annual basis in accordance with the EFQM criteria. A number of quantitative and qualitative data collection tools are used to facilitate valid and reliable performance outcomes. This serves to ensure self-assessment against the breadth of our quality standards guarantees measurement of progress related to the strategic direction and compliance against statutory regulation. The self-assessment report is published annually resultant of the outcomes of the self-assessment process and made available to all relevant stakeholders. The structure of the self- assessment report is based on the EFQM model, which facilitates an integrated approach to striving for organisational excellence. Strengths, areas for improvements and an improvement plan in line with the EFQM model conclude the report.
Product ref. SAR 2013
8
2
Methodology
2.1
EFQM: The self-assessment model and assessment criteria
The fundamental purpose of the self-assessment process is to support quality improvement via measurement of the implementation of the revised EFQM Excellence Model criteria and sub criteria which are set as our quality standards. This i n turn, ensures review of our strategic direction achievements and compliance with statutory regulation. Performance measurement tools were developed to generate evidence and subsequent analysis of this was therefore undertaken to ensure effective review of our performance. In evaluating our current management practices a number of approaches, data collection methods and tools (table 2.4) were utilised to obtain valid and reliable outcomes. The representation of outcomes within the self-assessment report follows the structure of the EFQM model as an integrated and systematic approach to striving for organisational excellence. The dynamic nature of the model and a brief explanation of what the nine criteria represent within our organisation are shown in figure 2.1. Full representation of 32 subcriteria/quality standards, assessed via established performance measurement tools, is provided in table 2.1. Enablers
Results
3. People How we manage our employees
1. Lead ership
How o ur Senior Management Team lead to d eliver a better organisation
2. Strategy Where is the organisation going and how it w ill get there
4. Pa rtnerships & Resources
7. Pe ople Results Are the employees satisfied
5. Processes, Products & Services How we manage and improve what w e do; how we ensure customer focus before and after delivery of products or provision of services
How we make best use o f what we need
6. C ustomer R esults Are our customers satisfied and want to continue to do business with us
8. Society Results
9. Business Results
Are we ahieving as much as w e could to ensure our sustainability
How do we ensure compliance with regulations and contribute to the so ciety; what do they think of us
Learning, Creativity and Innovation
Figure 2.1 Explanation of the EFQM model (2012) Product ref. SAR 2013
9
EFQM results criteria and sub-criteria (1st4sport quality standards) 6
Customer Results
6a
Perceptions
6b
Performance Indicators
7
People Results
7a
Perceptions
7b
Performance Indicators
8
Society Results
8a
Perceptions
8b
Performance Indicators
9
Business results
9a
Business Outcomes
9b
Business Performance Indicators
Table 2.1 EFQM criteria and sub-criteria (1st4sport quality standards) The nine structured assessment criteria enable the analysis of the cause and effect relationship between what the organisation does and the results to be achieved. As shown in table 2.1 five of these criteria act as 'Enablers' and four as 'Results' . The 'Enabler' criteria cover what we do as an organisation and the way we do it whilst the 'Results' criteria cover what we should be working towards to achieve. Each of the nine criteria has a definition, which describes what each criterion involves. To develop the high level meaning further, each criterion is supported by sub-criteria. The RADAR tool facilitates the assessment and scoring of the level of each sub-criterion. Therefore, to ensure that benefits are obtained from the effective application of the model the components of the fundamental concepts of excellence and the RADAR logic were utilised across the 32 sub-criteria to conduct self-assessment based on a multidimensional approach.
Product ref. SAR 2013
11
2.2
The RADAR logic as an assessment and management tool
The RADAR logic is a dynamic assessment framework and powerful management tool which provides a structured and logical way to question and report on the performance of the organisation. It also supports the identification of strengths and areas for improvement in addition to being the scoring mechanism behind the EFQM Excellence Award and other recognition schemes, which is one of our strategic aims over the next two years. Most importantly, the RADAR is an integral tool of the EFQM model which was used to assess and manage performance and indicate our level of excellence against the nine criteria and 32 sub-criteria of the model. RADAR ensures that diagnosis is based on an objective and analytical approach to determine the relative progress made each year. Results from our performance measurement tools are managed on an annual basis. Using the evidence generated from the breadth of our established performance measurement tools the RADAR scale was applied to assess each approach, which reflects the overall score. This in turn, makes the identification of strengths and required improvements required easier. In addition, it allows comparisons on our performance over the years, as well as on results achieved by other organisations regardless of the type of activity or sector who implement the EFQM model. The RADAR tool has been used for the last four consecutive years in the conduct of the self-assessment, however as the EFQM model was revised in 2010 and again in 2012, the updated RADAR scoring matrix and weighting criteria have slightly changed, therefore results were treated with caution when making comparisons. The RADAR logic comprises of the following four elements; Results, Approach, Deployment, Assessment and Refinement. In using the RADAR logic we reviewed whether we:
Determine the Results we aim for as part of our strategy (scope/integrity/segmentation/trends/targets/comparisons/causes) Plan and develop integrated set of sound Approaches (arrangements, processes, etc.) to deliver the required results Deploy our approaches in a structured way to ensure implementation Assess and Refine our deployed approaches based on monitoring and analysis of the results achieved and ongoing learning activities.
These elements represent our quality indicators used for the identification of performance levels and subsequent improvement plan. The use of the RADAR approach and related quality indicators determine the score, which reflects the level of performance and quality levels, achieved. The score contributes to the decision making of the relevant actions, which form an improvement plan for each year. The application of RADAR enabled an evaluation of performance levels across different operational areas to be conducted and provided justification for the outcomes. However, as with every metho d RADAR entails some limitations. The RADAR does not evidently reflect the causal relationship interpretation between the methods used and the results obtained. It requires thorough analysis and multidimensional feedback via the conduct of workshops and related consensus meetings to validate the outcomes. There is a high risk of subjectivity when this method is used by a single individual or assessor and validity of score could be questioned. To ensure that the assessor’s subjectivity is limited a consensus meeting was conducted following sufficient training, which served to increase the effort of our journey to excellence by expanding the methods applied and validating the outcomes thoroughly.
Product ref. SAR 2013
12
A representation of the way the RADAR logic and principles were applied is summarised in the RADAR scoring matrix. The score evidence sourced from the performance measurement tools was assessed in line with the RADAR scoring matrix to identify our current performance level against each sub criterion. Structured diagnosis and methodical assessment of available evidence was taken place using a scoring scale of 0% - 100%. The scores resulting from the assessment of the nine elements (five Enabler criteria and four Results criteria) were added together to make up the overall score for the specific year. The RADAR tool for Enablers Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Approach
Guidance
Sound
The approaches have a clear rationale, based on the relevant stakeholder needs, and are process based.
Integrated
The approaches support strategy and are linked to other approaches as appropriate.
Deployment Implemented
The approaches are implemented in relevant areas, in a timely manner.
Structured
The execution is structured and enables flexibility and organisational agility.
Assessment and Refinement Measurement
The effectiveness and efficiency of the approaches and their deployment are appropriate measured.
Learning & Creativity
Learning and creativity is used to generate opportunities for improvement or innovation.
Improvement & Innovation
Outputs from measurement, learning and creativity are used to evaluate, prioritise and implement improvements and innovations.
Product ref. SAR 2013
5
10
15
13
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
The RADAR tool for Results Unable to demonstrate
Limited ability to demonstrate
Able to demonstrate
Fully able to demonstrate
Recognised as Global Role Model
Scale
0%
25%
50%
75%
100%
Overall Score
0
Relevance & Usability
Guidance
Scope & Relevance
A coherent set of r esults, including key results, are identified that demonstrate the performance of the organisation in terms of its strategy, objectives and the needs and expectations of the relevant stakeholders.
Integrity
Results are timely, reliable and accurate.
Segmentation
Results are properly segmented to provide meaningful insights.
Performance Trends
Positive trends or sustained good performance over at least 3 years.
Targets
Relevant targets are set and consistently achieved for the key results, in line with the strategic goals.
Comparisons
Relevant external comparisons are made and are favourable for the key results, in line with the strategic goals.
Confidence
There is confidence that performance levels will be sustained into the future, based on established cause & effect relationships.
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95 100
Regardless of the limitations, RADAR was used to enhance the integration between the implementation of the model/quality standards and reporting of the performance levels via self-assessment. Primarily, we strive to increase the effort on our journey to excellence by expanding the methods applied and validating the outcomes thoroughly to ensure a coherent system is continually improved and delivers the intended strategy. Product ref. SAR 2013
14
2.3
Self-assessment validation approaches
A number of approaches are available for the conduct of self-assessment or can be used to support and triangulate the RADAR outcomes. However, there is no distinguished approach, each delivers different benefits and requires alternative resources. Approaches are recognised as successful when they suit the corporate culture and used to foster participation. The several approaches for selfassessment identified within the EFQM model are described in table 2.2.
Approach
Description
Pro forma
This approach involves the creation of a set of 32 forms, one of each sub-criteria, on which the strengths, areas for improvement and scoring results are addressed. To prepare and complete the pro formas, assessment teams or individuals have to gather the necessary information. The management team agrees on common strengths and areas for improvement and scores at a consensus meeting.
Workshop
The workshop approach consists of five distinct phases namely training, data collection, a scoring workshop, prioritisation of improvement actions and review of progress. The management team is responsible for collecting data and presenting at workshops with the aim of reaching consensus on issues related to strengths and weaknesses.
Questionnaire
The use of questionnaire is a practical approach to gain an insight view of the people’s perception within the organisation. Although there are some standard questionnaires associated with the self-assessment model, a simple yes/no or a more sophisticated questionnaire can be produced and used in conjunction with workshops to analyse the areas for improvement.
Matrix
The matrix approach involves the development of organisational-specific achievement matrix, which typically consists of statements of achievements ranging between a number of points on a scale of 1-10. The practical format, transparency and range of statements enable the respondent identify quickly and easily the level of the organisation.
Award simulation
The essence of this approach is the replication of applying for the award. The procedure involves the preparation and submission of an up to 75 page document; a team of trained assessors use the document to make an assessment, create a score and finally provide a report. Table 2.2 Available approaches to conduct self-assessment In assessing our performance against the EFQM criteria the self-analysis matrix and workshop were identified as the most appropriate self-assessment approaches. Benefits, limitations and rationale for the approaches selected are provided in the following pages. Product ref. SAR 2013
15
2.4
The self-analysis matrix as the selected approach
The self-analysis matrix was used to triangulate outcomes, therefore increasing the validity of the RADAR score, and was selected due to its transparent and practical features in determining the current level of performance punctually. The matrix approach is identified as the quickest and easiest to be completed by all employees. It represents the practical step for introducing self-assessment and at the same time is the simplest method to administer. It requires minimal training and provides a visual gap analysis to address the strengths and areas for improvement. Furthermore, it is a means of involving everyone in the self-assessment process and supports team discussion. The matrix approach can be used at any level with linkages to wider target setting, providing a view of our progress. Extensive analysis of the identified approaches against the training, time limitations and the development level supported that the matrix approach is the most advantageous to ensure our corporate facets and culture are appropriately explored. As we are making a high effort in line with the maturity effort chart (table 2.3), the tailored matrix is deemed as the most appropriate approach. In using the matrix and also the workshop approach the RADAR outcomes were triangulated and validated, therefore ensuring an accurate evaluation of our performance levels across different operational areas together with a rationale for the outcomes. The self-analysis matrix is a pre-validated evaluation tool, which provides a score towards the EFQM criteria and enables the assessment of the organisation’s relevant areas. The development of the tailored matrix was based on the UK North West Quality Award Model organisational self-analysis matrix. It consists of statements of achievements per criterion, which describe different levels of performance ranging between a number of points on a scale of 1-10, as presented in section 3.3 EFQM self-analysis matrix responses of this report. The matrix has been used for the last five consecutive years with a few changes in the statements to reflect the revised version of the EFQM model in 2012 and to provide an insight on our performance levels and related sustainability. The statements developed by our self-assessment team were adapted to increase respondents ’ understanding ensuring they were specific to the awarding function without deviating from the standard matrix concepts, to maintain the validity of the score. The statements were subsequently piloted with respondents to facilitate face validity and with the Quality Management Team for content validity. Prior to the completion of the matrix, 1st4sport employees received training on the linkage between the matrix and the EFQM model in order to identify the statement, which on their opinion best reflects the current level of our performance. The practical format, transparency and range of statements enabled the respondent to identify quickly and easily the level of the organisation. Resultant of this, an insight into our performance levels was provided, based on an overall score. The matrix score was then compared to this of the RADAR. The statements refer to specific practices and processes which could provide a checklist used to identify areas for improvement. Despite the fact that the structural range of statements facilitates in the identification of improvement levels, it does not evidently reflect the identification of strengths and areas for improvement. As every organisation and respective statements are different, it is difficult to make comparisons with the industry average using benchmarks of the scores achieved. Where this approach is to be used in conjunction with the RADAR and the workshop approach, these limitations are significantly reduced.
Product ref. SAR 2013
17
2.5
The workshop as the selected approach
Precautions were taken to preserve the validity of the matrix score and limit the risk of subjectivity, hence the workshop was selected as an additional approach to validate the matrix results via the conduct of a consensus meeting to establish the RADAR score. As a developing organisation and aiming to apply for the EFQM recognition scheme higher efforts were made this year. The workshop approach consists of five distinct phases training, data collection, a scoring workshop, prioritisation of improvement actions and review of progress. Upon data collection and training a workshop with the aim of reaching consensus on issues related with strengths and weaknesses and the respective assessment of our performance against the 32 sub-criteria was conducted by the selfassessment team. The significance of this process lies in active involvement and in the development of team working climate. This also helps to limit the degree of subjectivity when assessment is taken by a single assessor. As every approach, it has risks, such as the requirement for trained assessors or unrealistic and not objective assessment, which results with generous and inaccurate scoring. However, there are considerable benefits; the process itself increases organisational awareness, creativity and innovation and enables problem solving based on the opportunity for thorough analysis, discussion and effective communication. The workshop approach helped to identify gaps and take preventive and corrective action associated with deficiencies. Most importantly, it enabled benchmarks to be established which could be compared to other organisations that use the RADAR scoring system when selfassessing against the EFQM model. Based on the aforementioned benefits it was decided that the workshop approach will not only become a standard approach for self-assessment but also an integral part of our effort for continuous improvement and excellence.
Product ref. SAR 2013
18
2.6
Performance measurement tools
Our established performance measurement system is effectively implemented, monitored and improved to ensure that the data generated is valid, accurate and enables the identification of trends and comparisons of our performance over the years. Our performance measurement tools serve to maintain and increase the level of quality, leading to effective decision-making and future strategic planning. Implementation of our performance measurement system is shown in diagram 2.1.
The performance measurement system
System ready for implementation
Audit and evaluate regulatory compliance
Audit and evaluate strategic achievement
Audit and evaluate plan of provision achievement
Audit and evaluate partnership and consultancy effectiveness
Head of Quality Management
Quality Manager
Head of Qualification Development
Head of Qualification Development Data generated
Audit and evaluate stakeholder satisfaction Quality Manager
Audit and evaluate qualification demand and awarding performance
Audit and evaluate improvement p lan achievement
Audit and evaluate against industry benchmarks
Quality Manager
Quality Manager
Head of Quality Management
Diagram 2.1 Implementation process of the performance measurement system Product ref. SAR 2013
19
Table 2.4 represents the data collection tools that generated evidence, which were then evaluated using the RADAR against the 32 subcriteria of the EFQM model.
Data collection tools and representation of performance outcomes Tools
Outcomes
Data collection method/source
Annual reports from regulators
Education industry trends
Ofqual report and SQA review
1st4sport performance statistics
Trends in CR/QAP statuses Trends in accredited qualifications
n o i t c e r i D c i g e t a r t S / s d r a d n a t S y t i l a u Q / s n o i t a l u g e R
Data classification
Type of analysis
Quantitative feedback
Descriptive statistics
Athena
Quantitative feedback
Comparisons of descriptive statistics
Athena, Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Trends in course authorisations
Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Trends in learner registrations
Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Trends in learner certifications
Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Trends in learner participation by gender
Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Trends in learner participation by age
Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Trends in learner participation by ethnicity
Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Disability types in learner participation
Parnassus
Quantitative feedback
Comparisons of descriptive statistics
Trends in access arrangement requests
Athena
Quantitative feedback
Comparisons of descriptive statistics
Trends in incidents
Athena
Quantitative feedback
Comparisons of descriptive statistics
Strategic direction analysis
Achievement rate
Athena, Parnassus, Formic, Maginus
Quantitative feedback
Descriptive statistics
Statutory regulations review
Compliance with statutory regulations
Quantitative feedback
RADAR
Product ref. SAR 2013
Regulation documentation 20
Representation
chart/table
chart
chart
chart
chart
chart
chart
chart
chart
chart
chart
chart
chart/table
table
n o i t c e r i D c i g e t a r t S / s d r a d n a t S y t i l a u Q / s n o i t a l u g e R
Data collection Data classification method/source
Type of analysis
Comparison with other awarding organisations’ performance
FAB scheme
Qualitative feedback
Descriptive statistics
table
Stakeholder satisfaction levels
Formic
Quantitative feedback
Descriptive statistics
chart
Formic
Qualitative feedback
General inductive analysis(Patton,1990)
table
Athena
Quantitative feedback
Descriptive statistics
chart
Athena
Qualitative feedback
Deductive Analysis
table
Tools
Outcomes
Benchmarking Stakeholder satisfaction survey
Recognised Centre satisfaction questionnaire
Recognised Centre satisfaction levels
Representation
Learner evaluation forms
Learner satisfaction levels
Formic, Cognos
Quantitative feedback
Descriptive statistics
chart
Employee satisfaction questionnaire
Employee satisfaction levels
Survey
Quantitative feedback
Descriptive statistics
chart
Qualitative feedback
Deductive Analysis
table
External Verifier evaluation forms
EV satisfaction levels on CPD
Formic
Quantitative feedback
Descriptive statistics
chart/table
Qualification specific performance statistics
Variance in qualification demand
Parnassus
Quantitative feedback
Descriptive statistics
chart
Analysis of enquiries
Athena
Quantitative feedback
Descriptive statistics
chart
Analysis of incidents
Athena
Quantitative feedback
Descriptive statistics
chart
Analysis of actions and recognised centre risk profile
Athena
Quantitative feedback
Descriptive statistics
chart/table
Analysis of access arrangements
Athena
Quantitative feedback
Descriptive statistics
chart
Table 2.4 Data collection tools and representation of results In the collection, processing and analysis of the above set of data a number of IT systems and softwares were used as shown below:
Athena – the 1st4sport Quality Assurance System Parnassus - the 1st4sport Awarding Service System Cognos – Reporting Software Maginus – Stock, Sale and Management Software Formic – Survey Design and Deployment Software.
Product ref. SAR 2013
21
2.6.3
Strategic direction analysis
One of the most critical and integral areas of the self-assessment is the measurement of achievement rates and delivery of our strategy; hence a number of methods were used. In the implementation of our quality standards we must ensure that Results are also compared to the industry trends. As part of our assessment and review 1st4sport trends and achievement rates are also compared to the education industry trends to determine whether our strategic direction is effective. Developing and implementing a strategy is not sufficient unless this is in line with the current and future industry needs. Therefore, our systematic review ensures that current and future strategic planning will lead to accuracy in achieving sustainable advantage. For the majority of results, data was collected via standard IT based performance measurement methods. Data generated by Athena, Parnassus, Maginus, Cognos and Formic made evaluation of results quicker and accurate. Standardisation in the use of performance measurement methods has been very beneficial, allowing identification of trends and monitoring of the level of our sustainability between 2008 and 2012 for the vast majority of areas. This in turn, has provided sufficient evidence to assess the Results section of the EFQM model via the RADAR. In some cases insufficient data due to lack of performance measurement tools made the identification of achievement rates difficult. In order to measure specific targets, it was considered that data could only be collected via stakeholders’ perceptions. Where t his occurred alternative methods such as distribution of questionnaires were selected. Hence, perceptual data was used instead as evidence to measure achievement of these targets. Lack of respondents’ awareness when asked to evaluate specific areas has increased the ‘Not known’ responses rate and affected the target achievement rate results. Amendments and updates made in our strategic direction to ensure accuracy and validity did not allow comparison of descriptive statistics in certain areas due to lack of data from previous years resulting from newly introduced targets or enhancements in IT systems/data collection methods. Similarly, achievement rates could not be identified for all targets that are still to be achieved as they form part of the 5 year strategic plan. Outcomes from the analysis of our strategic direction and associated performance levels are presented in Appendix 3. 2.6.4
Self evaluation of statutory compliance
Systematic monitoring and self evaluation of our compliance with regulations is fundamental in order for us to maintain our awarding statuses and continue to operate within particular areas and award specific types of qualifications. As an awarding organisation recognised and regulated by Ofqual and SQA we are required to maintain and report on compliance levels across statutory regulations, including any guidelines and principles of good practice as stated in regulatory documentation, in the development, delivery and awarding of all our qualifications.
Product ref. SAR 2013
23
As we aim to expand every year in different qualification areas and markets which may entail compliance with specific regulations there was a greater need to establish a common method of compliance review to incorporate all different types of regulations within our awarding practices. Comprehensive evaluation of our compliance was undertaken against the following statutory regulations:
Awarding Body Criteria - SQA (2007) 1 General Conditions of Recognition - Ofqual (November 2012)
A range of performance measurement tools were utilised to collect evidence and review compliance levels. The review of our compliance was based on our awarding activity, management practices, documentation and processes. The collection and evaluation of evidence involved the conduct of interviews with relevant employees, review of documentation and a series of review meetings to ensure effective and objective evaluation of performance and valid results. As shown in table 2.5 the self-evaluation was conducted using the principles and scale of the RADAR tool for the established approaches, deployment and assessment and refinement activities, which enabled a comparison across the results of associated regulations.
Statutory Regulations RADAR Scale applied for the evaluation of approaches, deployment, assessment and refinement (Self-assessment against the EFQM criteria)
RADAR Scale applied for the evaluation of approaches, deployment, assessment and refinement (Manipulated for the self-evaluation against regulations)
Recognised as a global role model Fully able to demonstrate Able to demonstrate Limited ability to demonstrate Unable to demonstrate
Recognised as a global compliance role model Fully able to demonstrate compliance Able to demonstrate compliance2 Limited ability to demonstrate compliance Unable to demonstrate compliance
Table 2.5 Use of the RADAR tool to conduct self evaluation of our compliance levels with regulations Statutory compliance review outcomes are presented in Appendix 4.
1
On 1 April 2012 the SQA Awarding Body Criteria (ABC) (2007) were replaced by the SQA Accreditation Regulatory Principles (2011), which run alongside the ABC until 31 March 2013 during the transitional phase. As from 1 April 2013 SQA will regulate using only the Principles. As a result, 1st4sport commenced revision of all approaches, processes and systems in line with the new SQA principles in 2012. Therefore, a self-evaluation was conducted in line with the principles in preparation for comparison in the next annual cycle. 2 Any Condition which is identified lower than this point on the scale is classed as non-compliant. In 2012, Conditions that were confirmed as Able to demonstrate compliance or above w ere reported to Ofqual as compliant as supporting evidence to the statement and to prevent any confusion.
Product ref. SAR 2013
24
2.6.5 Benchmarking Benchmarking as part of our Total Quality Management approach determines the sharing of best practices and can be regarded as a tool for organisations to maintain their competitive edge in the respective industries. Benchmarking is considered as the process of obtaining information, to identify, understand and adapt best practice, which results as superior performance, through methodical change implementation and performance measurement. As part of our efforts for continuous improvement we conducted benchmarking activities to enable comparisons of our performance against our main rivals/best in class organisations in the industry. In light of the implementation of the Ofqual General Conditions of Recognition (2012), the Federation of Awarding Bodies (FAB) facilitated a scheme, which enabled awarding organisations to become part of a brokerage group with other awarding organisations. In doing so, 1st4sport are now able to undertake benchmarking with other similar organisations by which to improve operations and work towards achieving excellence. The group exists only t o facilitate each other’s knowledge and understanding of the regulations and of the awarding function of an organisation; supporting standardisation and best practice. There is no competition involved within the organisations involved and strong, trusting relationships have evolved. For confidentiality purposes, the names of the organisations involved within the brokerage group are not named, no details are supplied and all information related to the brokerage group is maintained securely. The sole information made available for the purpose of this report are the areas identified by 1st4sport as having been benchmarked and the 1st4sport Senior Management Team response to the benchmarking outcomes. This confirms which areas are identified as strength and/or areas for improvement (in which cases action plans have been devised). Outcomes from the benchmarking activity are presented in Appendix 5 . 2.6.6
Stakeholder satisfaction survey
In support of the self-assessment process, a number of established evaluation questionnaires are distributed throughout the year seeking feedback on 1st4sport performance. For the last five consecutive years the stakeholder satisfaction survey has been one of the most significant data collection methods as part of the standard tools used annually for the self-assessment. In conducting the survey we aimed to determine our performance, quality improvements and related achievements, establish stakeholders satisfaction levels on the services provided and most importantly identify stakeholder’s future needs. Respondents were asked to provide perceptions on different operational areas, which included:
the awarding organisation quality management qualification provision awarding services awarding systems.
Product ref. SAR 2013
25
Stakeholders (recognised centres, qualification partners, external verifiers and 1st4sport employees) provide their perceptions of the same set of questions. The standardised questions were developed by the self-assessment team and were used to make measurement more precise by enforcing uniform definitions. Standardisation of questions allows the identification of trends and related comparisons in respect of our performance during the last five years. As a result of the Ofqual General Conditions of Recognition and updates on the EFQM model amendments to the structure and content of the survey were made. The survey was developed in such a way to enable similar data to be collected from groups then interpreted comparatively (betweengroup study). Perceptual data was generated to enable effective assessment across all areas without affecting comparison with previous years. To enable ease of use, quick responses and production of statistically significant results the survey was distributed via email and completed online using our web based form, designed in Formic . It is of great significance that the stakeholder satisfaction survey enables areas, which could not be assessed via other performance measurement tools to be measured via stakeholder’s perceptions. In some cases areas comparison between factual performance and perceptual performance results was made, which facilitated in forming conclusions and understanding the cause and effect. Quantitative and qualitative data from the survey was analysed to ensure objectivity in assessment, the identification of strengths and areas for improvements. The survey was conducted in March 2013, and both quantitative and qualitative feedback from all populations was analysed and compared. To ensure accuracy of results a number of respondent populations were identified to facilitate analysis and to enable thorough comparisons to be made with the aim of drawing conclusions and presenting overall trends. The significant increase in the overall response rate in comparison t o previous years is indicated in tabl e 2.6.
Annual stakeholder satisfaction survey - respondent type and response rates Type of Respondent
Number of responses (2009)
Number of responses (2010)
Number of responses (2011)
Number of responses (2012)
Number of respondents contacted (2012)
Percentage of responses (2012)
Percentage rate received (of potential responses)
Recognised Centre staff members (RC) 1st4sport Qualification Development Partner (Technical Advisor)
35
87
106
208
1430
67%
15%
9
6
9
28
70
18%
40%
1st4sport External Verifiers (EV)
31
64
52
55
88
9%
63%
1st4sport internal staff members
17
18
20
18
19
6%
95%
309
1607
100%
19%
Total 92 175 187 Table 2.6 Population and response rate of the annual stakeholder satisfaction survey Product ref. SAR 2013
26
2.6.8
Learner evaluation forms
The standard method used to capture and analyse data in respect of qualification performance and learner satisfaction is the learner evaluation form implemented in Formic , the established IT system. Evaluation forms are distributed to all learners upon successful completion of the course for a specific qualification. The learner evaluation form is a significant tool in the collation of feedback from the end users on critical areas that determine the performance and value of each qualification and possible areas for improvement. In addition to the contribution to the annual self-assessment outcomes, learner evaluations play an important role in the continuous analyses of the qualification performance, contributing to the ongoing review of each qualification accreditation status, and thus, the annual plan of provision. The choice of surveys as opposed to a different data collection method for learner evaluation forms ensured construct validity. The rationale being that no other data collection method is as adequate in the collection of data across a number of large populations and in the measurement of their individual characteristics. The evaluation forms were designed by technical experts in consultation with the qualification partners ensuring content validity, and consist of a set of questions that enable rating against the effectiveness and quality of the course. In the evaluation form learners are asked to provide qualitative and quantitative feedback on aspects such as:
pre-course organisation and induction course programme assessment delivery and assessment resources tutor/assessor competency course and quality management qualification evaluation personal evaluation overall rating.
Data collected is captured prepared and analysed using Formic , which enables the production of reports that are also shared with the qualifications technical development partners. The amount of data collected is considerably high, therefore results are prepared in line with the established timescales for data processing, reporting and publishing to cover a twelve months period (January – December). The standardised form is designed to cover all types and levels of qualifications ensuring we also respond to the needs of our technical development partners. The total number of learner responses received shows 4% decrease since 2011, with the overall responsiveness rate being 27%. Several factors could explain this; the small decrease in the number of certificated learners in 2012, the nature and length of qualifications which may spread over two or three years and also the timescales of the forms submission which may be received by 1st4sport a long time after the completion of the course.
Product ref. SAR 2013
28
