Documentation

Executive summary :

 

In accordance with its strategic plan, the Board of the European Association for Quality Assurance in Higher Education (ENQA) has undertaken an analysis of the
rst tranche of review reports of agencies dating from the inception of the process in 2005 until December 2009. The intention of this analysis is to ful
l several purposes:

• the report can be viewed as a self-evaluation undertaken by ENQA; it captures, re ects on and evaluates review work carried out so far;
• the report provides valuable information on the use and applicability of the ESG in agency reviews; this information will be of interest not only to ENQA and its members but to other E4 members and beyond, particularly when considering the scope and content of any future review of the ESG themselves;
• lessons learned from the project will help to shape the second round of reviews which is about to start and
• the report will prove useful in the general evolution of the respective missions of and methodologies used by ENQA and the European Quality Assurance Register for Higher Education (EQAR).

 

The context within which the reviews of these agencies were carried out is important: quality assurance agencies were being reviewed against the principles set out in the ESG for the
rst time; the agencies are also diverse in mission, size, ethos and methodologies. In spite of this diversity the report evidences a signi
cant level of
compliance with the ESG.

 

In total, 34 reports were analysed. In terms of compliance with the ESG, it is apparent that standards 3.2 (Of
cial status) and 3.3 (Activities) present agencies with the fewest challenges. Full compliance against the other Standards was, however, achieved less often, speci
cally:

• Standard 3.1 (Use of external quality assurance procedures): 68% full compliance; 
• Standard 3.4 (Resources): 66% full compliance; 
• Standard 3.5 (Mission statement): 66% full compliance; 
• Standard 3.6 (Independence): 61% full compliance; 
• Standard 3.7 (External quality assurance criteria and processes used by agencies): 
• 58% full compliance and 
• Standard 3.8 (Accountability procedures): 65%. 

These six standards have clearly presented agencies and panels with greater challenges.

 

This report highlights, where possible, the extent to which any dif
culties encountered were as a result of shortcomings within the agencies, or dif
culties with the standards themselves, including their interpretation, or their use by panels. However, it would be wrong to suggest that there are wide scale problems; the report makes some recommendations with a view to assisting in the clearer and more consistent use of the standards in future.

 

Those recommendations and conclusions are grouped under six headings:

• The level of compliance and the ‘maturity' of the agency, the legal contexts in which agencies work, and the year of review
• Panels and their Judgements, including terminology and consistency 
• Problem areas - for the agencies or with the ESG?
• Training 
• Impact of the ESG and their role in safeguarding QA in HE (have they done the job so far?)
• Issues for any review of the ESG themselves.

 

The report is structured to allow the reader to see the main commentary and conclusions
rst, followed by an annex with a more detailed analysis against each of the ESG in order to demonstrate how the conclusions were reached. A list of the reports on agencies that were included in the project may be found at annex 2.