Analyses & Evaluation
Evaluation is a core phase of quality management. Both EQAVET and ISO 21001 base their approaches in the continuous improvement cycle developed by Shewhart and disseminated worldwide by Deming, also known by plan-do-check-act. Its third phase – check – is when organizations collect and analyze information to determine if activities have been performed as planned and envisioned results are being achieved.
EQAVET is not as prescriptive as ISO 21001 regarding how organizations should implement these evaluations, but ISO 21001 has specific requirements for internal audits to be performed, suggesting, inclusively, that the good practices on how to audit management systems, contained in ISO 19011, are used as guideline.
Criteria
EQAVET
| 3. Evaluation of outcomes and processes is regularly carried out and supported by measurement |
| 3.1. A methodology for evaluation has been devised, covering internal and external evaluation |
| 3.2. Stakeholder involvement in the monitoring and evaluation process is agreed and clearly described |
ISO 21001
| 9.1.1 General |
| 7.2.1 c) |
| 9.1.2.1 Monitoring of satisfaction |
| 9.1.3 Other monitoring and measuring needs |
| 9.1.4.2 c) |
| 9.1.5 Analysis and evaluation |
| 9.2 Internal audit |
| 9.3.2 Management review inputs |
VET21001 Tools
The set of Templates provided as tools to help implement this cluster are complementary to each other, with information entered in one usually being used by another. For this reason, in this cluster our recommendation is for users to adopt them as a set and not only one or two, which might disrupt the information flow.
Template for Internal Audit Plan
The template for internal audit plan was designed in line with the recommendations in ISO 19011 Guidelines for auditing management systems, which is internationally consensually considered good practices for auditing. It therefore contains fields to characterize an audit in terms of audit context (type, criteria, objectives, scope, duration and audit team identification), as well for a detailed description of the planned activities, such as which processes are expected to be audited in each time slot, by which audit team members and towards which criteria and which auditees are expected to participate. The allocation of processes to criteria and the identification of expected interviewees has two added values:
- triggers the curiosity of the auditees to (re)read the criteria applied to the processes they’re responsible for or collaborate with, which has a pedagogic effect;
serves as quick-reference additional check list to guide the audit team during the audit and allowing for self-monitoring of their effectiveness and efficiency.
Download this tool in editable format
Download an example of the use of this tool
Template for Audit Checklist
The template for audit checklists is supposed to be filled in two different moments:
- Before the audit: columns audit criteria should be filled before the audit and allows for a more detailed and personalized view on the criteria to be used by each auditor. If in the audit plan the criteria are usually identified by their number, here is can be identified with the content of the criteria in full (useful option for inexperienced or too experienced auditors), or with criteria transformed in questions or itineraries of expected objective evidence to be seek; or with key words or in any other way that facilitates the auditor’s job. Different auditors have different auditing styles and different needs, and audit checklists should be personalized to fit them.
- During the audit: the remaining columns should be filled while the audit is being conducted. At the field “Type of Finding” auditors are expected to categorize the finding (non-conformity, risk, opportunity for improvement, conformity, strong point) and at the filed “Supporting evidence” they should list the number of the evidence they collected (and registered at the Template for audit evidence list with a sequential number) that supports their judgement regarding that finding. In the field “Auditor’s notes” auditors can pre-draft the text of the finding as it will appear in the audit report and add any other information that might be useful for the audit team, such as those needed to triangulate information to validate the evidences. This allows for the traceability of audit findings, the justification of audit judgement and assures the transparency of the process.
Download this tool in editable format
Download an example of the use of this tool
Template for Audit Interviewees List
The template for audit interviewees list is a simple template for auditors to use while conducting an audit, where they can take note of the name and job role of each auditee they interview. The information entered in this template will support as needed the discussions held during audit team meetings and will feed the audit report.
Download this tool in editable format
Download an example of the use of this tool
Template for Audit Evidence List
The template for audit evidence list is used by auditors while conducting the audit and is meant to be a factual register of all objective evidences collected during the audit. It contains fields to describe the evidence and also to categorize it as collected by interview, by observation of by analyses of documented information. It is meant to be numbered in the same sequential order in which they were collected. This numbering will be reflected in the audit checklist and will avoid auditors to have to write more than once the description of a given objective evidence in the checklist, when it serves to justify the audit judgement toward more than one criteria.
The information entered in this template will support as needed the discussions held during audit team meetings and will feed the audit report.
Download this tool in editable format
Download an example of the use of this tool
Template for Audit Report
The template for audit report was designed in line with the recommendations in ISO 19011 Guidelines for auditing management systems, which is internationally consensually considered good practices for auditing. It therefore contains sections to:
- incorporate the audit plan, which will contextualize the audit and provide details for what happened at any given moment;
- identify the audit sample used in the audit;
- provide a brief summary of how the audit was conducted;
- provide a brief summary and also a detailed description of audits results – the set of findings arising from the comparison between the audit criteria and the objective evidences collected during the audit;
- state the audit conclusions – the audit judgement arising from the comparison between the audit objectives and the audit results;
- incorporate the audit evidences list.
The information contained in the audit interviewee list can be incorporated in the audit report in different ways: in the section “audit sample” as sample of staff interviewed; as entrances in the category of interviews in the audit evidence list, as an annex to the audit report (best option if signatures are collected), etc..
The audit checklist can also be added to the audit report as annex. It is very useful for the auditees as a pedagogic material matrixing audit criteria, evidences and findings. However, checklists should only be included in the annex if they are readable and do not contain notes with subjective emotional judgements (e.g. a note stating “the management review record was in a terrible mess” is not acceptable. A note stating “the management review record didn’t cover all input information required by ISO 21001 and EQAVET” is acceptable).
Download this tool in editable format
Download an example of the use of this tool
