For the Quality System to function properly, there must be a mechanism to monitor compliance with the pre-defined expectations. In the EQIPD quality system, spot checks play such a role.
A spot check is a focused review of a selection of experimental records, reports, procedures or the lab environment. Spot checks are performed either random or at scheduled timepoints against predefined expectations and performance standards.They are typically conducted by a Process owner or someone to whom this task is delegated. EQIPD does not require that the research unit maintains documentation on spot checks conducted and leaves it up to the Process owner and the research unit to decide how often spot checks are conducted, how the outcome is evaluated, discussed, reported or followed up. Some research units opt to build key performance indicators that quantify the outcome of the spot checks of key processes.
If a research unit undergoes an internal or external assessment, the assessors may also conduct spot checks to evaluate the overall performance and to identify processes or specific examples for an in-depth discussion with the research unit.
Examples for spot checks
Spot checks can be done to see how well the RU is doing by finding answers to questions such as:
- How easy is it to retrieve and reconstruct research data? (e.g. Core Requirement 8)
- The predefined expectation is the ability to find data from a publication or lab meeting. Hence, the process owner (or any dedicated person) could take a presentation from the last lab meeting and focus on one figure with results from an experiment. The person performing the spot check will then try to find the raw data and understand different stages of the experiment as well as the analysis. The guiding principle is the ability to understand the experiment to a degree that another researcher could perform it. If that is possible, the spot check was successful. If that is not possible, it points to the need for change, which could be a training for researchers documenting experiments or even establishing a systematic approach for traceability. However, there is no need to document such spot checks.
- Are study protocols completed prior to the start of the experiment? (e.g. Core Requirement 10)
- Study protocol should be written before the start of the study. This becomes a requirement for knowledge-claiming research which is part of CR10, whereas it might be useful to also perform spot checks on this. The spot check might look very different in research environments. In cases of an established ELN with a capability to lock study designs, it could be periodically double checked whether the date of finalizing the study design indeed fits the expectation. However, if such a system does not exist, this could also be done during personal meetings when study protocols are routinely discussed to see whether all requirements for the study are implemented and the study finalized.
- Have data been generated in an unbiased fashion? (e.g. Core Requirement 10)
- Reducing bias is one of the goals of the EQIPD quality system and spot checks on the experimental process shall support this endeavor. For example, it can be discussed during lab meetings whether requirements associated with knowledge-claiming research (CR10) are implemented. These two examples can be considered as a spot check.
- Have all results been reported? (e.g. Core Requirement 9)
- In the same line as described above, during meetings it could be pointed towards the importance of reporting all data. Additional data sets showing different results could also be discussed in such a setting.
- Are current method protocols available to all team members?
- For transparency reason, it is considered to be important that lab protocols are always of sufficient detail and available to all lab members. If an ELN is in place, the process owner could check that access rights are appropriately managed and up-to-date.
- Are reagents stored correctly? (Core requirement 13)
- This must not be a formal process as well but can be done when occasionally walking through the lab to talk to someone. Just having an open eye and awareness for reagents left on the bench that actually should be stored at a different temperature can be helpful.
- Are data records completeness? (Core requirement 6)
- Such spot checks can be done during one-on-one personal meetings when scientific data are discussed and it is occasionally looked at the primary notes and data by another person who will also have an eye the general documentation practices and whether it looks complete. However, such spot checks could also be done systematically with a check list by the process owner who looks randomly into different data records.