Difference between revisions of "2.1.7 Blinding"

From EQIPD
Jump to: navigation, search
(C. Resources​)
 
(16 intermediate revisions by 5 users not shown)
Line 1: Line 1:
​​​THIS ITEM IS UNDER CONSTRUCTION
+
== A. Background & Definitions ==
  
​​​
 
  
 +
'''Blinding''' refers to the masking of the treatment, physiological or pathological condition, genotype or any prior intervention for the person(s) performing the experiment, collecting data and assessing outcome. Blinding aims to make sure that each person involved has no information on the experimental units (animals, subjects or samples) in each experimental group that could systematically influence his/her performance. The intended result is that (as far as possible) all experimental units in the experiment are dealt with equally.
  
== A. Background & Definitions ==
+
In the discussion below, experimental groups refers to '''all''' groups within an experiment, for example: control, sham, treated with drug A, treated with drug B, etc.
 +
 
 +
Group allocation describes which experimental unit (animal, subject or sample) has been allocated to which experimental group.
 +
 
 +
The group allocation, actions and outcome assessments are ‘'''blinded'''’. People are ‘'''blind'''’ to particular information.
  
 +
Blinding requires at least 2 people, one blinded person (unaware of experimental condition) and a non-blinded person (knows the experimental condition and the blinding code). The non-blinded person is the keeper of the blinding code, which needs to be concealed until all processes under blinding are concluded.
  
'''Blinding''' refers to the masking of the treatment allocation for person(s) who perform the experiment, collect data and assess outcome. Blinding aims to make sure that someone has no knowledge about treatment allocation that may systematically influence his/her performance. The intended result is the equal treatment (as far as possible) of all experimental units (animals, subjects or samples) in the experiment.
+
The most effective blinding covers every step in an experiment - from allocation to treatment conditions, application of treatment to data collection and analysis - this is often referred to as '''full blinding'''.
  
In the discussion below, experimental groups refer to '''all''' groups involved in an experiment, for example: control, sham, treated with drug A, treated with drug B, etc.
+
Blinding should '''not''' be seen as "all or none". There are several situations in which partial blinding may be applied (i.e., blinding of the most risk-prone step(s) in the experimental process). For example, partial blinding can be considered when:
 +
* a research unit with no prior experience with blinding is introducing a blinding procedure and, for organizational or other reasons, follows a step-wise implementation
 +
* a research unit has significantly constrained human resources and does not intend to conduct knowledge-claiming research
  
Group allocation describes which experimental unit (animal, subject or sample) has been allocated to which experimental group.
+
In all cases, however, full and transparent reporting of how blinding was applied is expected.
  
The group allocation, actions and outcome assessments are ‘'''blinded'''’. People are ‘'''blind'''’ to particular information.
 
  
 
== ​B. Guidance & Expectations ==
 
== ​B. Guidance & Expectations ==
  
Effective blinding requires at least 2 people, one blinded person (unaware of experimental condition) and an unblinded person (knows the experimental condition and the blinding code). The unblinded person is the keeper of the blinding code which needs to be concealed until all processes under blinding are concluded.
 
  
​​Describe who was aware of the group allocation at the different stages of the experiment (during the allocation, the conduct of the experiment, the outcome assessment, and the data analysis).​ Any process using humans as perceptors, raters or interpreters needs to be blinded as long as decision-making is concluded.
+
​​EQIPD expects that the method(s) used to implement blinding are described in as much detail as possible:
 +
* either as a dedicated protocol (please see below for a template that may serve as an example of how to build such a protocol),
 +
* or as a separate section of a study protocol.
  
Allocation concealment in animal experiments
+
Depending on the breadth of research methods in use, a given research unit may have one or more blinding protocols that can support blinding for specific types of experiments.
  
The investigator shall not be aware and/or have the choice to which treatment group an animal is allocated to. This prevents selection bias. Therefore, the assignment to a specific group needs to be concealed and every animal should have the same chance to be assign to each of the groups. This can be achieved separating the assignment of animal_IDs to each animal (e.g. individual ear mark or subcutaneous chip) and randomization of treatments (see randomization) into two independent processes and then merge the two.
+
When preparing a blinding protocol, the main objective is to have a description that is understandable for the actual users - i.e., bench scientists (especially those that are new to the unit). Therefore, it should be written in a simple language with as many examples (specific to the research) as appropriate.
  
Effective blinding code
+
A blinding protocol may describe the following:
  
There are certain strategies to apply blinding. The most simple one is the blinding by numbers or alphabet letters. This can be problematic when human processing and rating is involved in outcome assessment. The assessor may not know the condition behind the code but the knowledge of a group affiliation of a sample is a bias that can influence rating. This should be avoided.
+
* Training and competence
 +
** is there any training needed?
 +
** are there any additional supporting tools or materials available?
  
Sometimes blinding conditions are added to another code such as animal_ID, measurement_ID or file name. Watch out for hidden cues in such IDs, containing temporal or sequential information that could increase rater bias. Also metadata, such as creation date and time of a file containg measurements can give away experimental conditions. 
+
* Feasibility assessment (to avoid applying blinding when it makes no sense or would actually do harm)
 +
** how high is the risk of unintentional unblinding?
 +
** are the required resources available?
 +
** are emergency scenarios considered?
  
If possible, check whether it is possible to use a blinding scheme without repeating codes. This is can be easily done with alphanumeric code consisting of 4 letter/number combinations, such as T7Z4. Such codes can be easily generated in Excel​
+
* "Who does what?"
 +
** сlearly define the roles for those involved in the experiment and the blinding procedure (e.g., see section 4 in the [https://paasp.sharepoint.com/:w:/s/EQIPD/EZbRvZmZGoRGtsjc1Wk_XOsBLsnNAbg-FBFVj9h199oYMA?e=vIXtkO blinding protocol]);
 +
** the blinding protocol should make clear who is aware of the group allocation at the different stages of the experiment (during the allocation, the conduct of the experiment, the outcome assessment, and the data analysis);
 +
** to effectively blind a study, create a sequence containing all experimental steps of the study and, for each step, indicate the name of each person involved in the conduct and analysis of the study. For each experimental step, document for each person whether they are blinded or not blinded to the condition. Such an overview systematically creates a transparent workflow of blinded and unblinded personnel and shows at which step unintended unblinding might occur. Such an overview (e.g., as a table) can be made part of the experimental documentation and reporting.
 +
** it is generally expected and strongly recommended that any process using humans as perceptors, raters or interpreters needs to be blinded until the decision-making is concluded.
  
=CHAR(RANDBETWEEN(65;))&RANDBETWEEN(0;9)&CHAR(​RANDBETWEEN(65;))&RANDBETWEEN(0;9)
+
* Blinding code
 +
** describe how the blinding code is developed and which specific steps are taken to practically apply it
 +
** one simple blinding strategy is to assign each subject / sample a separate number or letter (or a combination thereof). This approach may create compliance issues with large numbers of subjects / samples and the need to administer treatment repeatedly over extended periods of time.
 +
** another blinding strategy is to assign each experimental group a separate number or letter (or a combination thereof). This approach may be problematic when human processing and rating is involved in outcome assessment. The assessor may not know the condition behind the code but the knowledge of a group affiliation of a sample can influence rating.
 +
** the decision of which strategy to follow is made by the researchers, taking into account the specifics and associated risks for each experiment.
  
Enter this formula in a row of cells for which you need coded samples. Copy the outcome to another worksheet with the command Past Special-->Paste Values.
 
  
Planning to blind a study
+
'''PRACTICAL TIPS'''
  
To effectively blind a study, create a sequence containg all experimental steps of the study. In the neighboring headers indicate the name of each person involved in the conduct and analysis of the study. For each experimental step document for each person whether they are blinded or not blinded to the condition or not involved at all. Drawing this out systematically creates a transparent workflow of blinded and unblinded personnel and shows when unintended unblinding might occur. Add the final table to all experimental documentation and reporting.  
+
* Generation of alphanumeric codes for blinding
 +
** if possible, use blinding schemes with no repeating codes. This can be easily done with an alphanumeric code consisting of 4 letter/number combinations, such as T7Z4. Such codes can be generated in Excel​ using the following formula:
 +
*** =CHAR(RANDBETWEEN(65;))&RANDBETWEEN(0;9)&CHAR(​RANDBETWEEN(65;))&RANDBETWEEN(0;9)​
 +
*** enter this formula in a row of cells for which you need coded samples and copy the outcome to another worksheet with the command Past Special-->Paste Values.
  
 +
* Allocation concealment in animal experiments
 +
** to prevent selection bias, the investigator shall not be aware and/or have the choice of which treatment group an animal is allocated to;
 +
** therefore, the assignment to a specific group needs to be concealed and every animal should have the same chance to be assigned to each of the groups;
 +
** this can be achieved by separating the assignment of animal_IDs to each animal (e.g., individual ear mark or subcutaneous chip) and randomization of treatments (see randomization) into two independent processes and then merging the two.
  
  
 
'''RISK ASSESSMENT'''
 
'''RISK ASSESSMENT'''
* Under some circumstances, unintentional unblinding (e.g. due to a different appearance of a positive control in solution or suspension) may be a risk to be assessed and/or controlled
+
* Under some circumstances, unintentional unblinding (e.g., due to a difference in appearance of a positive control in solution or suspension) may be a risk to be assessed and/or controlled
* Experimental treatments may produce adverse effects and attending veterinarians and animal care stuff may need to be informed in advance about the possibility of such adverse effects occurring and, if necessary, have emergency access to the blinding protocol.
+
* Experimental treatments may produce adverse effects and attending veterinarians and animal care stuff may need to be informed in advance of the possibility of such adverse effects occurring and, if necessary, have emergency access to the blinding protocol.
 +
* if a blinding code is added to another code such as animal_ID, measurement_ID or file name, watch out for hidden cues in such IDs, containing temporal or sequential information that could increase rater bias. Also metadata, such as creation date and time of a file containing measurements can give away experimental conditions. 
  
  
 
'''PLEA​​SE DO NOT FORGET​'''
 
'''PLEA​​SE DO NOT FORGET​'''
  
* Blinding is sometimes not possible especially when certain cues cannot be blinded, such as skin color of transgenic mice or color of a solution in a well. It is important to document this and to communicate in reports where blinding could or could not be achieved.
+
* Blinding is sometimes not possible especially when certain cues cannot be blinded, such as skin color or weight/size of transgenic mice or color of a solution in a well. It is important to document this and to communicate in reports where blinding could or could not be achieved.
* ​Unblinding of the experimental conditions should be done when all blinded processes for the entire study are concluded. Early and partial unblinding for "checking" should be avoided and if necessary, be part of the protocol.
+
* ​Unblinding of the experimental conditions should be done when all blinded processes for the entire study are concluded. Early and partial unblinding for "checking" should be avoided and, if necessary, be part of the study protocol.
 
* Control group(s) (e.g., positive control group) should not be excluded from the blinding procedure.
 
* Control group(s) (e.g., positive control group) should not be excluded from the blinding procedure.
 
* Provide training on how to apply the blinding procedure.
 
* Provide training on how to apply the blinding procedure.
Line 61: Line 88:
 
Guidelines on reporting of blinding (in vivo research):
 
Guidelines on reporting of blinding (in vivo research):
 
* [[ARRIVE 2.0]] ​​​
 
* [[ARRIVE 2.0]] ​​​
 +
 +
Template to develop a written description of the method used to implement blinding:
 +
* [https://paasp.sharepoint.com/:w:/s/EQIPD/EZbRvZmZGoRGtsjc1Wk_XOsBLsnNAbg-FBFVj9h199oYMA?e=vIXtkO blinding protocol]
  
 
Reading material:
 
Reading material:

Latest revision as of 19:43, 23 March 2021

A. Background & Definitions

Blinding refers to the masking of the treatment, physiological or pathological condition, genotype or any prior intervention for the person(s) performing the experiment, collecting data and assessing outcome. Blinding aims to make sure that each person involved has no information on the experimental units (animals, subjects or samples) in each experimental group that could systematically influence his/her performance. The intended result is that (as far as possible) all experimental units in the experiment are dealt with equally.

In the discussion below, experimental groups refers to all groups within an experiment, for example: control, sham, treated with drug A, treated with drug B, etc.

Group allocation describes which experimental unit (animal, subject or sample) has been allocated to which experimental group.

The group allocation, actions and outcome assessments are ‘blinded’. People are ‘blind’ to particular information.

Blinding requires at least 2 people, one blinded person (unaware of experimental condition) and a non-blinded person (knows the experimental condition and the blinding code). The non-blinded person is the keeper of the blinding code, which needs to be concealed until all processes under blinding are concluded.

The most effective blinding covers every step in an experiment - from allocation to treatment conditions, application of treatment to data collection and analysis - this is often referred to as full blinding.

Blinding should not be seen as "all or none". There are several situations in which partial blinding may be applied (i.e., blinding of the most risk-prone step(s) in the experimental process). For example, partial blinding can be considered when:

  • a research unit with no prior experience with blinding is introducing a blinding procedure and, for organizational or other reasons, follows a step-wise implementation
  • a research unit has significantly constrained human resources and does not intend to conduct knowledge-claiming research

In all cases, however, full and transparent reporting of how blinding was applied is expected.


​B. Guidance & Expectations

​​EQIPD expects that the method(s) used to implement blinding are described in as much detail as possible:

  • either as a dedicated protocol (please see below for a template that may serve as an example of how to build such a protocol),
  • or as a separate section of a study protocol.

Depending on the breadth of research methods in use, a given research unit may have one or more blinding protocols that can support blinding for specific types of experiments.

When preparing a blinding protocol, the main objective is to have a description that is understandable for the actual users - i.e., bench scientists (especially those that are new to the unit). Therefore, it should be written in a simple language with as many examples (specific to the research) as appropriate.

A blinding protocol may describe the following:

  • Training and competence
    • is there any training needed?
    • are there any additional supporting tools or materials available?
  • Feasibility assessment (to avoid applying blinding when it makes no sense or would actually do harm)
    • how high is the risk of unintentional unblinding?
    • are the required resources available?
    • are emergency scenarios considered?
  • "Who does what?"
    • сlearly define the roles for those involved in the experiment and the blinding procedure (e.g., see section 4 in the blinding protocol);
    • the blinding protocol should make clear who is aware of the group allocation at the different stages of the experiment (during the allocation, the conduct of the experiment, the outcome assessment, and the data analysis);
    • to effectively blind a study, create a sequence containing all experimental steps of the study and, for each step, indicate the name of each person involved in the conduct and analysis of the study. For each experimental step, document for each person whether they are blinded or not blinded to the condition. Such an overview systematically creates a transparent workflow of blinded and unblinded personnel and shows at which step unintended unblinding might occur. Such an overview (e.g., as a table) can be made part of the experimental documentation and reporting.
    • it is generally expected and strongly recommended that any process using humans as perceptors, raters or interpreters needs to be blinded until the decision-making is concluded.
  • Blinding code
    • describe how the blinding code is developed and which specific steps are taken to practically apply it
    • one simple blinding strategy is to assign each subject / sample a separate number or letter (or a combination thereof). This approach may create compliance issues with large numbers of subjects / samples and the need to administer treatment repeatedly over extended periods of time.
    • another blinding strategy is to assign each experimental group a separate number or letter (or a combination thereof). This approach may be problematic when human processing and rating is involved in outcome assessment. The assessor may not know the condition behind the code but the knowledge of a group affiliation of a sample can influence rating.
    • the decision of which strategy to follow is made by the researchers, taking into account the specifics and associated risks for each experiment.


PRACTICAL TIPS

  • Generation of alphanumeric codes for blinding
    • if possible, use blinding schemes with no repeating codes. This can be easily done with an alphanumeric code consisting of 4 letter/number combinations, such as T7Z4. Such codes can be generated in Excel​ using the following formula:
      • =CHAR(RANDBETWEEN(65;))&RANDBETWEEN(0;9)&CHAR(​RANDBETWEEN(65;))&RANDBETWEEN(0;9)​
      • enter this formula in a row of cells for which you need coded samples and copy the outcome to another worksheet with the command Past Special-->Paste Values.
  • Allocation concealment in animal experiments
    • to prevent selection bias, the investigator shall not be aware and/or have the choice of which treatment group an animal is allocated to;
    • therefore, the assignment to a specific group needs to be concealed and every animal should have the same chance to be assigned to each of the groups;
    • this can be achieved by separating the assignment of animal_IDs to each animal (e.g., individual ear mark or subcutaneous chip) and randomization of treatments (see randomization) into two independent processes and then merging the two.


RISK ASSESSMENT

  • Under some circumstances, unintentional unblinding (e.g., due to a difference in appearance of a positive control in solution or suspension) may be a risk to be assessed and/or controlled
  • Experimental treatments may produce adverse effects and attending veterinarians and animal care stuff may need to be informed in advance of the possibility of such adverse effects occurring and, if necessary, have emergency access to the blinding protocol.
  • if a blinding code is added to another code such as animal_ID, measurement_ID or file name, watch out for hidden cues in such IDs, containing temporal or sequential information that could increase rater bias. Also metadata, such as creation date and time of a file containing measurements can give away experimental conditions.


PLEA​​SE DO NOT FORGET​

  • Blinding is sometimes not possible especially when certain cues cannot be blinded, such as skin color or weight/size of transgenic mice or color of a solution in a well. It is important to document this and to communicate in reports where blinding could or could not be achieved.
  • ​Unblinding of the experimental conditions should be done when all blinded processes for the entire study are concluded. Early and partial unblinding for "checking" should be avoided and, if necessary, be part of the study protocol.
  • Control group(s) (e.g., positive control group) should not be excluded from the blinding procedure.
  • Provide training on how to apply the blinding procedure.

C. Resources​

Guidelines on reporting of blinding (in vivo research):

Template to develop a written description of the method used to implement blinding:

Reading material:



back to Toolbox

Next item: 2.1.8 Randomisation