ORIGINAL ARTICLE
Content validity of a structured tool: knowledge questionnaire on behavioural problems
Nabanita Barman1, Mridula Saikia Khanikor2
1Lecturer, Regional College of Nursing, Guwahati, Assam, India, 2Vice Chancellor, Indira Gandhi Technological And Medical Science University, Monipolyang Campus, Monipolyang, Ziro, Arunachal Pradesh, India
Abstract
Background: Children can have different types of behavioural problems, i.e. inattention, restlessness, anxiety, sadness, fearfulness, lying, stealing, etc., which may lead to development of mental disorder if neglected at its early stage. The prevailing status of mental health problems among children requires early identification and management where school teachers can take an active role in a poor health resource country like India. Therefore, it is of utmost importance to assess the knowledge of school teachers on behavioural problems of children. Review of literature helps in constructing the tool for knowledge assessment which requires validity and reliability testing. This study aims at evaluating content validity of such a structured tool developed to assess the knowledge of school teachers on behavioural problems. Method: The structured knowledge questionnaire consists of 30 items. It has four domains of behavioural problems, i.e. meaning, causes, characteristics, and management of behavioural problems. The questionnaire is given to 18 number of experts from the fields of psychiatry, psychiatric nursing, clinical psychology, counseling psychology, education, sociology, social and preventive medicine, and biostatistics for their valuable input and experts are requested to complete within one month. Result: The content validity of the knowledge questionnaire is found to be excellent both at item level and scale level (I-CVI≥0.78 and S-CVI=0.99). This suggests that the structured knowledge questionnaire shows acceptable validity. Conclusion: The testing of validity of a structured questionnaire is very important in social and health science research as it gives confidence to readers about the tool. In addition, it is also mandatory to check the reliability of the tool which can be done in the future for the present study.
Keywords: Child. School Teachers. Research.
Correspondence: Nabanita Barman, M.Sc. in Psychiatric Nursing, PhD Scholar (Srimanta Sankaradeva University of Health Sciences, Guwahati), Lecturer, Regional College of Nursing, Guwahati-781032, Assam, India. nitubarman@yahoo.in
Received: 21 June 2018
Revised: 13 February 2019
Accepted: 14 February 2019
Epub: 5 March 2019
DOI: 10.5958/2394-2061.2019.00031.4
INTRODUCTION
Behaviour is the way one acts in conjunction with their environment. So, behavioural problem occurs when one cannot adjust with his or her surroundings. Behavioural problems can affect any age group. But, the most concerned group is the child group as a child’s health is the cornerstone of national progress. The causes of behavioural problems are multifactorial like prenatal, postnatal factors, factors during delivery, maladjustment, faulty emotional environment, pathological relationship, etc.[1-3]. The common behavioural problems of children are attention deficit, hyperactivity, conduct problem, aggression, depression, anxiety, nail biting, bed wetting, etc. It has been found from the various epidemiological studies done in different parts of the world that the range of childhood mental disorders is 12-51% though accurate estimate is difficult to find out.[4] In India, the overall point prevalence rate of childhood behavioural problems is 12.5% for children aged zero to 16 years whereas 7.90 to 16.78% in Assam.[5,6] Again, a study conducted in Assam shows that the prevalence of childhood behavioural problem, i.e. attention-deficit/hyperactivity disorder (ADHD) is 12.66% among children aged six to 11 years.[7] The high prevalence of childhood mental health problems demands their early identification and intervention.
Again, children spend a large portion of each day in school. So, it is the main responsibility of the school not only to build them intellectually capable but also to develop their physical and mental health. Teacher can play a very important role in early diagnosis of mental health problems, referring them to proper medical professional, and also, involving in various mental health promotional activities in the school. It demands more knowledge of mental health on the part of teacher. Knowledge assessment of school teacher is the preliminary part which gives the way for knowledge enhancement programme on identification and early management of behavioural problems. For these reasons, the researchers are interested to assess the knowledge of primary school teachers on behavioural problems.
Now, the next question is how the knowledge will be assessed. Literature review suggests no such appropriate research tool which caters the need for primary school teacher. This motivates the researchers to develop a structured questionnaire on behavioural problems for the assessment of knowledge of primary school teacher. To be standardised, a research tool must be valid and reliable. Therefore, the researcher has undertaken this study to find out the content validity of structured knowledge questionnaire on behavioural problems.
When a researcher develops a new scale, it is expected to provide detailed information regarding the reliability and validity of the scale. Validity of a scale refers to the degree to which a scale measures what it is supposed to be measuring. Content validity is one of the types of validity which is very important to know the quality of the scale.
Content validity is concerned with scope of coverage of the content area to be measured. It is mostly applied in the test of knowledge assessment.[8] It is the degree to which a study tool has sample items needed for the construct that is being measured: “Whether or not the items sampled for inclusion on the tool adequately represent the domain of content addressed by the instrument”.[9]
It is a case of judgement of experts about the content area of the research scale to measure a particular phenomenon. Judgement of the content validity is based on previous researchers’ and experts’ opinion about the content of research scale.
The content validity of knowledge questionnaire on behavioural problems is the extent to which its components are relevant to the underlying construct (i.e. behavioural problems) which is likely to be effective in achieving the particular purpose (i.e. knowledge assessment) in a targeted population (i.e. primary school teachers). There are three core dimensions for expert review which are relevance, adequateness, and appropriateness. Relevance is the extent to which a questionnaire is pertinent to its intended target as defined in the study. Adequateness is the extent to which a questionnaire is sufficient to achieve the target. Appropriateness is the extent to which the questionnaire is suitable or proper for a specific targeted population which may be defined by age, sex, culture, or other factors.
Theoretical explanation of methods of content validity
Content validity of a research tool involves two distinct phases: instrument development and expert judgement.[10-12]
Phase 1: Instrument design
The instrument design has three steps: determination of content domain, item generation, and instrument construction.[10]
Determination of content domain is the first step which determines the content domain of a construct that the instrument is developed to measure it. Content domain is the content area of the instrument. It requires strong conceptualisation of construct. It can be done by literature review related to the topic, interview with relevant populations, and discussion with experts.[13]
The second step is item generation which is formulated keeping in mind the research questions so that items are relevant to research questions.
Instrument construction is the third step of instrument design. Here, items generated from content domain are refined and the finalised items are organised sequentially in a suitable usable format.
Phase 2: Expert judgement
The constructed instrument is then confirmed by a panel of experts who is asked to rate each tool or instrument item in terms of relevance to the underlying construct. Experts should have relevant content knowledge. Subjects of target group can also be used as experts to represent the population for whom the tool is being developed. To be in proper balance, experts should be diverse in discipline and geography. A minimum three experts are needed for content validity and it is probably unnecessary to appoint more than ten experts.[11,12]
Quantification of content validity
The data of content validity can be analysed in both qualitative and quantitative way.
In qualitative analysis, the recommendation given by the experts are incorporated and tool is modified in terms of grammar, use of correct words and order of words, appropriate scoring.
In quantitative analysis, the most widely used method is calculation of content validity index (CVI). “CVI is the degree to which an instrument has an appropriate sample of items for construct being measured”. Computation of CVI is of two types: content validity of individual item (I-CVI) and content validity of the overall scale (S-CVI).[11,12]
For calculation of I-CVI, experts are asked to rate each tool item in terms of its relevance to the underlying construct. I-CVI is the number of experts who rates the item as relevant divided by the total number of experts. I-CVI should be 1.00 when there is five or less experts. If experts are six or more than six, the recommended I-CVI is not less than 0.78.[11]
S-CVI is defined as “the proportion of total items judged content valid”.[12] There are two methods for calculating it which are universal agreement among experts (S-CVI/UA) and average the item level CVIs (S-CVI/Ave).
S-CVI/UA is proportion of items on a scale that achieves a relevance rating by all the experts. Here, the number of items considered relevant by all the judges is divided by the total number of items. It demands 100% agreement that is practically difficult. On the other hand, S-CVI/Ave is the average of the I-CVIs for all items on the scale. That means, sum of I-CVIs is divided by the total number of items. A minimum S-CVI of 0.80 is recommended. To have excellent content validity, the S-CVI/Ave should be 0.90 or higher.[13,14]
METHODS
Phase 1: Development of the knowledge questionnaire
For the selection of content domain of the knowledge questionnaire on behavioural problems, extensive review of literature was done. Researchers had also discussed with the representatives of primary school teachers. Finally, the researchers had decided to select four content domains which were meaning, causes, characteristics, and management of behavioural problems.
In the second step, i.e. item generation step, 40 questions were developed from these dimensions. These items were checked for duplication and overlapping. The overlapping items were deleted.
Finally, the questionnaire was developed with 30 number of items with four domains. Each domain had different number of items (Table 1). The items of the primary questionnaire are listed below.
1) The term ‘behaviour’ means the way a person responds to certain situation or experience.
2) Behavioural problem is the behaviour that is not acceptable according to the situation.
3) Children never have behavioural problems.
4) Behavioural problems can be inherited.
5) Behavioural problems are more likely in dysfunctional families.
6) Behavioural problems can often be caused by sugar or food additives.
7) Aggressive behaviour in a child is caused due to negligence and abuse.
8) Difficult pregnancies, premature birth, and low birth weight may contribute in some cases to the child’s problem behaviour later in life.
9) Diseases affecting the brain can lead to development of behavioural problems.
10) Poor nutrition is also a cause of behavioural problem in children.
11) One example of behavioural problem is aggressiveness or violent.
12) A child with behavioural problem has frequent emotional outburst and minor things bother him or her.
13) Behavioural problem child may display destructive behaviour like hitting, throwing things, screaming, etc.
14) Children with behavioural problem are depressed or anxious.
15) Behavioural problems children are always obedient and listen to others.
16) Children are very friendly or have good friends.
17) Behavioural problems never affect the performance of child at school.
18) Children may be lying more often and may also involve in stealing.
19) They are unable to focus on one thing, get restless, extremely lazy, or disoriented.
20) Behavioural problems can be diagnosed by in-depth interviews with parents, child, and teacher.
21) It can only be treated with medication.
22) If medication is prescribed, educational interventions are often unnecessary.
23) The behavioural intervention of a child with behavioural problems should begin as early as possible.
24) Parent management training is one of the treatment options for behavioural problems.
25) Teacher should make sure that the student stays away from his or her peers when teaching students with behavioural problems.
26) When a child shows aggressive or violent behaviour, a teacher should scold the child to control the behaviour.
27) To prevent childhood behavioural problem, give the child proper love and attention.
28) When a child shows attention deficit, give the child high level of cognitive task.
29) When you find a child stealing his friend’s pencil, ignore the issue.
30) The current trend of placing students with behavioural problems is moving towards inclusion in the general education classroom.
Phase 2: Expert judgement on the knowledge questionnaire
The researchers had selected 18 experts who have minimum ten years of experience for the creation of expert panel. The experts were from the field of psychiatry (four experts), psychiatric nursing (three experts), clinical psychology (three experts), counselling psychology (two experts), education (two experts), sociology (one expert), social and preventive medicine (one expert), and biostatistics (two experts). The experts were contacted personally and a requesting letter for content validity was given which included study objective, synopsis, developed instrument, scoring key, the required instruction for responding, and content validity certificate. The experts were requested to judge the questionnaire in terms of relevancy, adequateness, and appropriateness within a period of one month.
The questionnaire from all experts were collected within approximately two months and analysed. The experts had given suggestion in terms of using simple words instead of technical words (e.g. depressed, aggression, high of cognitive task), correction of grammar, sentence pattern. Finally, the items were modified and beautified as per experts’ suggestion keeping in mind the same content of those items. The modified items were item number 8, 11, 13, 14, 15, 16, 18, 19, 21, 22, 24, 25, 27, 28, 29, 30.
RESULTS
The content validity of the knowledge questionnaire was assessed by calculating content validity index both item level and scale level.
Table 2 shows that item level content validity index (I-CVI) for each item of the knowledge questionnaire was ≥0.78. The scale level content validity index (S-CVI) was calculated by both universal agreement and average method. S-CVI/Ave can be calculated in three different ways which always yields the same results.[11] Accordingly, the S-CVI/Ave for the knowledge questionnaire was estimated with the data shown in Table 2. The details of the calculation are explained as follows:
- Calculation by averaging the proportion of items rated as relevant across the experts. So, S-CVI/Ave was (1+1+1+1+1+1+1+0.90+0.96+0.96+1+1+1+1+1+1+1+1)/18=17.82/18=0.99
- S-CVI/Ave = Average of I-CVI, i.e. summation of all I-CVIs and dividing it by total number of items, i.e. 29.68/30=0.989=0.99
- Another way is to count total number items rated relevant by all experts and then divided by total number of ratings. For the knowledge questionnaire, the total number of items rated relevant was 535 and total number of ratings was 540. So, the S-CVI/Ave was 535/540=0.99.
Again, the calculated value of S-CVI/UA for the knowledge questionnaire was 0.90.
From the findings, it has been seen that the content validity of the structured knowledge questionnaire on behavioural problem of children is excellent both at item and scale level.
DISCUSSION
The present paper describes the process of content validity of a new tool or instrument, i.e. content validity of structured questionnaire on behavioural problems developed specially for primary school teacher. CVI of the knowledge questionnaire is found to be excellent both at item level and scale level.
It has some limitations too. As the opinion of the experts is subjective, the study is subjected to experts’ biases. So, experts’ opinion cannot provide guarantee of the effectiveness of the knowledge questionnaire. Again, success of construct assessment is dependent on proper selection of content domain which includes proper items.
CVI and its recommended level are based on content validity research for the development of psychological measures. There is a need of additional research to find out the range of content validity index (I-CVI and S-CVI) in development of different types of research tool on knowledge assessment for various categories of people on multidimensional aspects of health.
Conclusion
Assessment of content validity can be a useful step in the process of development of knowledge questionnaire on behavioural problems. The structure knowledge questionnaire has excellent content validity in terms of I-CVI and S-CVI/Ave. The finding of S-CVI/UA is 0.90. The universal agreement is overly strict as it demands 100% agreement which is excessively conservative. But, it is very important to compute S-CVI both ways and to report both values along with I-CVI so that potential users of the tool can make informed conclusions about the quality of tool in terms of its content validity. The present study recommends the future research wok on testing reliability of the knowledge questionnaire. Once the knowledge questionnaire meets the requirement of a standardised tool, then it can easily be used by researchers to assess the knowledge of the school teachers on behavioural problems of children in different setting. This ultimately helps in planning and developing various health policies directed towards promotion of mental health and prevention of mental illness in the society.
REFERENCES
- Aguilar B, Sroufe LA, Egeland B, Carlson E. Distinguishing the early-onset/persistent and adolescence-onset antisocial behavior types: from birth to 16 years. Dev Psychopathol. 2000;12:109-32.
- Lieberman AF, Silverman R, Pawl JH. Infant-parent psychotherapy: core concepts and current approaches. In: Zeanah CH, editor. Handbook of infant mental health. 2nd ed. New York: Guilford Publications; 2000:472-84.
- Campbell SB, Pierce EW, Moore G, Marakovitz S, Newby K. Boys’ externalizing problems at elementary school: pathways from early behavior problems, maternal control, and family stress. Dev Psychopathol. 1996;8:701-20.
- Tuma JM. Mental health services for children. The state of the art. Am Psychol. 1989;44:188-99.
- Merikangas KR, Nakamura EF, Kessler RC. Epidemiology of mental disorders in children and adolescents. Dialogues Clin Neurosci. 2009;11:7-20.
- Dutta M, Jahan M, Kumar R. Prevalence of behavioural problems in school going children of Tezpur city, Assam. Indian J Clin Psychol. 2014;41:20-4.
- Ghosh P, Choudhury HA, Victor R. Prevalence of attention deficit hyperactivity disorder among primary school children in Cachar, Assam, North-East India. Open J Psychiatry Allied Sci. 2018;9:130-5.
- Sharma SK. Nursing research & statistics. Gurgaon: Elsevier India; 2011.
- Waltz CF, Strickland OL, Lenz ER. Measurement in nursing and health research. 3rd ed. New York: Springer Publishing; 2005.
- Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and implementation content validity study: development of an instrument for measuring patient-centered communication. J Caring Sci. 2015;4:165-78.
- Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health. 2006;29:489-97.
- Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35:382-5.
- Burns N, Grove SK. The practice of nursing research conduct, critique, and utilization. 2nd ed. Philadelphia: WB Saunders; 1993.
- Waltz CF, Strickland OL, Lenz ER. Measurement in nursing and health research. 3rd ed. New York: Springer Publishing; 2005.
Barman N, Saikia Khanikor M. Content validity of a structured tool: knowledge questionnaire on behavioural problems. Open J Psychiatry Allied Sci. 2019;10:146-50. doi: 10.5958/2394-2061.2019.00031.4. Epub 2019 Mar 5.
Source of support: Nil. Declaration of interest: None.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.