Open Access

Development of an observation instrument to measure flourishing learning environments

  • Hersh C. Waxman1Email author,
  • Héctor H. Rivera1,
  • Brenna Linn1,
  • Yolanda N. Padrón1,
  • Kayla B. Rollins1,
  • Anna W. Boriack2 and
  • Beverly A. Alford3
Journal of Chinese Studies20161:3

https://doi.org/10.1186/s40853-016-0005-9

Received: 2 May 2016

Accepted: 18 August 2016

Published: 28 November 2016

The Publisher's Erratum to this article has been published in Journal of Chinese Studies 2017 2:5

Abstract

The present study expands the array of learning environment instruments by developing a walkthrough observation instrument that examines the five elements of flourishing classroom learning environments: (a) positive emotion, (b) engagement, (c) meaning, (d) positive relationships, and (e) accomplishment. The Flourishing Classroom Observation Measure was used to observe 202 classrooms from 22 rural schools located in Texas. The findings indicate that these classrooms were not “flourishing” to a great extent on the five key elements. None of the mean values for the five scales were found to be higher than 1.53 on the 3-point rating scale. The findings from this study also indicate that there were few significant differences by content area, grade level, or school.

Keywords

Learning environments Classroom observations Flourishing classrooms

Background

Improving the teaching and learning in schools is one of the greatest challenges facing educators today. Many educators have found that the current emphasis on high-stakes standardized testing has lowered the quality of classroom instruction in schools [20, 24, 38]. This problem is even more severe in rural settings because of the difficulty that rural schools have in recruiting and retaining high-quality teachers [9, 21]. In addition, many students in rural schools are exposed to deleterious conditions like poverty, poor school facilities, and the lack of up-to-date instructional resources that contribute to a pessimistic and harmful school and classroom environment.

Despite these persistent and severe educational problems related to rural education, there have been few systematic research efforts that have tried to address these concerns. The present study addresses the need to focus on these issues by examining the extent that classroom learning environments in rural schools are flourishing or contributing to students’ wellbeing and academic success.

The present study expands the array of learning instruments that have been widely used over the past several decades by developing a walkthrough observation instrument that examines the five elements of flourishing classroom learning environments. More specifically, the purpose of this study is to report the development and use of the Flourishing Classroom Observation Measure (FCOM). We examined: (a) the reliability, validity, and use of the instrument, (b) findings from the observations of 202 rural school classrooms, and (c) whether there are differences on the five FCOM scales by content area, grade level, and school.

Perspective/theoretical framework

In the last decade, the theoretical and empirical work in the field of positive psychology has begun to have an impact in the areas of psychology and education. The focal topic of positive psychology has become “well-being theory” that includes the following five elements: (a) positive emotion, (b) engagement, (c) meaning, (d) positive relationships, and (e) accomplishment [14, 3032]. According to [30], (p. 29), “the goal of positive psychology in well-being theory is to measure and to build human flourishing”. He maintains that our goal should be to increase flourishing by increasing emotion, engagement, meaning, positive relationships, and accomplishment.

The wellbeing and flourishing theory has not been directly applied to school settings yet, but many of the five key elements relate to research on classroom learning environments. Several of the most widely used learning environment instruments (e.g., What is Happening in this Class?, My Class Inventory, Classroom Environment Scale) [1013], for example, include scales such as student cohesiveness, teacher support, satisfaction, and involvement that relate to many of the elements of the flourishing theory. The theoretical and conceptual work of other learning environment researchers like Moos [22], Fraser and Walberg [12], and Wubbels and Levy [40] also relate to flourishing theory.

This line of inquiry is also similar to the growing body of research that focuses on the importance of supportive school relationships and classroom emotional environment [16]. Although socio-emotional behaviors have been widely investigated in terms of classroom climate and learning environment research, the focus on flourishing behaviors or flourishing learning environments is a very recent development.

The most common issue of past classroom environment research has been encapsulated by the investigation of relationships between students’ cognitive and affective learning outcomes and their perceptions of the classroom environment [2, 12, 39]. The present study, however, uses systematic classroom observation rather than student perceptions to assess classroom learning environments in rural schools. Many studies assessing classroom instruction and learning environment have relied on self-report data from administrators, teachers, or students (e.g., [12, 23]). These types of data, however, are often unreliable and tend to be upwardly biased in the direction of over estimating the actual quality of classroom instruction. Few studies have actually gone into rural classrooms to examine the quality of instruction and learning environment. It is especially important to observe classrooms in rural settings because of the historically, pervasive problems of poor instruction and disengagement that commonly occurs in those settings.

Methods

Participants

The participants were 202 PreK-12 classroom teachers in six school districts in south Texas. All of the districts were rural school districts with student populations ranging from approximately 500 to almost 12,000. Two of the school districts had a majority minority student population, while the other four districts served predominantly white student populations.

Instrument

The Flourishing Classroom Observation Measure (FCOM), a high-inference walkthrough instrument, was developed to examine the five elements of wellbeing and flourishing theory: (a) positive emotion, (b) engagement, (c) meaning, (d) positive relationships, and (e) accomplishment. The Flourishing Classroom Observation Measure is considered a walkthrough or walkabout instrument that is designed to obtain multiple snapshots of classroom practices in order to provide a rich data picture [5, 17, 33].

Walkthrough instruments have been widely used in school districts for the past decade [18]. They have been primarily used by school administrators for purposes such as (a) evaluating teachers [5], (b) diagnosing classroom instruction [29], and (c) improving student learning [19]. Most of these walkthroughs are designed to be quite short (3–5 min) [4], but for research purposes we decided to use a larger time period (20 min) based on our previous experience using similar classroom observation instruments [1, 37]. Other recent research also has found that 20 min is an appropriate time period for assessing classroom instruction [8].

The Flourishing Classroom Observation Measure was adapted from the Classroom Observation Measure (COM) [26], which measures the extent to which certain effective instructional strategies are demonstrated during a class period. The COM has been used in a number of studies and found to be reliable and valid [27, 28]. It also has been adapted and used recently in a number of studies [37].

The Flourishing Classroom Observation Measure was used at the end of the classroom walkthrough to rate, on a 3-point scale (1 = not observed at all; 2 = observed to some extent (once or twice); 3 = observed to a great extent (3 or more times), the extent that the five elements of well-being and flourishing theory: (a) emotion, (b) engagement, (c) meaning, (d) positive relationships, and (e) accomplishment were demonstrated during the observation period. Each element had at least three items that were used to measure the element. Some sample items for each of the elements are as follows: (a) Positive emotion—Students displayed positive affect toward teacher (+); Teacher displayed negative affect toward students (-); (b) engagement—Students were engaged in classroom activities (+); Students displayed disruptive behavior (-); (c) Meaning—Teacher related concepts to students’ actual lives (+); (d) Positive Relationships—Teacher appeared to have warm, supportive relationships with students (+); Student displayed positive engagement with peers (+); (e) Accomplishment—Students solved problems using real objects in the classroom (+); Teacher initiated project-based learning activities (+).

Procedures

Trained observers observed 202 classrooms in 22 schools from all six districts for approximately 20 min each. The teachers were aware of the week that the observations were scheduled, but they were not aware of the specific day or time that their class would be observed. Classrooms that were involved in nontraditional instructional contexts (e.g., testing) were avoided and attempts were made to revisit them at other days or times. Experienced classroom observers obtained a 90% coding accuracy with the instrument during initial training sessions. A sample of about 10% of the classrooms was used to determine the inter-rater reliability for the data in the present study. The inter-rater reliability was .84, which indicates a high degree of consistency among observers.

Results

An exploratory factor analyses was conducted with all the observation items on the FCOM. The factor analyses yielded ten factors accounting for more than 65% of the variance. However, upon further examination, the results were not interpretable so we decided to use the five a priori scales that were initially developed based on the flourishing theory. The individual items for each scale were aggregated and averaged to create each element score. The correlations among the elements were moderate, averaging about r = .53, indicating that there was some overlap regarding the discriminant validity of the scales.

Table 1 displays the means, standard deviations, and inter-correlations for the five scales. The means for the five elements ranged from 1.19 to 1.53, indicating that these elements were observed rarely in the 202 classrooms. The most positive element observed was Positive Emotion (M = 1.53), followed by Meaning (M = 1.36) and Accomplishment (M = 1.26). The least positive elements observed were Positive Relationships (M = 1.19) and Engagement (M = 1.20). It should be pointed out that the standard deviations for all the scales are relatively low (SD < .5) indicating that there was not a great deal variance to the extent that these were observed in the classrooms.
Table 1

Means, standard deviations, and inter-correlations for flourishing environments scales

Scale

M

SD

Inter-scale correlations

1

2

3

4

5

1. Emotion

1.53

0.49

     

2. Engagement

1.20

0.33

0.49

    

3. Meaning

1.36

0.37

0.58

0.45

   

4. Positive relationship

1.19

0.32

0.53

0.55

0.50

  

5. Accomplishments

1.26

0.29

0.54

0.46

0.66

0.53

 

Note: The Flourishing Environment Scales were measured by the following key: 1 = not observed at all; 2 = observed to some extent (once or twice); 3 = observed to a great extent (3 or more times)

Analysis of variance (ANOVA) was used to examine if there were statistically significant differences on the five elements by (a) district, (b) grade level, and (c) content area. The (ANOVA) results revealed that the only significant difference was for district on the Positive Emotion element, but the Tukey post-hoc test failed to reveal where the significant differences were occurring. No other significant differences were found between districts. The one-way ANOVA revealed significant differences between subject areas for the meaning element. The Tukey post-hoc test showed that “other” subjects were significantly higher than mathematics (p = 0.034). There were no other significant differences between subject areas. The one-way ANOVA revealed no significant difference between grade levels for the five elements.

Overall, the findings of the present study indicate that these rural school classrooms were not “flourishing” to a great extent on the five key elements of (a) positive emotion, (b) engagement, (c) meaning, (d) positive relationships, and (e) accomplishment. None of the mean values for the five elements were found to be higher than 1.53 on the 3-point rating scale. The findings from this study also indicate that there were few significant differences by content area, grade level, or school.

Discussion

The findings from the present study are discouraging since they suggest that nearly all of the classroom learning environments observed are not flourishing and there is little attention provided to socio-emotional learning. The low scores found for positive relationships and engagement in the present study are particularly alarming since both variables have been found to improve achievement outcomes for students [3, 15, 20]. Instruments like the Flourishing Classroom Observation Measure (FCOM) may help educators begin to focus on meaningfully constructed instruction and designing learning environments that build on the strengths of all students. The FCOM may also make educators more aware of the importance of quality of teacher-student relationships.

In the present study, we found the Flourishing Classroom Observation Measure (FCOM) had good inter-rater reliability, but only modest discriminant validity. Furthermore, improvement is required for the construct validity of the instrument. Additional larger-scale studies are needed to further examine the measurement properties of the instrument. We found that it was easy to train observers to use the FCOM and that it was easy to use in the classroom. We also found that the 20-min time frame for observation was ideal for research purposes, which allowed us to capture the quality of the instruction and the classroom environment. However, whether another time frame (e.g., longer or shorter) would provide a better snapshot of the classroom remains to be investigated.

These findings also raise several other important questions that need to be addressed in future studies. Most of these questions center on determining: (a) the extent that these elements relate to student cognitive and affective outcomes, (b) how the observational data relates to students’ perceptions of their classroom learning environment, (c) the factors that constrain teachers from increasing these key elements, and (d) the types of support teachers need to increase the use of these elements in their classrooms. Further research also needs to explore how flourishing learning environments are related to improved student academic outcomes and greater student engagement and motivation to learn. More theoretical, conceptual, and empirical work is also needed on how flourishing theory improves the quality of education for all students. It may be important, for example, to examine how the concept of flourishing learning environments relates to the emergent research on (a) academic optimism [35], (b) resilience [25, 41], (c) grit and perseverance ([6]: [34]) and (d) other socio-emotional outcomes [7].

Conclusion

While success and failure in school are dependent upon a number of influential determinants, it appears that the classroom learning environment may be a contributing factor [36]. The results of the present study are discouraging in that they paint a bleak picture of students from who are from rural schools. The lack of positive relationships, positive emotion, engagement, meaning, and accomplishment observed in these schools and classrooms suggests that these students are seriously at risk of failure. These schools may need to establish rigorous professional development programs to help their teachers improve the quality of their instruction and relationships with students.

Notes

Abbreviations

COM: 

Classroom observation measure

FCOM: 

Flourishing classroom observation measure

Declarations

Authors’ contributions

HCW conceived and designed the investigation; HHR, BL, & YNP reviewed the research and drafted the manuscript; KRR helped develop and collect the observation data; AWB conducted the data analysis; BLA helped collect the observation data and edited the manuscript; All authors read and approved the final draft of this manuscript.

Competing Interests

The authors declare that they have no competing interests.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Texas A&M University Education Research Center
(2)
College of Graduate Studies and Adult Education, Concordia University
(3)
Urban Education, University of Houston-Downtown

References

  1. Alford BL, Rollins KB, Stillisano JR, Waxman HC (2013) Observing classroom instruction in schools implementing the International Baccalaureate programme. Curr Issues Educ 16(2):17Google Scholar
  2. Allen D, Fraser BJ (2007) Parent and student perceptions of classroom learning environment and its association with student outcomes. Learn Environ Res 10:67–82View ArticleGoogle Scholar
  3. Boykin AW, Noguera P (2011) Creating the opportunity to learn: Moving from research to practice to close the achievement gap. ASCD, AlexandriaGoogle Scholar
  4. DeBoer L, Hinojosa M (2012) Walkthroughs: observation timeframes: what works best? Natl Forum Educ Adm Superv J 29(4):1–5Google Scholar
  5. Downey CJ, Steffy BE, English FW, Frase LE, Poston WK (2004) The three-minute classroom walk-through: changing school supervisory practice one teacher at a time. Corwin, Thousand OaksGoogle Scholar
  6. Duckworth AL, Allred KM (2012) Temperament in the classroom. In: Shiner RL, Zentner M (eds) Handbook of temperament. Guilford Press, New York, pp 627–644Google Scholar
  7. Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, Schellinger KB (2011) The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Dev 82(1):405–432View ArticleGoogle Scholar
  8. Early DM, Rogge RD, Deci EL (2014) Engagement, alignment, and rigor as vital signs of high-quality instruction: A classroom visit protocol for instructional improvement and research. High Sch J 97(4):219–239View ArticleGoogle Scholar
  9. Eppley K (2009) Rural schools and the highly qualified provision of No Child Left Behind: a critical policy analysis. J Rural Educ 24(4):1–11Google Scholar
  10. Fraser BJ (1998) Classroom environment instruments: Development, validity and applications. Learn Environ Res 1:7–33View ArticleGoogle Scholar
  11. Fraser BJ (2002) Learning environments research: Yesterday, today and tomorrow. In: Goh SC, Khine MS (eds) Studies in educational learning environments: An International perspective. World Scientific Publishers, Singapore, pp 1–25View ArticleGoogle Scholar
  12. Fraser BJ, Walberg HJ (2005) Research on teacher–student relationships and learning environments: Context, retrospect and prospect. Int J Educ Res 43:103–109View ArticleGoogle Scholar
  13. Fraser BJ, Fisher DL (1986) Using short forms of classroom climate instruments to assess and improve classroom psychosocial environment. J Res Sci Teach 5:387–413View ArticleGoogle Scholar
  14. Fredrickson BL (2009) Positivity: Top-notch research reveals the 3-to-1 ratio that will change your life. Three Rivers Press, New YorkGoogle Scholar
  15. Hamre BK, Pianta RC (2005) Can instructional and emotional support in the first-grade classroom make a difference for children at risk of failure. Child Dev 76(5):949–967View ArticleGoogle Scholar
  16. Harvey ST, Bimler D, Evans IM, Kirkland J, Pechtel P (2012) Mapping the classroom emotional environment. Teach Teach Educ 28:628–640View ArticleGoogle Scholar
  17. Kachur DS, Stout JA, Edwards CL (2010) Classroom walkthroughs to improve teaching and learning. Eye on Education, LarchmontGoogle Scholar
  18. Kachur DS, Stout JA, Edwards CL (2013) Engaging teachers in classroom walkthroughs. ASCD, AlexandriaGoogle Scholar
  19. Lemons RW, Helsing D (2009) Learning to walk, walking to learn: Reconsidering the walkthrough as an improvement strategy. Phi Delta Kappan 90(7):478–484View ArticleGoogle Scholar
  20. Milner HR (2010) Start where you are, but don’t stay there: Understanding diversity, opportunity gaps, and teaching in today’s classrooms. Harvard Education Press, CambridgeGoogle Scholar
  21. Monk DH (2007) Recruiting and retaining high-quality teachers in rural settings. Future Child 17(1):155–174View ArticleGoogle Scholar
  22. Moos RH (1979) Evaluating educational environments: procedures, measures, findings, and policy implications. Jossey-Bass, San FranciscoGoogle Scholar
  23. Organisation for Economic Co-operation and Development (2009) Creating effective teaching and learning environments: First results from the Teaching and Learning International Survey. Organisation for Economic Co-operation and Development, ParisGoogle Scholar
  24. Payne CM (2008) So much reform, so little change. The persistence of failure in urban schools. Harvard Education Press, CambridgeGoogle Scholar
  25. Rivera HH, Waxman HC (2011) Resilient and non-resilient Hispanic English language learners’ attitudes towards their classroom learning environment in mathematics. J Educ Stud Placed Risk 16(3):185–200View ArticleGoogle Scholar
  26. Ross SM, Smith LJ (1996) Classroom observation measure observer’s manual. University of Memphis, Center for Research in Educational Policy, MemphisGoogle Scholar
  27. Ross SM, Smith LJ, Lohr L, McNelis MM (1994) Math and reading instruction in tracked first-grade classes. Elem Sch J 95(2):105–109View ArticleGoogle Scholar
  28. Ross SM, Troutman A, Horgan P, Maxwell S, Laitinen R, Lowther D (1997) The success of schools in implementing eight restructuring designs: A synthesis of first-year evaluation outcomes. Sch Eff Sch Improv 8:95–124View ArticleGoogle Scholar
  29. Saginor N (2008) Diagnostic classroom observation: Moving beyond best practice. Corwin, Thousand OaksGoogle Scholar
  30. Seligman MEP (2011) Flourish: A visionary new understanding of happiness and wellbeing. Free Press, New YorkGoogle Scholar
  31. Seligman MEP, Ernst RM, Gillham J, Reivich K, Linkins M (2009) Positive education: Positive psychology and classroom interventions. Oxf Rev Educ 35:293–311View ArticleGoogle Scholar
  32. Seligman MEP, Steen TA, Park N, Peterson C (2005) Positive psychology progress: empirical validation of interventions. Am Psychol 60:410–421View ArticleGoogle Scholar
  33. Smith DL, Cude KC, Braziel KC, Waxman HC, Smith LJ (2008) A walkabout observation of pre-service student behaviors during field placement: Do they practice what we preach? Texas Assoc Teach Educ Forum 33(2):1–11Google Scholar
  34. Von Culin K, Tsukayama E, Duckworth AL (2014) Unpacking grit: Motivational correlates of perseverance and passion for long-term goals. J Posit Psychol 9(4):306–312View ArticleGoogle Scholar
  35. Wagner CA, DiPaola MF (2011) Academic optimism of high school teachers: Its relationship to organizational citizenship behaviors and student achievement. J Sch Leadersh 21:893–926Google Scholar
  36. Waxman HC, Garcia A, Read L (2008) Classroom learning environment and student motivational differences between exemplary, recognized, and acceptable urban middle level schools. Middle Grades Res J 3(2):1–21Google Scholar
  37. Waxman HC, Padrón YN, Franco-Fuenmayor SE, Huang SYL (2009) Observing classroom instruction for ELLs from student, teacher, and classroom perspectives. Texas Assoc Bilingual Educ J 11(1):63–95Google Scholar
  38. Waxman HC, Padrón YN, Lee YH (2010) Accelerating the pedagogy of poverty in urban schools: Unanticipated consequences of the No Child Left Behind (NCLB) Act. ERS Spectrum: J Res Inf 28(2):37–43Google Scholar
  39. Wolf SJ, Fraser BJ (2007) Learning environment, attitudes and achievement among middle-school science students using inquiry-based laboratory activities. Res Sci Educ 38(3):321–341View ArticleGoogle Scholar
  40. Wubbels T, Levy J (eds) (1993) Do you know what you look like? Interpersonal relationships in education. Falmer, LondonGoogle Scholar
  41. Yeager DS, Dweck CS (2012) Mindsets that promote resilience: When students believe that personal characteristics can be developed. Educ Psychol 47(4):302–314View ArticleGoogle Scholar

Copyright

© The Author(s) 2016