Note: If you browse this site it will use temporary 'session cookies' to assist navigation and keep an anonymous record of page visits. More info…

Introduction

ii

The Girls Energy Conservation Corps (GECCo) program was designed by a partnership between TERC, an educational research and design organization, and the Girl Scouts of Eastern Massachusetts (GSEM), USA. The Girl Scouts (GS; known as Girl Guides in other countries) organization aims to empower girls and help teach values (e.g., honesty, fairness, courage, and citizenship) through organized activities that impart practical skills. The goal of the GECCo program was to support eight- to fourteen-year-old girls to connect knowledge about energy conservation and enactment of energy conservation measures to learning about climate change. The program focused on integrating engaging online and real-world activities that involved girls in learning about climate change and their role in it, saving energy, understanding the importance of collective goals and action to address climate change, and using new media creatively to educate peers and the community about energy conservation.

The design of science programs for free-choice environments is notoriously difficult, since learners—and instructors—in such environments are often novices with respect to science. In addition, levels of intrinsic motivation are highly variable, and environments are relatively unstructured (Falk, Reinhard, Vernon, Bronnenkant, Deans & Heimlich, 2007). The design of the GECCo program was no different. Many of the details of this informal context of program implementation imposed significant design constraints. For example, GS troops differed in size (from approximately 3 to 17 girls) and had a fairly wide age range of participants, unpredictable scheduling and levels of attendance, and variable levels of troop leader skill and style in facilitation. The time that girls could be expected to engage with program activities was short, and meeting spaces had limited computer access. Further, girls were free to make choices about what parts and how much of the GECCo program they wanted to complete.

The results of a mixed-method summative study focusing on the outcomes of the GECCo program have been reported elsewhere ( Puttick, Bernstein, Kies & Garibay, 2015). Based on pre-post testing, we know that the GECCo program was effective in meeting its goals. Positive changes in knowledge, behavior, and attitudes pre to post suggested that a carefully designed program could address the challenges of educating children about energy conservation and climate change at this age even if participant exposure to the program was brief. In addition to the summative study, a traditional formative evaluation to support program design, conducted during four rounds of design and revision, was employed. Initial design drew on a theory-based logic model, and subsequent refinement of the program was based on formative data ( Puttick, 2012).

In this paper, we retrospectively examine and discuss our design choices from the perspective of design-based research (DBR) ( Barab & Squire, 2004; Cobb, Confrey, DiSessa, Lehrer & Schauble, 2003; Sandoval, 2014). DBR, the systematic study of designed interventions, has gained traction in the recent decade. DBR can contribute to learning theory because designed learning environments explicitly embody conjectures about the relationship of intended outcomes to specific tools, materials, task, and participant structures ( Sandoval 2004, 2014). Further, a particular feature of DBR that makes it productive is the purposeful blending and adaptation of theories of learning and theories related to non-cognitive factors, e.g., organizational context, motivation ( Russell, Jackson, Krumm & Frank, 2013).

Can insights from a DBR approach be applied retrospectively? Sandoval (2004) thinks so. For example, a retrospective analysis of the BeGUILe program, a software tool, led to the discovery that an expected relationship between student inquiry activity and student beliefs about science was not supported. In this paper, we apply the DBR approach retrospectively because it allows us to look beyond the simple reporting of formative data aimed at improving the program. Retrospective examination of all project documents allowed us to identify and make explicit the design conjectures flowing from our theoretical framework that we had implicitly drawn upon to inform design, and to examine more closely how the tools, materials, participant and task structures were related to these. We hoped that this would allow us to extract a design heuristic to guide future design development.

In this paper, we address the following questions:

• Can a retrospective analysis of our program to explicitly define the design conjectures that underpinned design, and to examine how specific materials, task and participant structures relate to these, provide insights into the design process?
• To what extent can these insights contribute to a design heuristic that could inform future development of similar programs?

Theoretical framework

iii

Programs and materials addressing energy conservation in the context of climate change education, designed for use in both formal and informal settings, have proliferated over the past decade. This trend has been motivated in part by concerns over lack of understanding of critical energy-related issues, such as the lack of sustainability of our reliance on fossil fuels (DeWaters & Powers, 2011; Southwell, Murphy, DeWaters & LeBaron, 2012; NEETF, 2012).

The cognitive and perceptual challenges to learning about climate change are well known (Gautier & Rebich, 2005; Sell, Herbert, Stuessy & Schielack, 2006; Grotzer & Lincoln, 2007; McCaffrey & Buhr, 2008). Because climate change is considered complex, many of the underlying physical phenomena that contribute to understanding climate change are taught at the high school and college level, yet students are capable of laying the foundation for this understanding in middle school or even earlier (Dauer, Miller & Anderson, 2013, 2014; Holthuis, Lotan, Saltzman, Mastrandrea, Gray, Bofferding, & Sullivan, 2012).

At the start of the GECCo project, the design of the program was theoretically principled, being derived from a synthesis of the literature that took context and culture into account. While based on widely established principles that support science learning, such as learning through inquiry (Schwab, 1962; DeBoer, 1991; Minner, Levy & Century, 2010), the design was also supported by constructs from conservation psychology and theories of behavior change. We briefly discuss the constructs that underpinned the program in the paragraphs that follow, to support our analysis in this paper.

Culture

1

In 2008, the Girl Scouts’ Research Institute ran a nationwide survey of girl scouts, and found that 50% of girls aged 8 to 10 indicated that “helping animals or the environment” was very important to them (Schoenberg, Salmon & Fleshman, 2008). Across the U.S. GS, there is an organization-wide emphasis on the importance of participation in community action, and, indeed, an expectation that girls will earn GS patches or badges through such participation. Girls are inspired to “develop a lasting commitment to the environment” through ‘Journeys’, camping experiences, and the GS Ranger program (Girl Scout Research Institute, 2012). Finally, there is a GS identity, long established through dress, behavior, and social structure in troops, with which individual girls strongly affiliate themselves.

Program components that emphasize norms of social identity (Schwartz, Luyckx & Vignoles, 2011), care of the environment, and taking community action such as those that exemplify the GS organization can be effective in supporting learning (Britner, 2002; Tan & Calabrese Barton, 2008). Therefore, we decided that a focus on energy conservation with respect to its influence on climate change would be a natural fit with, and could draw on the strengths of, GS culture.

Conservation psychology

2

Because we were dealing with young learners and an overwhelming topic, we turned to what has been written about the particular challenges of teaching and learning about climate change (American Psychological Association, 2009; National Research Council, 2011). There is some agreement among educators that findings from the relatively new field of conservation psychology, which focuses on study of the reciprocal relationships between humans and the rest of nature, can effectively encourage conservation of the natural world (Saunders, 2003; Brook & Clayton, 2005; Leiserowitz, 2005).

The GECCo program was built on the assumption that those who move to action now—for example, by engaging in conservation behaviors—can raise awareness in their communities about climate change (Doherty & Clayton, 2011). In particular, for young people for whom realizations about the reality of climate change and the resulting feelings of powerlessness can be overwhelming, taking action can combat a sense of paralysis (National Research Council, 2010; Koger & Winter, 2010; Stern, 2011; American Psychological Association, 2009). We conjectured that a sense of empowerment engendered by taking action might ultimately result in young people enlisting others in behavioral changes with them (Leiserowitz, 2005). Opportunities for involvement in actions that make a difference can reinforce conservation skills and attitudes in the face of a pressing global problem (Leiserowitz, 2005, Chawla & Cushing, 2007; Thogersen & Olander, 2003).

Social norm messages, which provide information about other people’s behavior and beliefs, have been shown to affect conservation behavior change (Goldstein, Cialdini, & Griskevicius, 2008; Nolan, Schultz, Cialdini, Goldstein & Griskevicius, 2008; Schultz, Nolan, Cialdini, Goldstein & Griskevicius, 2007; Bernedo, Ferraro & Price 2014; Bernstein & Puttick, 2014). For example, programs can harness the power of social norms by asking participants to set group goals and share their conservation efforts (Nolan et. al. 2008; Abrahamse, Steg, Vlek & Rothengatter, 2005). We have also shown the efficacy of social norm messages in sustaining conservation behavior beyond the duration of girls’ participation in the GECCo program (Bernstein & Puttick, 2014). As we will show, we drew on social norms to shape several aspects of the GECCo program.

Theories of behavior change

3

Since taking action was a core component of our envisioned design, we drew on theories of behavior change (Ajzen, 1991; Nigbur, Lyons & Uzzell, 2010). Behavior change is a complex and multidimensional construct that is increasingly being examined with respect to energy savings (Abrahamse et al., 2005; Osbaldiston & Schott, 2012) and to action related to climate change (Dietz, Gardner, Gilligan, Stern & Vandenbergh, 2009; Rabkin & Gershon, 2007). It includes many processes that have been defined in various behavior change models (e.g. Ajzen, 1991; Stern, Dietz, Abel, Guagnano & Kalof, 1999; Kolmus & Agyeman, 2002). For example, people who express an explicit commitment to action are more likely to change their behavior than those who do not (Monroe, 2003). Also, extrinsic incentives are practical in ‘kick-starting’ a behavior, though not effective in maintaining it over the longer term (DeYoung, 1996).

The trans-theoretical model of behavior change (Prochaska, Redding & Evers, 1997), in particular, is useful in understanding the stages of behavior change. The model theorizes that behavior change is a process occurring over time through a sequence of stages. The stages include both covert and overt activities. Four such stages around which we designed the program (Puttick et al., 2015) were:

1. raising consciousness about energy conservation and climate change including alternative actions to take (increased knowledge or understanding),
2. weighing the pros and cons of energy saving behaviors to influence motivation to change,
3. creating reminders about saving energy, and
4. showing the intention to engage in a behavior by making a commitment.

The GECCo program

iv

At the outset of program design in 2009, we developed a logic model that defined project goals, summarized a general description of ideas for activities, and related these to intended outcomes as shown in Table 1. The project addressed four impact areas: knowledge, skills, behavior, and attitudes. The program included a variety of troop-time and homework activities that addressed these areas. Note that explicit constructs from our theoretical framework were not present in the logic model, nor were explicit descriptions of specific program embodiments (i.e., materials, task structures, and participant structures). As we will show, our retrospective analysis reveals a design conjecture map that describes these program underpinnings in substantially different ways.

The full GECCo program in its final iteration consists of the 6 ‘patches’ shown in Figure 1. Patches are worn on the back of the GS vest or sash. They are similar in substance or style to the badge activities approved by the National GS Council. Patches, however, are developed by local GS Councils. The GECCo patches are designed for Juniors, aged 8-11, and Cadettes, aged 12-14. The six patches in the program are described elsewhere (Puttick et al., 2015) and can be found on the GS of Eastern Massachusetts website at: www.girlscoutseasternmass.org/gecco/.

Each GECCo patch includes seven or eight activities carefully designed to engage young people in informal learning settings (Falk et al., 2007). They are aligned with the kinds of activities that girls and troop leaders expect to find in a patch program to maximize their use within the GS free-choice environment. Each patch includes an arts-and-crafts activity; an ‘up-and-moving’ activity; a computer, card, or board game; an action in the community or taking leadership with younger girls; at least one science investigation; and a field trip investigation. Each activity addresses at least one of the intended outcomes of the program. To earn a patch, girls are required to complete at least four activities. Completion involves an average of two to three hours of troop time over one or two troop meetings, which are typically weekly or bi-weekly. In addition, some out-of-troop time may be taken to complete activities such as tracking energy savings at home.

All activities include a brief overview that presents learning objectives and some science. They all also include brief ‘making sense’ questions for optional discussion at the end of the activity to strengthen its connection to the learning goals. Finally, optional activity extensions are also suggested.

Formative research methods

v

Our process of design, formative implementation research, and revisions included four iterations conducted in 2009 and 2010 (Table 2). Details of the formative research we conducted to improve the program has been reported elsewhere (Puttick, 2012). In brief, we employed standard methods (Mathison, 2005), collecting qualitative and quantitative data through observations, girl and troop leader interviews, pre-post surveys, and brief pre-post content knowledge assessments. Four hundred and two girls in 39 troops participated over the 4 iterations of the program. Prior to the first iteration, we established baseline data by interviewing focus groups of troop leaders, both to share our logic model and to help us develop a deeper understanding of the implementation context. During the original development of the first iteration, we informally tested selected activities to improve our understanding of time frame, setting, and age-appropriateness. We implemented them in a troop setting and interviewed girls afterwards about their interest in particular activities, their general attitudes towards topics such as climate change and energy generation, and their feelings of efficacy in reducing personal energy use. We also gathered data from discussions during snack break or during the course of the activities about how energy is produced and what climate change means. These informal focus groups allowed developers to hear girls’ conceptions of these constructs for themselves. In subsequent iterations, data sources included surveys of girls and troop leaders, review of artifacts, observations of sample troops, and a troop-leader focus group.

Program design and implementation

vi

In baseline interviews, troop leaders told us that they and the girls expect to earn a patch or badge after only 2 hours of participation and that girls are not expected to do any add-ons outside of troop time. Also, troop leaders told us that they were not predisposed to lead science activities and that they knew very little science, although they expressed interest in activities that related to the environment and sustainability. Prior to their participation in GECCo, all troop leaders had completed fewer than two environmental or science badge or patch programs in the last year. They represented the type of audience we anticipated for the program: adults with not much science background or professional science experience.

These front-end findings posed two immediate design dilemmas. First, since the intended learning goals included some complex science content, we felt that extended exposure would be essential. Accordingly, we ultimately located the important content related to climate change in a first patch that would be a requirement for all girls. In addition, we included learning goals in the activity guides, and included directions for supporting discussion and reflection at the end of each activity. Second, models of behavior change suggest that opportunities to practice behaviors are an important component of the progression towards energy conservation habits. Though not optimal to the audience, we nonetheless asked participants to engage in energy conservation at home. We also built in opportunities to do this across all patches, and we provided supports in the form of reminders, setting troop goals, etc. By the end of our design process, we decided that six patches in total would provide multiple opportunities for girls to learn and engage in diverse contexts, and that this would increase exposure since we expected that girls’ participation would extend to multiple patches.

In the following sections, we present and discuss the design and revision of selected program components as exemplars of the ways in which our designs changed over successive iterations of the program. They are grouped under embodiments related to constructs of the theoretical framework, viz., Culture and Conservation Psychology/Theories of Behavior Change.

Embodiments related to culture

vii

Energy Angels and The GECCo Challenge

Finally, the tracking sheet in iteration 3 included a space where girls could explicitly enter a total troop commitment, thus affirming GECCo girls’ affiliation with the GS culture as stewards of the environment. When girls made their GECCo Challenge pledge to reach a troop goal at the beginning of any patch, they were also invited to respond to an open-ended prompt that probed their motivations for saving energy. As part of our research, the responses of 119 girls in iteration 3 were coded into categories. The results (see Table 3) showed that ‘saving the environment’ was the most frequent category of responses, followed personal goals related to survival and well-being, and, least frequently, specific to saving energy for the future.

As we will discuss more fully in the next section, these changes were intended to more explicitly draw on GS culture, positioning girls as people who cared about the environment, because they could more concretely see the positive results of their actions, not only in their tally of points, but also in terms of a monetary contribution to a cause that was important to them. It capitalized on girls’ interest, at this age, in ‘helping the planet’, and also, we assumed, would align with the theorized commitment from behavior change theory by engendering feelings of empowerment as girls realized their goals. In addition, the accumulation of small actions to make a difference draws from conservation psychology, which tells us that young people need action to counter powerlessness, and which positioned them as girls who ‘change the world’, that is, as people with a conservation identity. This was particularly effective in supporting action; by the end of their participation in the patches in iteration 4, the cumulative result of girls’ actions was to contribute over $2,500 to these three causes.

The great majority of girls (90%) reported that saving energy to accrue points that would earn money for a conservation cause motivated them to save energy. Two troop leaders reported:

“They were very excited about saving a Gecko. This was their true motivation to know what they did does make an impact on a specific animal. We went to a pet store to learn more about geckos. The girls loved this!”

“The girls really took to the GECCo Challenge. They talked about it to others outside of troop time and during troop meetings they discussed ways of earning more points. […] They were very excited about raising money to benefit the causes.”

From ‘Sun and Shade’ to ‘Sunny Side Up’

2

In the Warm Home Cool Home patch, the Sun and Shade activity is a hands-on science investigation in which girls explore the effects of shade in regulating temperature (Figure 5). In iteration 1, the activity was written with components and language familiar in science class, e.g., “record and discuss the implications of your results”. This turned out not to work within the GS culture.

In the 2nd and 3rd iterations, the Sun and Shade science investigation was better aligned with our observations about activities appropriate to GS troop meetings, but still retained some of the ‘school’ feel because the changes were somewhat superficial. For example, we changed some of the language—e.g., ‘Instructions’ became ‘What to Do’ and ‘Sun and Shade’ became ‘Sunny Side Up.’ In addition, shifts related to identity included the addition of an invitation to take action at home by exploring how window shades were used and a link to the GECCo Challenge to report their actions in this regard.

In the final iteration, this investigation was more substantially revised to align with GS culture. The activity was further simplified and shortened. In addition, girls were more directly invited to save energy through using shades themselves at home. The discussion was replaced with a ‘Simon Says’ game through which the girls embody the results of the investigation. In the game, girls stand up when ‘COLD’ is called (the blinds should be open to let in the sun when it is cold outside), sit down when ‘HOT’ is called (the blinds should be closed to keep out the sun when it is hot outside). This embodied activity helped the girls consolidate learning in addition to providing a concrete way they could remember to save energy at home.

Embodiments related to Conservation Psychology and Theories of Behavior Change

viii

Increased awareness or knowledge about the arena in which behavior change is desired is an important factor in all behavior change models. Therefore, throughout the patches we included activities explicitly designed to support learning about energy conservation and climate change. We also provided information about energy conservation and climate change in those activities that were focused directly on action.

Little Things Add Up

1

The Little Things Add Up paperclip activity is an example of how conservation psychology supported our design. The stated purpose in the iteration 1 guide for this activity was:

“to see how the little decisions we make every day, like how long we decide to stay in the shower, can have a big impact—either good or bad—on the environment. By changing your shower or bath routine just a little bit, you can save a lot of water and energy.”

In iteration 2 of Little Things Add Up, girls were asked to take the additional step of attaching the 3 individual paperclip chains into a single, continuous chain to visualize the cumulative impact. This was intended to better convey the scale of water use by providing a concrete intermediate visualization between individual showers (before chains were linked), showers of the three girls, and their projections of overall hot water use for the troop and the whole GS Council. In addition, the mathematics was removed in favor of providing the information on the hot water totals, including information on the thousands of gallons saved over the course of a year if the fictional girls followed through with the GECCo Challenge pledge. The activity still concluded with a brief discussion of how together girls can make a difference.

In the final iteration of this paperclip activity, we removed the numbers altogether and let the comparison of the chains speak for itself. We also moved the discussion of the results from being a wrap-up of the activity into the chain-making itself, so that the talking happened while the girls were connecting the paperclips. Finally, we linked girls’ own water conservation efforts to the GECCo Challenge. The shift in focus from calculating exact numbers not only reduced the school-like aspect but also aligned the activity better with our assumptions related to the theory of behavior change, namely, that building awareness is a first step.

GECCo Movie, The Carbon Ballet, and U2CC Card Game

2

One core component of the GECCo program content is the connection between energy usage and climate change. We knew from the beginning that addressing this highly complex and scientific connection was going to be challenging. We also knew that we could not rely on troop leaders to convey it to students, so starting with iteration 2, the GECCo project included a short movie cartoon that presented information about the connection between demand for energy and the climate effect of burning fossil fuels. It specifically showed how energy use generates carbon dioxide and how this gas contributes to the greenhouse effect. It ended by introducing ways in which energy can be saved by small actions, within the girls’ spheres of influence, such as saving hot water, turning off lights, and shutting down electronic devices when not in use. However, exposure to this information was brief, so by iteration 3 we saw the need to develop additional materials to address this content.

Like the Carbon Ballet for younger girls, the U2CC card game (Figure 8) was designed to explicitly teach older girls about the direct contribution of their own energy use and climate change. Of course, since it is a quick, snappy card game, it is aligned with GS culture too. Girls’ responses in the post-survey in the 3rd iteration showed a significant increase in their awareness that energy use ‘by people like me’ is directly connected to climate change. As one troop leader commented:

“We loved the U2CC card game. It was a fun way to learn about the connections we each personally have on the use of energy and climate change.”

Retrospective analysis of program design

ix

Retrospective analysis of the early project materials, including the logic model, revealed that explicit design conjectures originating from the theoretical framework were not specified in the first couple of years. It was in some cases a challenge to link the specific tools, task, and participant structures back to specific design conjectures. Nevertheless, some patterns emerged. Our first round of (retrospective) mapping of design conjectures and related embodiments from early in the project makes the mismatch between design conjectures and embodiments evident, particularly with respect to the design conjecture relating to culture. Table 4 shows the embodiments (tools, task and participant structures) that we retrospectively identify and the links with design conjectures and intended outcomes made at the outset of design.

At the outset of the design process, an unstated assumption was that we could implicitly rely on GS culture, which stressed care of the environment, taking leadership, and focusing on community, to provide strong enough intrinsic motivation for girls to engage in school-like activities such as brainstorming, debriefing discussions, and reflective thinking to unify learning experiences about energy and climate change. However, this didn’t take into account another key aspect of GS culture—expectations of what GS activities should look like. The girls and troop leaders had very definite ideas about the types of activities they did in GS, such as ones that involved making a tool or craft (e.g., constructing a playful draft detector), movement (e.g., creating and performing skits), or team-based challenges (e.g., scavenger hunts) that were engaging, safe, and capable of being completed by the range of girls in one or two troop sessions. Whatever science we hoped to teach needed to be embedded in such activities.

The retrospective analysis surfaced evidence that constructs related to culture and theories of behavior change were explicitly informing design by the time later iterations were implemented. The analysis makes it clear that if we had been able to more explicitly and accurately define the design conjecture about GS culture at the outset (as we do in Table 5, cf. Table 4) – that girls expect typical GS-type activities — we might not have generated such a cultural mismatch in our design of materials (e.g., science investigations) and task structures (e.g., inquiry-based) in the first couple of iterations. The multiple redesigns of the Sun and Shade and the Little Things Add Up activities (Figures 5 and 6) exemplify this point. After iteration 1, we realized that we needed to expand the modes of engagement (Fenichel & Schweingruber, 2010) available to girls in each patch. Accordingly, our design shifted towards the ways in which science can be learned more ‘stealthily’. To do this, the science was unobtrusively woven into the fabric of the activity without the obvious trappings of science learning.

By iteration 4, the traditional science activity embodied in Sun and Shade (see above) was completely transformed to more closely resemble an activity that aligned with girls’ expectations for typical GS activities (Table 5), as were the Carbon Ballet (Figure 7) and U2CC (Figure 8). Table 5 summarizes how retrospective analysis maps the new embodiments designed in iterations 2 and 3 with a refined design conjecture about culture. The refined conjecture was stimulated by the motivation responses summarized in Table 3. They represent good examples of the embodiments that most clearly aligned design conjectures and expected outcomes: these aligned well with the types of activities girls expect to do in troop time.

Retrospective analysis also brought to the fore how important identity became as a heuristic for designers. It is clear that an expanded awareness of the importance of girls’ identity at the outset would have pointed to insights early on that could have shaped activities in the patches to achieve successful outcomes much earlier. Realizing at that point that girls’ identity as Girl Scouts who care for the environment, and for whom the social norm of doing so would be important (Table 5), might have led to the design conjecture that engagement and behavior change could be supported by designing embodiments that built on identity. As it was, we began to design ways to stress identity only in iteration 3. For example, we ‘branded’ the program with the cute GECCo gecko, and supported girls to set and achieve troop goals, not individual goals. The conservation implications of GS identity were stressed by giving girls the opportunity to tally their actions towards support for a conservation cause.

We do not know from this data if there is a causal connection between girls’ interest in contributing to conservation causes and their identity, as others have found (Clayton & Opotow, 2003). However, girls’ reported intention to continue saving energy because it was now automatic aligns well with our conjectures drawn from theories of behavior change. As predicted by Prochaska’s trans-theoretical model of behavior change (Prochaska et al., 1997), necessary ingredients of a successful effort towards action include raised consciousness, expressing motivation and commitment to change, opportunities to practice a desired behavior, and social norms that support motivation and action.

Implications for design

x

We believe that the value of our retrospective analysis was manifold. Here we draw out three important design heuristics that could guide future program development.

Defining design conjectures

1

First, the analysis shows the value of surfacing and clearly defining all the design assumptions that underpin program development, linking them closely to desired outcomes, and then, most important, explicitly drawing out how the intended tools, materials, task structures, and participant structures mediate the connection between the designer’s assumptions and intended outcomes. A logic model is not enough. As we described earlier, at the outset we had developed a detailed logic model that built carefully—but implicitly —on our theoretical framework. However, retrospective analysis from a DBR perspective provided an analytic framework for explicitly examining the individual components of our design in relation to identity and culture. The methodological implication for us was to highlight the importance of gathering data on observable processes such as peer-to-peer talk, and on participant-generated artifacts, in order to determine how embodied design conjectures mediate desired program outcomes. The retrospective analysis highlights the research differences between DBR and formative evaluation approaches. Our formative evaluation focused on what worked, and not how it worked, which as Sandoval sees it, “is a key distinction between design-based research […], and formative research aimed simply at making something better” (Sandoval, 2004, p. 20).

The results of our retrospective analysis persuade us of the benefits of DBR, and cause us to add ours to the groundswell of voices that advocate a DBR approach to program design and development.

Understanding culture

2

Second, DBR is productive for allowing the purposeful blending of theories of learning and those related to non-cognitive factors such as organizational context (Russell et al., 2013). At the outset of our collaboration, the Girl Scouts of Eastern Massachusetts partners expressed strong support for a prominent emphasis on approaching science and technology more rigorously in their GS programs. However, as we worked with troop leaders and girls, we discovered a well-established culture, that in many cases has been handed down over two or even three generations of troop leaders, that worked against this organizational commitment. This project showed that, to be effective, a design project needs to be sufficiently familiar with the real culture of the institution as it is enacted, and to fully recognize the identities and expectations of participants to the extent possible. If activities are mediated in institutional settings for which the main point is not content learning, but personality and character development or growth in a craft or skill area of interest, science activities that include analytic learning may not be successful.

We are not unique in suggesting that it is important to take culture into account as a constraint and limitation in formal and informal educational settings (e.g., Vossoughi, Escudé, Kong & Hooper, 2013; Wheaton & Ash, 2008). However, we suggest that it can also serve as a valuable affordance if built upon and incorporated as a program strength. Again, our retrospective analysis shows that, once we revised the materials and task structures to more closely align with GS culture, and to draw upon aspects of GS identity, they proved to be highly engaging to participants. Girls learned about alternative actions to take in using energy, demonstrated the motivation to change, identified as people who want to change, and showed the intention to change by making a commitment. They followed through on that commitment, successfully saving energy across the duration of the program (Puttick et al., 2015), and extending it beyond the life of the program (Bernstein & Puttick, 2014). The retrospective analysis thus illuminated how design conjectures based in a thorough familiarity of culture and identity of participants could effectively guide program design from the outset.

Linking learning and action

3

Demeo, Feldman, & Peterson (2013) advocate the integration of energy education with education about ecology and the environment. In our case, the connection we made between these three domains was associated with positive outcomes in all three arenas (Puttick et al., 2015). We suggest that a focus on conservation psychology and behavior change as theoretical constructs can enhance the effectiveness of energy education efforts, particularly if linked to the context of climate change.

Some have argued that individual action is totally inadequate to meeting the global challenge and scope of climate change. However, like Dietz et al. (2009), we argue that household actions, though totally inadequate in themselves to address climate change, can provide a behavioral wedge that may impact carbon emissions in the future. Many have noted that opportunities for involvement in any actions that make a difference, however small, can nurture more general conservation skills and attitudes in the face of a pressing global problem (Leiserowitz, 2005; Chawla & Cushing, 2007; Thogersen & Olander, 2003). More important, we argue that the GECCo program has empowered girls to make potentially lifelong changes to their energy-related behaviors, attitudes, and identities. This would be a valuable area for future research.

One other of our design conjectures worth mentioning concerns the importance of countering powerlessness in the face of the global problem of climate change (American Psychological Association, 2009). Our design conjecture was that taking action was an important factor in engendering a sense of efficacy in addressing the problem, specifically because there are many ways that young people this age could take action within their sphere of influence. However, embodiments of this conjecture—girls learn how to save energy, practice energy conservation, set group goals, and understand the effect of small actions adding up to make a difference—appear to have had an uncertain effect with respect to empowerment. Girls’ perception of their self-efficacy actually decreased pre to post participation in the GECCo patches (Puttick et al., 2015). Even so, the combined evidence shows the GECCo patch activities were effective in helping increase the girls’ awareness of the impacts that collective energy conservation efforts can have on reducing global warming (Puttick et al., 2015). Powerlessness can be felt with respect to various layers of an individual’s scope of action and decision. It is interesting to speculate whether some of their actions might reach beyond their personal sphere of efficacy to, for example, leading conservation efforts at their school, as their sphere of influence expands with age and autonomy. This would be a valuable area for future research.

Acknowledgments

xi

We acknowledge the skill and creativity of Meaghan Donovan who participated on the design team. Marian Grogan, project director, and Rachel Hayes, research assistant, kept the project on track, freeing developers and researchers to focus on implementation research. Conversations with Carol Saunders about conservation psychology and theories of behavior change proved to be invaluable. Polly Hubbard, formative evaluator, contributed not only evaluation findings but also creative recommendations to improve the program. Joan Reilly, co-Principal Investigator at GSEM, advised us on all aspects of development and implementation, and successfully recruited participating troops. The National Science Foundation provided funding under grant #DRL-0813434. Any opinions, findings, and conclusions or recommendations expressed in this article are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References

xii

© ISDDE 2018 - all rights reserved