Critical Issues in Evaluating the Effectiveness of Technology

We are far enough along in the technological revolution and its application to learning that it is time for systematic review and analysis of what works best.

U.S. Secretary of Education Richard W. Riley National Conference on Educational Technology Washington, D.C., July 12, 1999

The Secretary's Conference on Educational Technology: Evaluating the Effectiveness of Technology on July 12-13, 1999, in Washington, D.C., noted a shift in schools' focus on technology. Where once the emphasis was on building and implementing a technology infrastructure, today it is on evaluating the effectiveness of its use in schools and classrooms. Parents and teachers, school boards and administrators, governors and state legislatures, and Congress all want to know if the nation's investment in technology is providing a return in student achievement. Indeed, if resources are to be expended on technology, it is becoming a political, economic, and public policy necessity to demonstrate its vital effectiveness.

In his opening address, U.S. Secretary of Education Richard Riley remarked, "The primary reason for this conference is to gather information from all of the outstanding schools, districts, and states represented here-so that we can study it, share it, and learn from it. Just as important as learning what works, we must learn what does not work. We must not assume everything that employs technology is going to be successful. That is why evaluation is so important. And then we must use that evaluation to create positive change."

The conference drew on the insights and collective wisdom of its attendees, starting with an emphasis on state-level technology evaluations. Evaluators from West Virginia explained how they isolated the effects of their Basic Skills/Computer Education initiative. They found that the more access to technology students had and the more their teachers believed that technology could help and were trained to use the technology, the higher students scored on the Stanford 9 (11% of the total gain scores). Idaho attendees described a four-year study focused on eight specific goals by which to evaluate the impact of the state's technology investment. Significant results included statewide academic gains as measured by the Iowa Test of Basic Skills (ITBS) and the Test of Academic Proficiency (TAP) for 8^th and 11^th graders.

From there, the conference turned its spotlight on school practitioners who expressed the need for more formative evaluations than the summative evaluations described by state policymakers. Lively debates arose among teachers, district curriculum and technology coordinators, administrators, state curriculum and technology coordinators, state and national policymakers, and researchers in the evaluation of educational technology about ways of identifying and collecting technology evaluation data that is relevant at the local level but also useful for other stakeholders.

During what Dale Mann of Interactive, Inc., called, "this developmental moment," conference participants exchanged promising evaluation strategies and techniques and considered how to respond to the many voices demanding to know technology's effects on schooling. The following seven critical issues in evaluating the effectiveness of technology in education arose as a consequence of the interaction among stakeholders:

• The effectiveness of technology is embedded in the effectiveness of other school improvement efforts.
• Current practices for evaluating the impact of technology in education need broadening.
• Standardized test scores offer limited formative information with which to drive the development of a school's technology program. Most schools are looking for additional means for collecting useful data for this purpose.
• Schools must document and report their evaluation findings in ways that satisfy diverse stakeholders' need to know.
• In order for evaluation efforts to provide stakeholders with answers to their questions about the effectiveness of technology in education, everyone must agree on a common language and standards of practice for measuring how schools achieve that end.
• The role of teachers is crucial in evaluating the effectiveness of technology in schools, but the burden of proof is not solely theirs.
• Implementing an innovation in schools can result in practice running before policy. Some existing policies need to be "transformed" to match the new needs of schools using technology.

Linking technology with core instructional objectives is what makes good, effective use of technology. That's the message we need to communicate. It's a process-not a number.

Margaret Honey
Center for Children and Technology

A school's vision of the education it strives to provide students contains many elements, of which technology is but one. Other elements in this vision include administrative procedures, curricula, classroom organization, teachers' pedagogical approaches, time and space designations, school-community partnerships, and logistical and social factors. Developing ways to isolate the effects of technology within a dynamic environment where so many elements work together is one of evaluation's most challenging issues.

Evaluators at the conference argued that social phenomena such as learning contain so many interacting factors that traditional experimental designs don't yield effective information. They support using evaluation designs that penetrate the effects of implementing technology at both individual, organizational, and sometimes even community levels. Evaluation designs of this type may be based on a system of learning benchmarks and other new forms of assessments that take the "localness" of evaluation into account.

The high-stake decisions linked to technology implementation pressure educators to demonstrate that technology makes a difference in student learning. Many educators fear that evaluation places their technology programs at risk if they cannot produce measurable results in a relatively short time. The message that needs to be conveyed about the effectiveness of technology is that implementation of any sort produces outcomes. These outcomes, however, will be different at different stages of implementation.

• At Mantua Elementary School in Virginia, technology is viewed not as an end in itself, but rather as a tool that augments the following four pillars of the Basic School, which are (1) the School as Community (bringing into focus how people relate to one another and work cooperatively to solve problems); (2) a Curriculum with Coherence (bringing an interdisciplinary approach to the acquisition of knowledge); (3) a Climate for Learning (providing the physical and motivational factors necessary for effective teaching and learning); and (4) a Commitment to Character (emphasizing how the school experience shapes the ethical and moral lives of children). The school community members use technology to simplify, facilitate, and enhance individualized and social learning processes within its interdisciplinary curriculum. Teachers are seen as leaders, facilitators, and mentors, well grounded in technology implementation strategies and well trained in the use of the most current computing equipment and software applications. Children exposed to interdisciplinary units of study use technology as a tool to become literate, cooperative, problem-solving, self-motivated learners and that is what Mantua is all about. What most distinguishes education at Mantua Elementary is that its students are not passive recipients of knowledge, but rather, active participants in the full educational process.
• A technology-rich environment can support initiatives focused on improving learning outcomes as shown in Union City, New Jersey. The district framed its technology evaluation in conjunction with evaluation of school reforms such as students' development of literacy, higher-order thinking and collaboration skills. With district funds and funds from Bell Atlantic and the National Science Foundation, technology became a key catalyst for school improvements that led to measurable academic achievements. But, as one of the most impoverished urban communities in the United States, Union City faced an uphill battle against state takeover with more than a plan to implement technology. Technology was just one of eight key reform strategies integral to the district's reported success in making school improvements.

To a certain extent, we are living out the decisions reflected in previous evaluation methods, which constrain our thinking about the purpose and effectiveness of technology in education.

Walter Heinecke
Curry School of Education at the University of Virginia

The issue that confronts schools is broader than technology. It is about learning and the need to find new ways to identify and measure the skills and knowledge that students gain from using technology. It is about stakeholders' needs for information beyond self-report analyses and traditional standardized testing. It is about building the capacity of teachers to evaluate technology resources and to align their uses with the learning goals and content standards of the curriculum. It is about evaluating technology implementation efforts, curriculum integration methods, and learning processes in order to make sound decisions for continual improvement. Ultimately, the issue is about involving the key stakeholders, identifying appropriate measurable indicators, and developing reliable instruments that will yield insightful and valid information about what makes educational technology effective.

The multimedia and networked capacities of the technology infrastructure are radically altering the face of technology-related practices in schools. However, the same rich diversity of technological tools that has created new learning opportunities for students has complicated the standardization of technology assessment. The fact that technology tools have undergone rapid cycles of innovation, causing constant change in the types of evaluation questions that need to be asked, compounds the difficulty even more. Educators, evaluators, and developers of measurement instruments struggle to keep current with the rush of information needs having to do with technology's effectiveness.

In order to understand changing evaluation practices, stakeholders from the policy level on down to the home, need information on how using technology changes teaching and learning, its organizational impact, and the outcomes that can be reasonably expected at different stages of technology's implementation. In short, the challenge facing educators and evaluators is to compile enough evaluation data to substantiate and articulate technology's place in student and teacher learning.

• Many schools at the conference reported that they do use standardized tests as a part of their technology assessment program, but they also look at other outcomes. Educators in the Cherry Creek School District in Colorado, for instance, are using methods they call "far from refined." They evaluate progress based on district goals such as developing students' higher- order thinking skills, promoting collaboration among students working on projects, and honing the research skills of students around real-world topics. Instead of conducting quantitative research, they rely on best practices uncovered by currently published research to guide their technology implementation. The district's philosophy regarding evaluation is that isolating technology as the cause of achievement, productivity, or change is impossible. Therefore, they evaluate systemically: looking at SAT and ITBS scores related to programs in which technology is used; analyzing results from their Technology Integration/Student Achievement Specialist Survey; using the National Educational Technology Standards to develop ways of measuring student progress in technology foundational skills-to name a few of their multiple measures.
• Technology "ubiquity" in supporting other programs has convinced skeptics of its value. Schools at the conference suggested that were it not for the access to people, resources, and ideas that technology provides, school programs, from student peer-mentoring and summer enrichment to teacher professional development, would be seriously crippled.

Who gave legislators reading and math test score to begin with? We did. We need to give them other measures, tell them how technology works, and help them see the results.

David Dwyer
On-Track Learning, Inc.

Standardized tests scores have become the accepted measure with which policymakers and the public gauge the benefits of educational investments. But educators and evaluation researchers argue that these scores say little about how to improve technology's effectiveness in schools. For this, they need information from formative evaluation.

Formative evaluation tells what technology applications work, under what conditions, and with which students. It supplies information on how technology affects student attitudes toward learning. It can show the impact of technology on promoting collaboration among diverse learners. It can track technology literacy skills development and indicate the impact of technology access. Formative evaluation can tell teachers about their students' progress toward developing the skills to access, explore, and integrate information; think at high levels; and design, experiment, and model complex phenomena.

Formative evaluation also yields information on the effectiveness of professional development activities, the adequacy of school management systems, and other issues having to do with building the school technology infrastructure.

The good news is that schools have access to more information on these questions than they might think. Evidence of technology effectiveness may lie in fewer disciplinary referrals, students' completing more complex homework assignments, a new robustness in student performances, students taking more difficult electives or requesting particular teachers and courses, increases in requests for equipment and technical assistance, declines in special education placements, lower drop-out rates, rises in college applications and acceptances, increases in student job offers, and more parent participation.

Other information collected through simple observations and questionnaires is formative as well. What technologies do teachers and students use and why? What is their attitude toward them? How has technology changed how teachers teach? How has it affected students' engagement with learning materials? Even the use of physical space, such as the rearrangement of study carrels in spaces where students can engage in learning with their peers, for example, can symbolize changes brought on by the use of technology.

The problem is not so much the lack of data. The controversy revolves around accountability measures that ask the right evaluation questions; identify appropriate data sources; systemically capture the data; and analyze, interpret, and report the data in its appropriate context.

• An educator from East Brunswick Public Schools in New Jersey maintained that of all the ways to evaluate technology integration, including hiring external consultants to conduct an evaluation, "the easiest to look at is standardized test scores." The most compelling evidence, however, is in what the district calls "secondary indicators." According to one of these indicators, when technology was integrated into a ninth-grade science curriculum, enrollment in chemistry classes swelled by nearly 500 percent, with overall enrollment in science courses growing by 17 percent.
• Other educators at the conference reported discovering innovative indicators with which to evaluate technology's effectiveness:
  • High school humanities teachers in Oswego, New York, noted more varied citations in student papers after students began doing their research on the World Wide Web.
  • The technology director in Montgomery, Alabama, observed that teachers put more detail and illustrative resources into their lesson plans than they used to.
  • Educators in Iowa used Bloom's taxonomy of cognitive learning as a guide to observing technology-integrated learning units. They found that technology-integrated learning reached higher in Bloom's hierarchy than nontechnology integrated learning.
  • A count in several districts showed that interdisciplinary instruction was more prevalent in technology-supported instruction.
  • A technology coordinator from Anderson County Schools, Tennessee, summed up technology's effectiveness this way: "I know there is great impact because if the file server drops, teachers want to call the buses and go home."
• When test scores in Blackfoot School District, Idaho, revealed that students who used technology in their coursework scored 15 percent higher than those who did not use technology, no one in the community questioned technology's role or the capital investments that the district had made. Yet, when officials couldn't pinpoint a more exact effect of technology on student learning, they knew that their evaluation of technology effectiveness had to go deeper. They analyzed each piece of their technology system, including the role of learning benchmarks in content areas and grade levels, the usefulness of professional development activities, the unique effects of particular software, and the nature and goal of instructional activities.
• Reporting students' achievement of core competencies on network technology has provoked new interest in school improvement in several communities. In almost every case, posting these competencies sparked districtwide debate about the relevance of present standards. These dialogues drove districts' examination of student achievement deeper than ever before, resulting in teachers being better informed and more committed to addressing agreed-upon competencies.

We cannot survive on the random story anymore.
Linda Roberts
Office of Educational Technology at the U.S. Department of Education

Interest in the effectiveness of technology is at an all time high. Parents want to know if children are developing a sound content base and thinking skills, and if they are going to be capable of lifelong learning in a fast-paced technological society. Teachers want to know if technology tools will help facilitate what they want to happen in the classroom. Administrators want to know if professional development activities are improving the way teachers use technology to teach. Funders, policymakers, and taxpayers want to know if technology is sufficiently promising to continue investing in it. Documenting and reporting evaluation data to meet these diverse stakeholders' need-to-know presents educational evaluators with a daunting series of challenges.

The difference in the data needs of policymakers and educators is particularly acute. While policymakers want to see data on the isolated effects of technology, educators need information that is tied to systemic practices. Policymakers tend to value summative reports documenting student achievement while teachers and administrators value formative reports documenting implementation outcomes in order to make sound decisions about their technology plans. Many kinds of data are important, but each fails to satisfy the other. The best hope of closing this gap lies in helping all stakeholders to see (1) the importance of technology as an effective component of the educational system, (2) how technology is and isn't capable of making a difference in curriculum and instruction, and 3) how innovative practices of teaching and learning with technology require multiple measures in order to verify its impact.

Comparative language speaks loudly in this regard. It is useful to show technology's effects in a tangible way by, for example, comparing the instructional practices and learning opportunities that students have with technology to instruction without technology. Open dialogue and an understanding of mutual expectations for performance throughout the technology implementation process can resolve much of what differentiates stakeholders' interests in technology outcomes. What information do these groups need? What type and how much documentation do they require? What standards of documentation are most useful to different stakeholders? These are useful questions to consider.

Finally, communicating about evaluation requires "speaking to" the stakeholder audience. What is the audience's level of technology sophistication? How knowledgeable is the audience about evaluation terms and procedures? Speaking the language of the audience-by converting effect sizes into months of academic gain, for example-influences the way people think about technology and their support for it. The technology infrastructure, itself, can be a useful tool for capturing, interpreting, and reporting data from multiple measures into understandable terms for a variety of stakeholder audiences.

• When schools in the Juneau, Alaska, instituted electronic report cards to inform parents how and in what ways their children were meeting core content standards, communications between teachers and parents surged. A middle school teacher described this new type of access to parents as "empowering" her partnership with parents to guide their children's learning.
• Educators at the conference considered parents one of their most important audiences. Once parents understood the value of technology, they became advocates. Parents, in fact, were often instrumental in moving technology into the classroom. The question that lingers is how to spread the message from parents to legislators. Some part of the answer, conference goers maintained, lies in encouraging parents to bear the message to policymaking bodies.
• Conferees voiced concerns about the media's portrayal of technology programs in many of the nation's schools and districts. While applauding the media's role in informing the public about technology, educators charged that its interest in profiling technology growth and use "in one chart on one page" shortchanges the diversity of outcomes that technology produces. Participants suggested that the best way to encourage more comprehensive portrayals of school technology programs in the media is to link technology outcomes to goals that are deeply embedded in the mission and culture of the school.

You have to show people the qualitative difference in what kids can actually do.

Eva Baker
National Center for Research on Evaluation, Standards and Student Testing at the University of California Los Angeles

Dialog among stakeholders plays a central role in evaluation efforts. Stakeholders must be attuned to common goals for the uses of technology, information needs, cultural terms, and methods for measuring outcomes. They must have consensus around roles and a clear vision of where they are going and the steps they need to take to get there.

State-level consortia, made up of representatives from many stakeholder groups, can help develop guidelines that address schools' questions such as: What are important technology-induced indicators in our state and what instruments are available to measure these indicators? Where are the gaps in evaluation needs and measurement tools within our school communities? How can district educators and university researchers collaborate to develop evaluation instruments that will measure technology's effectiveness in our schools?

Educators have known for a long time that technology can help students learn basic skills. But the tools that measure basic skills don't evaluate how technology supports students in developing capacities to think creatively and critically and vice versa. There is a need to develop additional evaluation tools that can help measure whether students are learning the "new basics" such as computer literacy, collaborative teamwork skills, and lifelong learning abilities.

Left to themselves, schools have little time to develop and test such evaluation tools. While the successful evaluation of a school's technology does not necessarily require that researchers and evaluators be on the scene, seeking such expertise can be helpful, especially in evaluations that encompass several buildings or districts. Many universities offer technology evaluation expertise. In addition, regional educational laboratories and technology education consortia allocate many of their resources to helping schools address evaluation issues. Other for-profit and not-for-profit organizations can also be helpful. Still, it is difficult for schools to identify what assistance is available. The field is ripe for developing scalable approaches, tools, and strategies for evaluating the effectiveness of educational technologies.

The most useful tools yield information that is specific to the given student population and that allows teachers to track students' progress over time. Tools also need to measure those aspects and outcomes of learning that would otherwise be unattainable without the use of technology. Evaluation that demonstrates what students can do with technology that they couldn't do before access to the technology shows impact. For example, performance measures-observations of what students do and where they go on the Internet and how students collaborate with each other-help teachers track the impact of technology on student learning. Other measures that tap into education's broader curriculum aims include projects, essays, and extended performances.

• Several school district representatives reported replacing student technology competency requirements with technology/content area integration standards as a basis for benchmarking grade-level technology integration. Their rationale was that this shift emphasizes technology's supportive role in teaching and learning rather than making technology use an end in itself. These educators believe that indicators articulating the components of a model instructional unit in fourth-grade science, for example, are more useful than technology competencies students should demonstrate at the fourth-grade level.
• Early in the conference, it became clear that technology has spurred new terms and/or word meanings in our vocabulary. The term "engaged learning," for instance, had a very different meaning for participants from Chicago Public Schools than it did for educators in Cherokee County School District in Alabama. Similarly, when one participant referred to "performance standards," educators from New Hampshire's Campton School District envisioned a very different set of standards than did their colleagues from the Okaloosa County School District in Florida. These exchanges illustrated the need to come to consensus on the terms and language of the evaluation process. Terms such as "technology integration," "benchmark," "core competency," "alternative assessment," and even "evaluation" and "student achievement" elicit different meanings from a range of educators and, unless they are made clear, can undermine evaluation efforts.

Technology has revolutionalized what teachers do. It has added new breadth and depth to instruction. This, in turn, has transformed the role of the classroom teacher. In reformed educational settings, teachers guide students in using telecommunications to interact with astronauts in space, searching the Internet for up-to-the-minute information, and programming technology systems to help solve local or global problems.

The countless hours teachers spend observing and interacting with students makes teachers a rich source of data about the impact technology has on student learning. Teachers are the first to recognize increases in students' self-esteem and confidence, enhanced content area understanding, and more informed and empathic responses to world events as a result of using technology. This new role for teachers underscores the need for high-quality professional development in the use of technology and in determining what and how students learn best with technology tools.

What teachers know about their students and about technology determines their competence in day-to day classroom decision-making. Good teachers evaluate their students and make decisions about how technology can boost their learning on a daily basis. Do students have access to the appropriate technology resources and tools? Are students using the technology efficiently? What kinds of learning tasks will challenge students' creative and critical thinking? In this new technology environment where there is not one instructional strategy but many, teachers need to know how to manage interactive group dynamics as well as technological systems.

Professional development in schools that have implemented and evaluated educational technologies successfully helps teachers link effective uses of technology to impacts on student learning. Evidence of technology literacy, faculty meeting agendas, lesson plans, and classroom observations are all ways to determine a teacher's grasp of technology as a learning tool. The most useful program evaluation is one in which a strong formative element examines the connection between instructional practice, technology uses, and learning outcomes.

Teachers are integral to the process of evaluating technology initiatives. They can act as partners with researchers to identify the sometimes very subtle impacts associated with technology uses. Teachers can also play key roles in measuring and documenting changes in student learning as they occur. Some of the best results in evaluating technology come from schools recognizing and harnessing the expertise teachers have in identifying technology-induced learning outcomes.

Teachers who have learned to use technology effectively in the classroom are convincing their colleagues of technology's potential. Teachers training teachers to evaluate the usefulness of technology in the classroom remains a potent professional development strategy.

• Evaluators must learn to trust teachers' ability to determine and describe technology's "ripple" effects, Margaret Honey, director of the Center for Children and Technology, explained during the conference. Success in studying school technology programs, according to Dr. Honey, often hinges on teachers contributing to the development of research questions and sharing ideas on how to record key indicators of effectiveness.
• Lennox School District in southern California builds teachers' capacity to evaluate student learning with technology by having teachers collaboratively score students' work. Examining student products together builds consensus among teachers on the curriculum's core goals and the types of assessments that measure the achievement of those goals.
• The nature of the questions that teachers ask their technology coordinators are data for evaluation. Their questions can indicate a school's position along the continuum of technology implementation. A technology coordinator from Helena, Montana, observed that when schools first deployed technology, teachers' questions centered on getting the hardware to work. Only a couple of years later, these same teachers' questions revolved around content and accessing resources through the network.

Our goal should be first, to understand the conditions of pro-social technology use and second to employ that understanding for learning improvement. Both require more penetrating analysis than has heretofore been the standard.

Dale Mann
Interactive, Inc.

Today's classrooms are expected to be technologically up to date. The same should be true for the policies that govern technology uses. When federal, state, or local district or building level policies do not keep up with classroom practices, innovative and effective practices can grind to a halt. To this end, educators have a leadership role in using evaluation information to shape the conversation around the kinds of policies that are most supportive in validating best practices that enhance the work of the school community.

Policy issues rise to the surface around data. Who should have access to what data in the student information system? In theory, information about a student's family situation, for example, can help teachers understand and respond to student learning and behavioral problems. With today's information technology networks, accessing all kinds of personal family information in student files is possible-but what are the ethical policy implications for doing so?

Still another example of how the lack of policies can slow down reform has to do with the equitable allocation of computers and other technology resources. Does a school distribute computers to students who need them the most, or to those students whose teachers show the most computer proficiency? What is a school's responsibility for out-of-school computer access? How are scarce technology lab resources scheduled for use by the school community during and after school hours? Does a school have a policy governing the use of its technology to address adult technology literacy needs?

Many school communities have recognized the need to create and enforce Internet usage policies, for instance, but what other less obvious technology-related policies are required to support and govern the best practices associated with implementing technology innovations into the school system? Local educators have the experience to help shape such questions and define successful practice for state and federal policymakers. These policymakers can then respond by developing policies that support the effective use of technology at the local level on a systemic basis. An important part of policy reform is to give policymakers a common language and data with which to speak to their constituents so that support for effective uses of technology will be widespread throughout the community.

• Kayenta Unified School District (KUSD) is a small rural Navajo community located in an isolated region in the northeastern corner of Arizona, near the magnificent Monument Valley. This school district serves 2,600 students from Kayenta and several other smaller, more rural communities. The nearest public library is 100 miles away, while the nearest museums, bookstores, and universities are 150 miles. This isolation has provided the motivation to use technology to assist in increasing literacy, while permitting students to sustain critical elements of the rich traditional life of generations of Diné. After ten years of hard work building their technology infrastructure, KUSD presently has all six schools and administrative buildings, and all classrooms, offices, and administrators connected to an Internet/intranet e-mail system. Determining ways to complement traditional instruction and community values with the global access provided by the Internet has been a compelling policy, as well as curricular, challenge faced by Kayenta and other rural isolated school districts. Kayenta distance learning policies opened the school community to the outside world in many ways.
• More and more as teachers and parents gain access to e-mail communications and evaluation data via the Internet, schools are finding it difficult to maintain current information policies governing information access. A case in point is the use of e-mail for parent-teacher communication. While in theory, frequent communications between parents and teachers is a positive move forward, practitioners pointed out that having to respond to frequent requests from parents about their children's schoolwork tears them away from instructional planning time. This raises policy questions about teachers' obligations to respond to individualized e-e-mail requests from parents.
• Educators at the conference demonstrated their broadened view of outcomes by recounting stories of timely access, attitudinal change, and increased motivation. Most initiatives in their early stages feature such stories. Their telling is an important step in shaping realistic public and legislative expectations for technology evaluation and supportive policies.

Schools that have partnered with other schools, universities, and educational service agencies to collaborate on technology planning, implementation, and research show compelling and productive applications of technology. Now comes the call for rigorous technology evaluation designs that are innovative and relevant to showing its impact.

Researchers and educators are finding ways to partner in evaluating the technology initiatives that they've instituted. Such partnerships are revolving around many different purposes. Universities are partnering with schools to construct the next generation of evaluation tools and processes. State and federal governments are beginning to reserve grant monies for evaluation activities in order to identify and disseminate information about technology practices that work and that may benefit schools in other contexts. Policies are beginning to be discussed that will support these and other innovative practices.

The Secretary's Conference on Educational Technology: Evaluating the Effectiveness of Technology took a step forward in bringing together federal, state, and local evaluators with school practitioners to understand the many puzzle pieces involved in evaluating the effectiveness of technology in education. In many respects, the discussions and presentations at the conference raised more questions than they answered. In other respects, the diversity of research and best practices shared by participants represents the "state-of-the-field" in evaluating the effectiveness of technology in schools. The need to evaluate the effectiveness of technology in schools fuels a vast potential for collaboration among schools, universities, research organizations, businesses, and community groups.

A multimedia CD-ROM will be available for those wishing to further delve into these critical issues raised at the conference (see the order form). In addition to providing examples of and expert commentary on spotlight school evaluation practices, it also contains templates for guiding school leaders' thinking about designing an evaluation plan. The U.S. Department of Education has launched a conference Web site at http://www.ed.gov/Technology/TechConf/1999/. Check it for the conference proceedings, announcements, and online events. In addition, follow-up regional conferences are being planned for the millennium year to focus more in-depth on the impact technology has on schooling and the evaluation needs to show the nature of those impacts.

Conference Agenda | Spotlight Schools | Spotlight School Contacts | Conference White Papers | Featured Speakers | Acknowledgements | Conference Summary

Return to Conference Main Page

Page last updated on November 30, 1999 by [eid]