Sherri Raschke10/30/20018:49:20 PMoops I got kicked off
Bruce Long10/30/20018:49:35 PMI did too.
Paulette Mangold10/30/20018:49:37 PMI think I am back
Mark Hawkes10/30/20018:49:40 PMWe all got washed out Sherri, my doing since we froze up.
doug greenway10/30/20018:49:55 PMWelcome again
Sherri Raschke10/30/20018:50:01 PMok so I don't get a complex
Mark Hawkes10/30/20018:50:19 PMRight Doug, welcome again
Bruce Long10/30/20018:50:34 PMI accepted calls from several people and now they are back.
Mark Hawkes10/30/20018:50:49 PMThanks Bruce
doug greenway10/30/20018:50:55 PMMark, Can we ask questions to start or can that come later?
Mark Hawkes10/30/20018:51:02 PMNow all we need is our guest of honor
Tammie Munsen10/30/20018:51:11 PMI'm back
Mark Hawkes10/30/20018:51:44 PMLet's get PRachee back on line, and give her the first word, then we'll pop questions at her.
Sherri Raschke10/30/20018:52:09 PMsounds like a plan
Mark Hawkes10/30/20018:53:04 PMOkay, let's talk about your DD2's how are they coming?
Christi Harter10/30/20018:53:24 PMhello
Paulette Mangold10/30/20018:53:31 PMgood here
Joanne Ustad10/30/20018:53:43 PMYou mean stage 2s
Mark Hawkes10/30/20018:53:47 PMAre you making sence of this process of data collecyion and analysis?
Mark Hawkes10/30/20018:54:05 PMOh, thanks Joanne, right, stage 2's! Whew
Cliff De Long10/30/20018:54:54 PMAndy refurbed ours - Stacy and I have reviewed it - are looking to meet on it Thursday. Data Collection and Analysis make sense.
Heidi Noem10/30/20018:55:08 PMAe we supposed to include issues such as validty and reliability with our data collection as that was mentioned in the few of the articles??
Mark Hawkes10/30/20018:55:21 PMThanks Cliff, anyone else with a progress report?
Bruce Long10/30/20018:55:27 PMIs Prachee back yer. I just accepted a call from her.
Christi Harter10/30/20018:56:03 PMDoug, Bruce and Christi have revised Phase I and are meeting to complete Phase II Wed. night
Mark Hawkes10/30/20018:56:32 PMGood Christi, sounds like progress.
Christi Harter10/30/20018:56:41 PMyes, thanks for all the tips
Mark Hawkes10/30/20018:57:20 PMGood, glad they helped Christi
Mark Hawkes10/30/20018:58:01 PMOne of the things I'll look for are how much consistency there is between your data sources and the questions you ask
Christi Harter10/30/20018:58:47 PMokay, what else?
stacy phelps10/30/20018:59:03 PM[Private] have i missed much
Mark Hawkes10/30/20018:59:07 PMI would hope you're really clear on the type of sources you're using too!
Mark Hawkes10/30/20018:59:15 PMHi Prachee, ready to go?
Cliff De Long10/30/20018:59:17 PM[Private to stacy phelps] nope
Heidi Noem10/30/20018:59:35 PMMark, how much detail are we supposed to go into regarding validity and reliability issues as that was brought up in several readings.
Mark Hawkes10/30/20018:59:38 PMSorry we got bumped, just another day on the chat.
Joanne Ustad10/30/20018:59:50 PMcan you elaborate on on the consistancy thing. Do you mean that they are connected?
Mark Hawkes10/30/20019:00:22 PMNo much Heidi, I assume that the validty and reliability issues will take care of themselves once you get data saturation
Cliff De Long10/30/20019:01:09 PM[Private to stacy phelps] A common 21st century affliction that - data saturation
Mark Hawkes10/30/20019:01:15 PMJoanne, I mean that you're aksing the right kind of questions to the right kind of sources
Joanne Ustad10/30/20019:01:27 PMok thanks
Cliff De Long10/30/20019:01:34 PM[Private to stacy phelps] A common 21st century affliction that - data saturation
Mark Hawkes10/30/20019:01:41 PMYou probably would ask about professional development opportunities to students or parents, right?
Mark Hawkes10/30/20019:02:18 PMLooks like prachee is caught behind enemy lines. Is anyone getting her posts?
Paulette Mangold10/30/20019:02:31 PMI'm not
Christi Harter10/30/20019:02:31 PMno
Sherri Raschke10/30/20019:02:36 PMnope
Jennifer Nelson10/30/20019:02:46 PMno
Bruce Long10/30/20019:02:47 PMNope, but I am getting a lot of calls still: Andy , Sandy, Cherykl
Mark Hawkes10/30/20019:02:58 PMLisa Tolliver, are you on chat?
Cliff De Long10/30/20019:03:07 PMA common 21st century affliction that - data saturation ... Please define your meaning redundantly.
Mark Hawkes10/30/20019:03:26 PMRight Bruce, I've lat them in several times already. Must not be registering my access.
Heidi Noem10/30/20019:03:28 PMMark, I haven't seen any messages from her either.
stacy phelps10/30/20019:03:34 PMit took me about a half hour to get into the chat
Mark Hawkes10/30/20019:03:52 PMSorry Stacy, thanks for haning in there.
Christi Harter10/30/20019:03:57 PMme too, Stacy
Mark Hawkes10/30/20019:04:04 PMSaturation--should have explained it in the first place.
Bruce Long10/30/20019:04:04 PMCan we invite Prachee back?
Mark Hawkes10/30/20019:04:18 PMWe can try Bruce.
Sherri Raschke10/30/20019:04:42 PMher name shows on my screen now
Sherri Raschke10/30/20019:05:18 PMis she back or am I out?
Mark Hawkes10/30/20019:05:22 PMWhen you have your sources repeating the same themes with the same intensity, you've reached saturation.
Cliff De Long10/30/20019:05:29 PMyou are still here Sherri
Mark Hawkes10/30/20019:05:34 PMYou're in Sherri
Christi Harter10/30/20019:05:35 PMMark, do expect a table for the data section?
Cliff De Long10/30/20019:05:37 PMthanks mark
stacy phelps10/30/20019:05:42 PMseems like more people are back in. our count is up to 18 now and well 17 now
Mark Hawkes10/30/20019:05:52 PMI would like to see that Christi, a table organizs information so much better
Sherri Raschke10/30/20019:05:59 PMoh oh 16
Christi Harter10/30/20019:06:05 PMokay, thanks
Joanne Ustad10/30/20019:06:10 PMit is easier to work with too
Bruce Long10/30/20019:06:17 PMWhat was Christi's question?
Cliff De Long10/30/20019:06:29 PM[Private to stacy phelps] tabular works for us - that's what we put in on the Phase 1
Christi Harter10/30/20019:06:35 PMMark verified that a table for data sources is needed
Mark Hawkes10/30/20019:06:38 PMWhen you've reached that point of saturation, additioal data collection is likely not to be fruitful because it will not yield any new insights
stacy phelps10/30/20019:06:40 PMdoes this chat have a capacity, cause it seem as if someone has to get kiccked out before others can get in? i encounter that almost all the time
Mark Hawkes10/30/20019:07:02 PMI don;t think so Stacy. I'll check into it.
Christi Harter10/30/20019:07:12 PMsubliminal message
Mark Hawkes10/30/20019:07:34 PMSaturation is generally associated with qualitative techniques like interviews and focus groups.
Heidi Noem10/30/20019:08:24 PMMark, when we say we're going to survey the teachers, do we have to have our actual survey completed to show you??
Cliff De Long10/30/20019:08:39 PM[Private to Prachee Mukherjee] Wow - I'm getting saturated - how about you Pratchee?
Christi Harter10/30/20019:08:42 PMso once you get a recuring theme, quit collecting?
Joanne Ustad10/30/20019:08:59 PMa little is good, too much gets in the way of understanding
Bruce Long10/30/20019:09:02 PMYou said we didn't need to design the tools, right Mark?
Mark Hawkes10/30/20019:09:11 PMHeidi, no, there will be little instrumentation required of you . . .
Mark Hawkes10/30/20019:09:44 PMIn a few weeks however, you will come up with a software evaluation protocol that is a part of you evaluation plan
10/30/20019:10:07 PM30 minutes later, I'm back in!
Mark Hawkes10/30/20019:10:09 PMChristi, right. Generally, saturation signals a need to move into other topics, areas.
Cliff De Long10/30/20019:10:32 PMhardware, netware, instructional methodology to follow?
Bruce Long10/30/20019:10:36 PMCould saturation mean you are asking the wrong questions?
Mark Hawkes10/30/20019:10:39 PMIS that you Prachee? Or Sandy?
Prachee Mukherjee10/30/20019:10:58 PMI'm here Mark... where did you guys disappear?
Mark Hawkes10/30/20019:11:08 PMRight Cliff, Our hardware and netware will be part of a technology audit process. Already though, some plans have that built in.
10/30/20019:11:11 PMI'm back through Bruce Long
Cliff De Long10/30/20019:11:12 PMcyberspace - fell right off into it
Mark Hawkes10/30/20019:11:26 PMHio Prachee, somehow you got caught behind enemy lines!
Mark Hawkes10/30/20019:11:34 PMThree cheers for Prachee not giving up on us!
Sherri Raschke10/30/20019:11:35 PM its magic black evil magic
Mark Hawkes10/30/20019:11:38 PMHip hip . . .
Cliff De Long10/30/20019:11:44 PMhoo-
Cliff De Long10/30/20019:11:45 PMray
Mark Hawkes10/30/20019:11:58 PMOkay, that will do.
Mark Hawkes10/30/20019:12:09 PMPrachee, tell us about what you do.
Prachee Mukherjee10/30/20019:12:14 PMAs long as I'm getting your messages I guess I'm okay
Prachee Mukherjee10/30/20019:12:35 PMWhat I do ... I work as an evaluator for a school district, that's the formal part
Mark Hawkes10/30/20019:12:50 PMWhat does that all entail?
Prachee Mukherjee10/30/20019:12:51 PMBasically people think of me whenever they need to do surveys
Christi Harter10/30/20019:13:17 PMa high school?
Cliff De Long10/30/20019:13:23 PMAre you an internal evaluator?
Prachee Mukherjee10/30/20019:13:23 PMOn the other hand, I get requests from the Board of Education or Program Directors/Curriculum Supervisors to evaluate their programs
Bruce Long10/30/20019:13:28 PMAre you a consultant or an employee of the district?
Prachee Mukherjee10/30/20019:13:30 PMI'm an internal evaluator
Mark Hawkes10/30/20019:13:40 PMAh surveys, for folks who think eveything is a nail, they'll use a hammer
Prachee Mukherjee10/30/20019:13:41 PMAn employee of the district
Christi Harter10/30/20019:13:49 PMdo you work outside of your district?
Cliff De Long10/30/20019:13:59 PMon a consultant basis?
Prachee Mukherjee10/30/20019:14:06 PMSometimes...I do freelance
Christi Harter10/30/20019:14:08 PMhow many people to you work w/ to do an eval?
Mark Hawkes10/30/20019:14:43 PMINternal in the sense that she is employed by the district, to ply the lens of evaluation on their programs, curriculum, etc.
Prachee Mukherjee10/30/20019:14:49 PMIt depends... usually it involves 2-5 central administators who get it going
Christi Harter10/30/20019:15:01 PMand how long does it take? normally?
Mark Hawkes10/30/20019:15:20 PMPrachee, how is it being an internal evaluator? A folks more or less apt to use your evaluation findings?
Prachee Mukherjee10/30/20019:15:34 PMIt can also involve parent/community members... we have acoountability advisory committee members
Sherri Raschke10/30/20019:15:46 PMso that we have an idea, how large is the districts enrollment
Prachee Mukherjee10/30/20019:16:24 PMTypically the timeline is a school year or a semester
Prachee Mukherjee10/30/20019:16:43 PM35K students
Mark Hawkes10/30/20019:16:52 PMThat's lots of surveys!
Christi Harter10/30/20019:17:06 PMwhat type of projects do you work with?
Mark Hawkes10/30/20019:18:06 PMI think I know part of that answer Christi, internal evaluators do a lot of evaluation on development programs. IS that true Prachee?
Prachee Mukherjee10/30/20019:18:28 PMI've done things like evaluating the implementation of a new math textbook, kit-based science curriculum, interactive-video based foreign lang ;program
Cliff De Long10/30/20019:18:38 PMGiven the volume that you work with - that seems to be a large amount of tabulation - are you beginning to implement web-based surveys - do you contract with ETS, or just scan bubble sheets?
Christi Harter10/30/20019:18:44 PMinteresting
Mark Hawkes10/30/20019:18:54 PMPretty eclectic
Prachee Mukherjee10/30/20019:18:56 PMdo you mean development as in pilot programs?
Mark Hawkes10/30/20019:19:06 PMDevelopment in teacher training
Prachee Mukherjee10/30/20019:20:04 PMI do work on staff dev evaluation... not as extensive as I'd like it to be because we never get into 'impact' evaluation...
Mark Hawkes10/30/20019:20:21 PMWe like that word--"impact"
Prachee Mukherjee10/30/20019:20:37 PMThe web is not our medium yet because schools still have a lot of problems with technology
Prachee Mukherjee10/30/20019:20:49 PMThe network is not always up if you know what I mean
Mark Hawkes10/30/20019:21:15 PMAre there poilcy or access issues with technology in your district Prachee?
Prachee Mukherjee10/30/20019:21:21 PMWe have two automatic scanners and software to design our own surveys... more friendly than bubble sheets
stacy phelps10/30/20019:21:22 PMWell it would seem that an evaluation of the technology and it services is in order?
Prachee Mukherjee10/30/20019:21:54 PMcertainly an evaluation of our technology plan implementation is long overdue
Mark Hawkes10/30/20019:21:55 PMWell put Stacy, and you happen to be closer to C Springs than the rest of us!
Prachee Mukherjee10/30/20019:22:12 PMBut the district chooses to skirt that issue... it is too politicized
Bruce Long10/30/20019:22:14 PMIt could be a class project!
Prachee Mukherjee10/30/20019:22:17 PMLet me tell you a real life story
Mark Hawkes10/30/20019:22:32 PMSo when you look at technology use in the district Prachee, what kind of evidence are you gathering?
stacy phelps10/30/20019:23:11 PMIt would seem that here in South Dakota we are so overwhelmed by technology that it is just seems to be natural integration or force of technology. i guess i forget that the rest of the country is still in an infancy in regards to technology
Prachee Mukherjee10/30/20019:23:12 PMThe district needed to determine the bond amount to fund its tech program... called in the business community to detremine the bond size and the business community said we should promise the public a guarantee in test rsults
stacy phelps10/30/20019:23:21 PMthat does not mean we use it well though...
Christi Harter10/30/20019:23:41 PMwow
doug greenway10/30/20019:23:54 PMOur staff can get real tired of another survey, what other instruments r methods are used in your district to gather feedback?
Mark Hawkes10/30/20019:24:27 PMGuarantees are easy to make, do they expect them to be kept?
Prachee Mukherjee10/30/20019:24:39 PMWhen I have looked at technology in the past it has been at bits and pieces. One elementary school was being used as a pilot by MCI to showcase its educational capability/support and I looked at teacher attitudes toward technoogy in the beginning
Cliff De Long10/30/20019:24:58 PMcontract = offer and acceptance - hopefully the district told the business community where to put their bond issue and went with e-rate and fed. funding
Prachee Mukherjee10/30/20019:25:02 PMThen we moved to use of technology in the classroom and in personal/professional tasks. It was a 2-year project
Christi Harter10/30/20019:25:07 PMwho obtains your results of an eval?
Mark Hawkes10/30/20019:25:24 PMYEs, some of us are looking at that, becasue as Stacy suggests, that has a lot to do with how well the technology might be used
Prachee Mukherjee10/30/20019:25:31 PMResults are public information, shared with the clients and schools
Cliff De Long10/30/20019:25:38 PMURL of MCI results?
Mark Hawkes10/30/20019:25:39 PMTechnology asa productivity tool.
Christi Harter10/30/20019:26:05 PMwho're your stakeholders, normally?
Prachee Mukherjee10/30/20019:26:22 PMEvaluation results are not on the district website or any website... but I can get them to you if you're interested
Prachee Mukherjee10/30/20019:26:50 PMStakeholders are teachers, principals, parents and students. Support staff and community members at times
Cliff De Long10/30/20019:26:55 PMemail to Mark please...
Mark Hawkes10/30/20019:27:12 PMI'll distribute Cliff
Prachee Mukherjee10/30/20019:27:51 PMWith increasing demands on staff time any kind of data collection becomes an extra chore for the staff
Cindy Rall10/30/20019:28:07 PMWhat methods do you use to analyze the data that is collected?
Prachee Mukherjee10/30/20019:28:12 PMI have tried but not had much luck with data collection from staff via email
Bruce Long10/30/20019:28:24 PMDo you use focus groups?
Prachee Mukherjee10/30/20019:28:33 PMYes focus groups and interviews are used
Mark Hawkes10/30/20019:28:35 PMRight, teachers are so pressed as it is. Few of them want to answer questions from a nosy evaluator.
Cliff De Long10/30/20019:28:59 PMSerious or cynical Mark?
Christi Harter10/30/20019:29:00 PMhow do you make the teachers except you and your work?
Prachee Mukherjee10/30/20019:29:02 PMThe Superintendent had me do focus groups to gather feedback on his first year's performance from various employee groups
Cindy Rall10/30/20019:29:33 PMThat had to be a touchy subject
Bruce Long10/30/20019:29:36 PMDid you work with some one inthe focus gorups or did yo do it all?
Cliff De Long10/30/20019:29:37 PM[Private to Prachee Mukherjee] waste of taxpayer dollars that
Paulette Mangold10/30/20019:29:39 PMDid that help him to make the second year better
Mark Hawkes10/30/20019:29:45 PMPrachee, are you a single evaluator, or what does the evaluation staff look like?
Prachee Mukherjee10/30/20019:29:45 PMUsually the teachers aceept me based on what their content level supervisor or dept head tells them about the importance of the evaluation I'm doing...
Bruce Long10/30/20019:29:51 PMGood question Paullette
Prachee Mukherjee10/30/20019:30:05 PMI'm a single evaluator in the sense I'm the only one with formal training in it
Prachee Mukherjee10/30/20019:30:35 PMI have 2 assistants... one of them is studying for a degree in Social Work and I have stated using her as an assistant for focus groups, training her as I go along
Prachee Mukherjee10/30/20019:30:54 PMThe other asst works mainly with data entry and prep stuff
Bruce Long10/30/20019:31:02 PMAs a savvy notetaker?
Bruce Long10/30/20019:31:58 PMOr do you record focus gorup sessions?
Prachee Mukherjee10/30/20019:32:02 PMFor the sup's evaluation, the themes that emerged were consistent across at least 2 groups for any theme and he found it treemendously useful, especially the comments in peopl's own words
Tammie Munsen10/30/20019:32:45 PMpeople's own words seem to mean more and are more effective than numbers
Prachee Mukherjee10/30/20019:32:50 PMWe record focus group sessions and transcribe them but I have a pretty good memory and I always take my own notes also and work with them right after the focus group .. at least do a rough draft
Mark Hawkes10/30/20019:33:22 PMNothing more powerful than a school board member than a constituates words.
Prachee Mukherjee10/30/20019:33:31 PMExactly.
Bruce Long10/30/20019:33:32 PMA lot to keep track of.
Prachee Mukherjee10/30/20019:33:54 PMI also have learned that 'focus groups' is a term loosely used by others, in a non-research sense
Mark Hawkes10/30/20019:33:59 PMPrachee, what advice do you have for us busy teachers but novice evaluators?
Cliff De Long10/30/20019:34:28 PM[Private to Prachee Mukherjee] You could just take in a laptop and have it transcribe the session using voice recognition... Would that save time on data entry?
Prachee Mukherjee10/30/20019:34:47 PMI think what i've ;earned is to keep it simple... sounds really naive and it used to annoy me at first but
Christi Harter10/30/20019:35:26 PMthank you for being here, you were very informative
Bruce Long10/30/20019:35:29 PMOK Mark, we will work on simple!
Cliff De Long10/30/20019:35:37 PMK.I.S.S.
Heidi Noem10/30/20019:35:39 PMSimple is good!
Bruce Long10/30/20019:35:48 PMYes, thanks Prachee.
Prachee Mukherjee10/30/20019:35:50 PMIf you can ask the really important question and leave out the stuff that 'seems' interesting... the response burden is reduced
Paulette Mangold10/30/20019:36:08 PMThat also makes tallying easier I would think
Prachee Mukherjee10/30/20019:36:10 PMAnd of course unobtrusive data collection
Mark Hawkes10/30/20019:36:15 PMGood advice that last part.
Andy Conrad10/30/20019:36:19 PMmakes sense!
Cliff De Long10/30/20019:36:20 PMthat applies to instructional chat quite nicely
Mark Hawkes10/30/20019:36:45 PMCliff, very observant.
Prachee Mukherjee10/30/20019:36:46 PMTeachers are inundated with paperwork so it's worthwhile to find out what they're already doing and use that to answer evaluation questions
Mark Hawkes10/30/20019:37:24 PMThose are extant data sources. Things that we already have like bsentism rates, roles, computer logs, etc.
Mark Hawkes10/30/20019:37:32 PMPrachee, you've been a great chatter. Thanks for joining us.
Prachee Mukherjee10/30/20019:37:37 PMI ahve found that interviews are something teachers love... rthey say they can only spare 30-40 minutes but then they can't stop talking
Paulette Mangold10/30/20019:37:41 PMThen observation formal, informal is a big part of your evaluation process?
Prachee Mukherjee10/30/20019:38:06 PMOkay... thanks a lot for having me... this was fun.
Christi Harter10/30/20019:38:12 PMthank you
Jennifer Nelson10/30/20019:38:18 PMthanks
Mark Hawkes10/30/20019:38:19 PMI see Bruce starting to fall asleep
Tricia Neugebauer10/30/20019:38:21 PMthank you, Prachee
Mark Hawkes10/30/20019:38:26 PMSo, we have to get him to bed.
Paulette Mangold10/30/20019:38:27 PMThank you
Andy Conrad10/30/20019:38:31 PMParchee, say HI to Bill !
Heidi Noem10/30/20019:38:31 PMThanks.
Bruce Long10/30/20019:38:36 PMNite everyone.
doug greenway10/30/20019:38:40 PMgood night
Tammie Munsen10/30/20019:38:40 PMThank you
Mark Hawkes10/30/20019:38:46 PMSorry the chat started off so rough. BUt we had a strong finish
Tammie Munsen10/30/20019:38:49 PMsee ya
Prachee Mukherjee10/30/20019:38:54 PMI will Andy. Gouys you have a geat teacher in mark I'm sure you all know that already.
Christi Harter10/30/20019:38:54 PMbye all, take care
Cindy Rall10/30/20019:38:58 PMthanks