Assessment Report Computer Assisted Debate Project 2005-2006 The Computer Assisted Debate (CAD) Project began in June 2004 targeting middle school students from two Atlanta Housing Authority communities (Bowen Homes and Hollywood Court) who attend Benjamin S. Carson Honors Preparatory School. In June of 2005, the program expanded to students from two additional AHA communities (Bankhead Courts and Collier Heights) who attend Harper Archer Middle School. One hundred and eighty-four students attended the CAD program during 2005-2006. With the longest running CAD program, Benjamin Carson became the primary focus on this assessment report. All measures were applied to the Carson students who were either present the last week of the CAD program or had time flexibility during finals week to complete the various measures. The Harper Archer students in attendance during their last week at CAD were given a self-report survey and the Personal Willingness to Communicate Form only. The data reported here focuses on the Carson students unless otherwise noted. School Engagement School attendance by the CAD students improved dramatically in 2005-2006. On average, the students dropped from eleven absences in 2004-2005 to 4 absences in 20052006. The steepest decline came from eighth graders, who have the longest involvement with the CAD program. On average, they dropped from 14.8 absences per child in 20042005 to 4 absences in 2005-2006. Self-reports from the students indicate that their shift to a more active involvement in school may prove to have lasting benefits. CAD students at Harper Archer and Benjamin Carson were asked to rate their desire to go to college. Eighty-nine percent of the students reported their desire as excellent and another six percent marked that their desire was good. Only five percent reported that they were neutral, and none indicated that their desire was only fair or poor. Student Conduct The positive impact of the CAD program on student behavior in school continued into the 2005-2006 school year. This study replicated the methodology of the 2004-2005 assessment report that compared the CAD students with the entire student population at Carson on documented disciplinary referrals. Disciplinary referrals constituted out of school suspensions, in-school detentions, and mandatory teacher/parent conferences. For the school wide population, disciplinary referrals dropped 11% from 2004-2005 to 20052006. On a per student basis, the average number of referrals dropped from 2.8 to 2.4 over the course of the school year. The CAD students, however, once again outperformed their peers in improved conduct. Disciplinary referrals for the CAD students dropped 43% from 2004-2005 to 2005-2006. On a per student basis, the referrals dropped from 2.1 to 1.2. The sharpest change occurred in the 8th grade CAD students, who have participated in the CAD program for two years. Their referrals dropped 82%, from 3.5 on average per child to .6. Also notable is the fact that the CAD students appear to connect debate as the causative factor behind their change in behavior. CAD students at Carson and Harper Archer were asked to assess the effect of debate on their behavior, conduct and conflict resolution skills. Fifty-three percent reported the effect to be excellent, while another 25% thought it was good. Only 22% considered the effect to be neutral, fair or poor. Communication Apprehension To evaluate the communication apprehension level of the CAD students, the assessment team used a modified version of the Personal Report of Communication Apprehension form. The Personal Report of Communication Apprehension is a scale developed by Dr. Robert McCroskey of West Virginia University and approved by the National Communication Association as a valid measure of student’s perceptions of their communication apprehension. The only modification to the scale was to change the reading level of the items to a 6th grade level. Sixty-eight students completed the scale at Carson and Harper Archer. The CAD students reported reduced communication apprehension in general, in meetings, in group settings, in their interpersonal dyadic relationships, and in public speaking contexts. The following charts report the average change in student reports of their communication apprehension from 2004-2005 to 2005-2006. The instrument reports the student’s anxiety on a five-point scale, with five being the highest level of anxiety. Reduced Communication Apprehension Overall 2.662.2600.511.522.532004-20052005-2006 Reduced Communication Apprehension in Group Settings 2.762.2500.511.522.533.52004-20052005-2006 Reduced Communication Apprehension in Meetings 2.562.200.511.522.532004-20052005-2006 Reduced Communication Apprehension in Interpersonal Dyadic Relationships 2.442.1822.12.22.32.42.52004-20052005-2006 Reduced Communication Apprehension in Public Speaking Contexts 2.892.4222.22.42.62.832004-20052005-2006 Grade Point Averages CAD students at both Harper Archer and Benjamin Carson were asked whether their grades had improved since they had begun debate. Thirty-three percent indicated that their grades had greatly improved and forty-five percent indicated their grades were better. Eleven percent saw no change and none of the students indicated that their grades were either worse or much worse. Final grade reports at Carson verified the student’s perceptions. Eighth graders saw their average cumulative GPA increase by 1.4 points on a 100 point scale, which augments the 4 point increase they experienced from their first year in the program in 2004-2005. Seventh graders increased their cumulative GPAs on average by 3 points (this figure excludes the performance of one student who experienced a substantial drop in his academic performance due to the death of his guardian and the resulting upheavals in his home placement). The grades of the 6th grade students were not calculated because the elementary schools that feed into Carson have a variable set of methods for reporting grades that do not compare with Carson’s 100-point cumulative GPA scale. Knowledge of Current Events CAD students at Harper Archer and Benjamin Carson were asked to evaluate their desire to keep up with current events. Sixty-seven percent of the students rated their interest to be either excellent or good. Eleven percent of the students were neutral and eleven percent rated their desire as fair. None of the students indicated their knowledge of current events was poor. Given their high level of interest, the students were asked to evaluate their actual knowledge of current events. Seventeen percent reported their knowledge to be excellent, 43% as above average, 34% as fair, 3% as below average, and 3% as poor. One factor arguably contributing to the current events knowledge was the students’ perceptions of their ability to conduct internet research. Sixty percent of the students reported their ability to conduct computer research to be excellent, while another 33% rated themselves as good. Reading In a June 2005 summer workshop, a baseline reading examination, the Gray Oral Reading Test (GORT), was administered to students participating in the CAD program. The test measured both reading fluency (rate and accuracy), as well as reading comprehension. Each CAD student’s reading assessment lasted approximately 30 minutes in a one-on-one session with a trained GORT test administrator. In May of 2006, a follow up reading examination was conducted. Form A of the GORT test was used in both pre-and post-test conditions, with a chance of error of .05. 8 th Graders: Reading Fluency Level 2005 Baseline Data 29% 0%0% 29% 0% 14%14%14% 0% 00.10.20.30.40.51st Gr.2nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr.8th Gr.> 8th Gr. 8 8% 0%0% 8% 33% 8% 16% 8% 16% 00.10.20.30.41st Gr.2nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr.8th Gr.> 8th Gr. th Graders: Reading Fluency Level 2006 Data 8 th Graders: Reading Comprehension Level 2005 Baseline Data 14% 0% 29% 0%0% 29% 14% 29% 0% 00.10.20.30.41st Gr.2nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr.8th Gr.> 8th Gr. 8 th Graders: Reading Comprehension Level 2006 Data 0% 8% 0%0% 33% 16% 8% 0% 33% 00.10.20.30.40.50.61st Gr.2nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr.8th Gr.> 8th Gr. 7 th Graders: Reading Fluency Level 2005 Baseline Data 19% 13%13% 19% 13% 0%0% 00.10.20.32nd Gr3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr. > 7th Gr. 7 th Graders: Reading Fluency Level 2006 Data 0%0% 10% 20%20%20%20% 10% 00.10.20.31st Gr.2nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr.> 7th Gr. 7 th Graders: Reading Comprehension Level 2005 Baseline 13%13%13% 7% 33% 13% 7% 00.10.20.30.42nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr.> 7th Gr. Reading Comprehension Level -7th graders 2006 Data 0%0%0% 20%20% 0%0% 60% 00.10.20.30.40.50.60.71st Gr.2nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.7th Gr.> 7th Gr. 6 th Graders: Reading Fluency 2005 Baseline Data 10% 15% 25% 15% 30% 5% 00.10.20.30.42nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.> 6th Gr. 6 th Graders: Reading Fluency 2006 Data 0% 11% 22% 45% 0% 22% 00.10.20.30.40.52nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.>6th Gr. 6 th Graders: Reading Comprehension 2005 Baseline Data 10% 20%15%15% 25% 15% 00.10.20.30.42nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.> 6th Gr. 6 th Graders: Reading Comprehension 2006 Data 0% 11% 0% 45% 0% 44% 00.10.20.30.40.50.62nd Gr.3rd Gr.4th Gr.5th Gr.6th Gr.> 6th Gr. Oral Communication Skills 10 The CAD students experienced remarkable success at the City Debate Championships, a tournament of over 100 students from Atlanta. Last year, the students won no awards at the city championships. This year, they placed as the 4th, 6th, 8th, and th teams in the junior varsity division. In the individual speaking award category, they placed 7th, 8th, 10th 14th, 19th and 20th. Notable amongst these achievements were the 7th and 8th grade speaker awards that went to two sixth graders who were competing in junior varsity (a division normally populated by 7th and 8th graders) due to their high level of previous success in the novice division. In the novice division of the City Debate Championships, the CAD students also excelled. They placed as the 6th, 7th and 10th teams, and won the 4th, 9th, 10th, 12th, 13th and 19th individual speaking awards. Most notable, however, was the success of the CAD kids at the State Championships. One of the CAD students won the 12th place team award for junior varsity debate and another won the fifth place speaker award for novice debate. Instructional Effectiveness CAD students at both Harper Archer and Benjamin Carson were asked to evaluate the instructional portion of their after school program. Sixty-eight percent of the students rated the instruction as excellent and another 21% rated it as good. Six percent were neutral and six percent evaluated it as fair. None of the students rated the program as poor. The students were also asked to evaluate the chances they would recommend the CAD program to their fellow students, friends or family members. Seventy-five percent of the students reported that they were highly likely and another twenty-three percent said they were likely to do so. Only two percent were undecided. No student reported that they were unlikely or highly unlikely to do so.