How the District Analysis Review Tool (DART) Enables Better Research

By Peter Ciurczak

March 30, 2017

The District Analysis Review Tool, or DART, facilitates a deeper understanding of student and teacher performance across every school and district within the state. The Massachusetts Department of Elementary and Secondary Education (DESE) makes DART data accessible online so that researchers, parents and other interested parties can more easily investigate things like: how many students attending Boston Latin are economically disadvantaged (363) or what percentage of African-American students at Boston Arts Academy scored proficient or higher on their English Language Arts Massachusetts Comprehensive Assessment System (MCAS) test (95%).

There is a learning curve with the online tool, however. DART provides lots of raw data all in one place, and its design assumes a pre-existing familiarity with educational measures, requiring newer users to look elsewhere for detailed definitions. For instance, DART presents “Student Growth Percentiles (SGP)” without any definition of what this means or why it’s important (SGP compares growth in a student’s MCAS scores to growth among peer students with similar characteristics).

DART Versions

Different DART datasets are also available for download, which presents an alternative means of getting at school and district information. The first of these looks solely at the performance of English Language Learners within Massachusetts schools. The second looks at Staffing and Finance. The third file – the Success After High School file– gives users a window into how students are performing in high school and persisting through college. The rest of this article provides a walkthrough of how one could use the SAS dataset to compare student outcomes across two schools.

How to use DART

When you first open the Success After High School file, you’re greeted with the “Home” tab – and two drop-down boxes in the top left. These allow you to select the schools you’re looking for. For our purposes, let’s say you’re a researcher interested in comparing how students at Burke High School and West Roxbury Academy perform on their English Language Arts (ELA) MCAS tests. The first thing you’re going to do is find “Boston – Burke High” in the “Select your school” dropdown. Then, click on the box below it and select “Boston – West Roxbury Academy.”

Once you’ve selected your two schools, you’ll notice the space below them – the “School Overview” section – has populated itself with schools that are most similar to your chosen school on a variety of metrics. Reading across these columns, you get a preliminary sense of the school’s enrollment as well as high school and post-secondary education outcomes. The school you’ve selected – in this case Burke High School – is highlighted in orange, while the highest performing school out of the comparable schools is highlighted in blue.

If you’re looking for more specific information about these schools, you can jump to it using the Table of Contents (as above, but only after you’ve selected the schools), or by clicking through the tabs that run along the bottom of the sheet. Each of the tabs within DART provides specific information about a school or district’s students and is briefly summarized in the table below.

While each of these tabs has something interesting to say about the students of Burke and W. Roxbury, let’s focus our efforts on the High School Performance tab. Here, we can see that on MCAS assessments, Burke scores higher than West Roxbury Academy on every measure of student proficiency across English Language Arts (ELA), Mathematics and the Science and Technology/Engineering subject areas (don’t know what MCAS is? Scroll to the right and select the “More about this data” link). In Mathematics, Science and Technology, these proficiency gaps are quite large. In English Language Arts however, the gap is significantly smaller (72 percent vs 70 percent in 2016), and the schools have alternated leading each other over the years.

But these comparisons do not control for differences among the two school populations. Fortunately, DART allows a more granular analysis of specific student populations. By clicking on the green “Select Subgroup” drop-down, you can run these same analyses through every student population the system tracks. When you do this for “Economically Disadvantaged” students (students who live in lower-income families), the story is reversed. Economically disadvantaged students at W. Roxbury actually perform higher than their peers at Burke High by about seven percentage points on English Language Arts. Additionally, W. Roxbury’s African-American students also score 1 percentage point higher than Burke’s African-American students.

These comparisons underscore the importance of disaggregating by student population when comparing outcomes by school. And the DART tool is a great resource for doing this.