Algebra 2 is amorphous and has multiple heads

and defies reduction into compact and self-contained little parts. I am still trying to do just that, however. This continues the discussion about applying Dan Meyer's assessment system for Algebra 2, and anyone not deeply interested in this narrow topic may as well go on to the next item in their Reader.

Some time in June, Glenn Waddel wrote:

I have a rough draft of my skills checklist done right now. I am not sure I am going to post it yet. I am not happy with it. I think I am stuck in the “do I have to assess everything?” mode.

More than a month later, that's where I am still... and since time is running out I'm going to post the incomplete work in case that helps accelerate the process.

Glenn has meanwhile posted a carefully worked out list, and written about the tension between assessing specifically enough without introducing an intimidating number of concepts. Reading his list reminded me that the differences between the various versions of Algebra 2 around are significant, and that our final lists will have to be different to accommodate our respective course specifications and student groups.

In particular, my Intermediate Algebra course leaves conics and discrete math for the Trig/Precal course, which uses the same textbook and picks up where my course leaves off. I do not need to include assessments on these topics, then. On the other hand I must make sure that graphical features of quadratics, polynomials and exponentials are covered carefully, as this will not be repeated in Precal, and this increases the number of skills for these topics beyond what may be needed in Glenn's course. Also, Intermediate Algebra is for the students who do not make it to Honors Algebra 2, and so I need to include a lot of Algebra review. Dan Greene's version of Algebra 2 is similar to mine in the topics it covers, but students arrive directly from Algebra 1 without Geometry between, and so may need somewhat less review. I'm guessing that Sam Shah's course is for relatively advanced students. However, while our lists will need to be different in order to take these things into account, comparing notes could still be very useful.

Dan Greene and I met a few weeks ago, and made some progress on breaking down the chapter on Exponential and Logarithmic Functions, a unit where I am replacing all of my concept test items from last year. So far, the unit on Numbers and Functions has been the most demanding unit, I think. There are so many big, abstract and quite unfamiliar ideas there. On the one hand, the process of breaking down this chapter into parts that can be practiced separately may therefore be all the more necessary in order to make it accessible to students. On the other hand, much of the point is for students to recognize an abstract idea, such as the transformation of a function, across pretty different contexts, and this is just hard to assess in a piecemeal way. Or so it seems to me.

Finding a convenient format or platform for technical discussion of how to slice a topic into discrete skills and concepts is a challenge of its own, though. A series of blog posts, one for each chapter, seems both clunky and overly time-consuming (and school starts in just over a week over here). Instead, I've stored my work-in-progress on this Google Site, and if you have time and inclination to think some about what are the essential things to test for each topic or anything else, suggestions would be much appreciated.

Coffee and math ed readings

This summer I've met with a doctoral student of mathematics education a couple of times. Her area of interest is mathematical learning disabilities (MLD). Last time we met in a coffee shop to discuss an article by Geary et. al.¹ on students' placing of numbers on a number line, a topic I've been fascinated by for some time.

As it turns out, number lines constitute an active area of study in cognitive psychology and neuroscience, of theoretical interest

because magnitude representations, including those that support the number line, may be based on a potentially inherent number-magnitude system that is supported by specific areas in the parietal cortices ... (p. 279)

Geary's article cites earlier work by Siegler and Opfer² which suggests that young children use a more or less logarithmic scale when placing numbers. Children tend to perceive the difference between 1 and 2 as being greater than the distance between 89 and 90 in a semi-systematic way (p. 279), so that most numbers get clustered to the left hand side of the number line. This tendency is thought to reflect the postulated "inherent number-magnitude system."

Geary et. al. compared first and second graders' placements of numbers on a blank number line. They found some evidence that mathematically learning disabled students' placements not only failed to conform to the linear pattern at a rate comparable to that of their peers. In addition, their pre-instructional number placements also looked less like the logarithmic placement of non-disabled children:

Even when they made placements consistent with the use of the natural number-magnitude system, the placements of children with [mathematical learning disabilities] and their [low achieving] peers were less precise than those of the [typically achieving] children in first grade, that is, before much if any formal instruction on the number line. The implication is that children with MLD and LA children may begin school with a less precise underlying system of natural-number magnitude reprsentation. (p. 293)

Geary et. al. report correlations between performance on the number line tests with a battery of other cognitive tests. Unfortunately I know neither enough statistics nor enough cognitive psychology to extract terribly much information from these parts. Of rather more immediate interest to me as a teacher is, in any case, the question of how to go about eliciting the kind of cognitive change that's needed here. It certainly is not the case that all kids have the linear scale all figured out by the end of second grade - many of my 9th and 10th graders last year had not. The good news is that for this important topic, instruction tends to work. Cognitive Daily reports on more recent work by Siegler and Opfer showing that second graders responded quickly to some targeted feedback on their number placements, and that

once the linear form is learned, the transformation is quick, and permanent.

In other news, this thing of chatting about research in MLD over a morning coffee has been immensely enjoyable. The readings are demanding enough that I'd be much less likely to work through them if I were studying alone, but I'm awfully glad to be learning some of this.

I'm wondering how much interest there would be for some kind of regular math teacher/ math ed researcher meetups, such as a discussion of a predetermined article over coffee on Saturday mornings. Many new math teachers already have ed classes scheduled at that time, though, and older math teachers typically have family to take care of during weekend mornings, so how many would remain? And would there really be many researchers interested in talking with teachers? Still, given the curious absence of contact between researchers working on math education and math instructors working in schools, even in cases where their buildings are in the same geographical area, it would seem that some thinking should be done on how to afford more "vertical alignment" in Dina Strasser's sense of the term.

1. Geary, D. C., Hoard, M. K., Nugent, L. and Byrd-Craven, J. (2008). Development of Number Line Representations in Children with Mathematical Learning Disability. Developmental Neuropsychology 33:3, 277 - 299

2. Siegler, R. S. and Opfer, J. (2003). THe development of numerical estimation: Evidence for multiple representations of numerical quantity. Psychological Science, 14, 237 - 243.

Have you used Algebra tiles?

I introduced integer tiles to my Algebra classes early last fall, and then quickly gave it up. Several students balked at using such middle school measures for studying math, and my arguments that being able to represent math statements in many different ways, including with concrete objects, failed to persuade. In a class of insecure freshmen still figuring out their relative positions in the class, and in some cases still stinging from having been placed in Algebra rather than in Geometry after the placement test, using materials perceived as childish just wasn't socially acceptable.

I quietly dropped the project, only including a problem on modeling integer subtraction as an extra credit problem on a unit test some time later. Not one student got it right. Later in the year, when a number of students continued to demonstrate confusion about combining signed integers and combining like terms, I sometimes wished I'd stuck with the manipulatives a little longer.

Now I'm trying to make up my mind about whether - and, if so, how - to use tiles in my Algebra classes in the fall. Apart from the probable social issues to deal with, I'm wondering about the efficacy of Algebra tiles. A point I've picked up in passing while reading this summer (I'm sorry I can't recall where!) is that the same students who are likely to have much trouble with elementary Algebra are also likely to have difficulty picking up how to manipulate Algebra tiles.

There is, after all, no magic involved. The rules for representing addition and subtraction of integers with bi-colored tiles are not self-evident or trivial. Even for me, the representation of subtraction problems by adding the necessary number of "zero pairs" came as a bit of a surprise. And while I then found the very idea to be very cool and exciting, that is more than I can take for granted that my students will, even if they aren't unable or unwilling to master the rules of the game.

So, what are your experiences with Algebra tiles? How to you go about changing the image of tiles as being all too elementary? And assuming that you have gotten your young charges to take the tiles seriously, how much do you feel that the students learn this way that they do not learn just as well by simply reiterating the formalism of signed numbers and like terms?

Applying Dan's assessment system, Part II - scoring

Note: A discussion of more general lessons learned while applying this assessment system is posted here. This entry is a dry, technical discussion of scoring and grade calculations, of interest only to teachers thinking of applying this system themselves.

Dan Meyer assesses students at least twice on every test item, scoring out of 4 each time. At the second round of assessment, he alters the possible points from 4 to 5. If a student scores a 4 on both rounds of assessment, she nets a 5. Otherwise, her highest score applies. Dan makes the second round of assessments a little harder than the first, so that a second 4 indicates greater skill than the first 4.

Altering the possible points from 4 to 5 entails that students who do not actually improve their performance from one assessment to the next automatically see their grade drop. For example, if a student scores a 3 on the first assessment, and then another 3 on the second, more demanding assessment of the same skill, the grade on that particular concept drops from a 3/4=75% to a 3/5=60% - from a solid C to a D-. This caused problems that almost made me abandon the system early last fall. For one, students got upset when their grade dropped without their knowledge having changed. Secondly, having grades drop after a progress report has been issued is not actually legal - or so I was told by my Department Head. An evening shortly after the first progress reports had been printed found us manually going through all the scores in the gradebook, altering the scores so that the grades would be the same as before, for example by changing 3/5 to 3.75/5, since that is equal to 3/4=75%.

Another problem with this system was that a scale from 0 to 4 seemed fairly coarse grained. Students who made a mistake significant enough not to merit a top score on the first assessment would be marked down by 25 percentage points, and if they did not improve markedly by the second assessment they would net a D-. Improvement from this D- would be possible only if they subsequently scored a perfect score. I first thought that the large number of skills and the repetition of assessments would lead to an adequate continuity of the total grading scale, that students might average a C by scoring perfectly on some skills and poorly on others. However, some students seemed, even when working hard, to be unable to ever score a 4. They'd always make some or other significant mistake, but not enough to make a D- seem appropriate. Now I am sure that in the mutual adjustment of quiz difficulty and scoring practice there is some wiggle room for making this work in a fair way, and I assume Dan Meyer has figured out a balance here. However, I ended up changing my grading scale.

Solving these problems proved pretty difficult without losing important features of the original system, however, and I found no perfect solution. I wanted my score assignment to do what Dan's did, in particular, to make it necessary for students to take every assessment twice, in order to ensure stability and retention of knowledge. Dan's practice of increasing the possible points does just that - students can not just be satisfied with their 3/4=75% and decide not to attempt the second assessment of the same skill. In the end I decided not to report students' scores online until they had had both assessments. I made the two assessments of equivalent difficulty (which simplified things for me) and then grades were assigned based on students' best two scores according to the following table:

In summary: For scores of 3 or lower, the higher score applies. If both scores are above 3, the grade is the average of the two. If one score is above a 3 and the other below, the grade is the average of 3 and the higher grade. With this score assignment, students still had an incentive to demonstrate perfect mastery twice, in order to net a grade of a 100.

A disadvantage of this system is it's clunkiness compared to Dan's simpler system. Much of the appeal of this whole approach to grading was its transparency to students, the clarity it could afford them about what to focus on. Some of this is lost with this conversion table. Also, since the best two scores count, the system appears to have somewhat more inertia; poor scores don't go away as fast as they seem to in the original system, where the better score always counts. This slower improvement is more appearance than reality, since two 4's are necessary to achieve a 100 in Dan's system too, but appearance matters in this context. The main disadvantage, however, was switching to this different scale after the first progress report, which caused some confusion and, I think, some loss of buy-in from students. They seemed a little less enthusiastic about completing their tracking sheets after that.

As an alternative, I experimented a little with just entering both of the best two scores into PowerGrade this spring, labeling the entries "Skill 14A" and "Skill 14B," for example, and assigning half weight to each. I am undecided about whether I will do this in the fall or just enter the composite grade. It is of paramount importance that the students understand the relation between the scores on the papers they get back and the scores on their grade printout, and this system would help in that regard, but it would make for a large number of gradebook entries, which means more messiness.

Finally, a note on the scoring of any quiz item: In some cases it made sense to assign a point value to different components of the test item, and sometimes I wrote the test items to make this possible. Other times, I evaluated the complete response to the test item as a whole, and assigned scores as follows:


Frankly, for some skills that did not lend themselves well to decomposition into parts with point values for each, I'd score based on my mental image of what a D-, a B- and an A would look like. If grades are supposed to be derived from scores rather than the other way around, that introduces some circularity that one might argue about, but I don't care. I think grades as descriptors of performance levels rather than as translations of some numerical score make more sense anyway. But that is another story that would make for a separate discussion.

And since this scoring business turned out so much trickier than I'd anticipated, well-thought out suggestions for making it clearer and fairer would be appreciated.

Applying Dan's assessment system, Part I

Dan Meyer breaks his courses into some 35 discrete skills and concepts, keeps separate records on students' performance on each skill, and keeps retesting students and counting their highest scores. The following two entries are some notes on things I learned while applying an adapted version of his system to my Algebra and Intermediate Algebra this year. The second entry is a dryly technical discussion of scoring.

In accordance with my Department Head's recommendation, I did not entirely replace traditional comprehensive tests with this more piecemeal system. For Algebra 1, these concept quizzes were weighted at 40% of students' grades while comprehensive tests made up the remaining 30% of the assessment grade. For Intermediate Algebra I weighted the two types of assessments at 35% each. My experiences were that...

...this system worked significantly better for Algebra 1 than for Intermediate Algebra.

In Algebra 1, I felt that pretty much everything the students really needed to know was covered by the concept quizzes – I might as well not have done chapter tests at all. For Intermediate Algebra, however the skills tended to get cumbersomely complex or impossibly many, and the supplemental chapter tests were necessary and useful.

One reason is that Intermediate Algebra, which is essentially the first 70-80% of Algebra 2, covers much more¹. Another reason is that synthesis and solution of multi-step problems are inherent, irreducible goals of Algebra 2, and these skills need to be assessed, too.²

... for diagnosing and remedying deficiencies in basic skills, this system was beautiful.

At some point early in the semester I realized that a number of incoming Algebra 1 students did not know the concept of place value and could place neither decimal numbers nor fractions on a number line. Writing an assessment on placing decimals on the number line made it possible to separate out who was having trouble with this, and to know when a critical mass of students had caught up in this area. As a tool for probing missing background skills and for placing these skills clearly and definitely on the agenda this was powerful.

... writing effective assessment items was harder than I thought.

When an assessment may potentially be repeated two, three, even five or six times, what it measures had better be really important, and the assessment had better actually capture the intended skill. It is not as easy as it may sound to decide which elements of the course really are that important; which are the parts on which other understanding hinges. My list of concepts to be assessed always tended to get too long, and trimming down to the real essentials was a constant challenge. As for designing valid measurements of students' skills, I guess only experience makes it possible to figure out what kinds of problems will really show that they know what they need to know, what kinds of problems plough just deep enough without getting too involved, what kinds of misunderstandings are typical and must be caught in order to make necessary remediation possible.³

... assessments are not enough. Improvement is not automatic.

That's obvious, of course. How silly to think otherwise. Frankly, part of what I found attractive about this assessment system was the idea that with goals broken down into such small, discrete pieces, students would become empowered and motivated and take the initiative to learn what they needed to make the grade. That was actually to a significant extent the case. Tutoring hours were far more efficient due to the existence of these data, and students knew what to do to "raise their grade." However, a lot of students continued to score poorly, repeating the same mistakes, after three, four, five rounds of assessment on the same topic. Some would come during tutoring hours to retake a quiz and still make exactly the same mistakes... For weaker students especially, then, it is important to remember that the assessment data are tools for me to actually use. There is no automaticity in the translation of this very specific feedback into actual understanding.

... the transparency of the system means bad things are out there for everyone to see.

That's what we want, don't we? The direct and honest reporting involved was a major appeal of this system. However, it takes some foresight for this not to lead to discouragement. While it is pretty common practice among math teachers, any teachers, to rescale test scores so that the class average turns out okay, this could not be done in any simple way with these conceptwise assessments. The only way to improve class grades was by reteaching the material and testing again. This involved a time delay during which the grades, which were published in an online gradebook, could be quite low. This was especially true during the first month or two of school, when the grades were constituted by relatively few entries, and - well - the first months of school may not be the time you want parents to worry about what you're doing when you're a new employee. In the early stages I ended up scaling chapter tests a good deal in order to compensate for some low concept quiz scores and make the overall grades acceptable. With time, a combination of rewriting certain concept quizzes that were needlessly tricky and teaching some topics better made this less necessary. ⁴

In conclusion, I am definitely keeping this system for Algebra 1, probably increasing the weighting of these assessments and reducing the number and importance of comprehensive tests. For Intermediate Algebra I am keeping chapter tests, and writing a new set of piecemeal assessments to cover just the basics, so that I can have the hard data on who is really lost, but without even trying to force these assessments to cover the entire curriculum. I'll need to make sure that the first skills are very well taught and mastered before the first round of assessments: thinking a little strategically to make sure the early results are good increases buy-in, and student ownership is after all much of the point here.


Notes

1 By way of example, a comparison of the content of the chapters on exponents in the two courses: To assess mastery of this chapter for Algebra 1, I needed to check that students knew the definition of a natural power as repeated multiplication, that they could apply the power rules to simplify expressions, that they could deal with negative and zero powers, that they could complete a table of values of a simple exponential function such as 2^x and plot the points to sketch a simple exponential graph. For the chapter on exponential and logarithmic functions for Intermediate Algebra, however, I needed to check whether students could do all of the above, plus convert between exponential and logarithmic form, apply the properties of logarithms, solve exponential and logarithmic equations by applying one-to-one properties, solve such equations by applying inverse properties, apply the change-of-base formula, apply the compound interest formula, identify transformations of the exponential function, understand that exponential and logarithmic functions are inverses of each other, plus a few other things that I just skipped. The number of chapters to be covered is pretty much the same for both courses, but the number of concepts and skills? Different stories. Writing broader concept tests for more advanced courses is a possibility, but the advantages of this piecewise assessment system over the usual comprehensive test system is quickly lost this way.

2 For an example of how some core skills of Intermediate Algebra are by nature multi-step and integrative, consider the case of solving a third degree polynomial equation by first finding a root by graphing, then dividing by the corresponding linear factor, then applying the quadratic formula to find the remaining roots. This task is too complex for a concept wise assessment to be very useful. I had separate assessments on 1) identifying factors given the graph of a polynomial, on 2) polynomial division and rewriting a polynomial using the results of the division process, on 3) stating and applying the factor theorem, and 4) applying the quadratic formula. I still wanted to check whether the students could put it all together.

3 As for the assessment being valid, actually capturing the important skill, here's an example of a failed attempt: I wrote one concept quiz about identifying the difference between an equation and an expression, about distinguishing the cases where you solve for a variable from the case where you can only simplify – but success on this assessment did not mean an end to confusing these two cases. Does that mean that the assessment was poorly written, or rather that this distinction just doesn't lend itself to being assessed once and for all in a little concept quiz? Is understanding equivalence, and distinguishing equations as statements that are true or false from expressions that just are, too abstract ideas to be covered this way? I don't know, but my impression is that the quiz did little to eradicate the common mistake of treating expressions as if they were equations, for example by adding or subtracting new terms in order to simplify.

4 This is at a private school, where determining the required level of mastery of each standard is to a larger extent up to the teacher, since no state testing is involved in defining the bar.

I love inverses :)

It's sheer nerd joy, finding the inverse of an exponential or a quadratic function; confirming that entering the output of a relation into its inverse really does return the original input; finding that the graphs of a relation and its inverse really are reflections of each other in the line y = x. I think that requiring all Intermediate Algebra students to do this would be demanding a bit too much, so I offer some worksheets on inverses as extra credit opportunities, and under such conditions many students are more than willing to try. With appropriate enthusiasm, one student highlighted parts of this graph of a quadratic and its inverse in red pencil before turning it in:

I find that this work on inverses deepens students' understanding of the meaning of solving equations, and helps them appreciate the idea that the operations needed to isolate the variable are operations that undo operations previously performed on it. The students need a lot of help on the first examples, and then are quite pleased with themselves when they find they can do this initially hard bit of algebra on their own.

Following Dan Greene, I emphasize the three representations of a relation (Equation! Table! Graph!) again and again, and it is helpful to reiterate these alternative representations when working with inverses. We can find the inverse by interchanging x and y in the equation, by interchanging the values of x and y in the table, or by interchanging the coordinates of each point on the graph of a relation. Talking this way in the context of finding inverses in turn reinforces the idea of equations, tables and graphs as representations of the same information - another nice thing about working with inverses.

Worksheets:
- Exponential and logarithmic functions as inverses, Word and PDF
- Quadratics and square root relations as inverses, Word and PDF

Maybe manipulatives aren't the answer?

This NYT article suggests that

... it might be better to let the apples, oranges and locomotives stay in the real world and, in the classroom, to focus on abstract equations ... Dr. Kaminski and her colleagues Vladimir M. Sloutsky and Andrew F. Heckler ... performed a randomized, controlled experiment. ... Though the experiment tested college students, the researchers suggested that their findings might also be true for math education in elementary through high school ...

In the experiment, the college students learned a simple but unfamiliar mathematical system, essentially a set of rules. Some learned the system through purely abstract symbols, and others learned it through concrete examples like combining liquids in measuring cups and tennis balls in a container.

Then the students were tested on a different situation — what they were told was a children’s game — that used the same math. ... The students who learned the math abstractly did well with figuring out the rules of the game. Those who had learned through examples using measuring cups or tennis balls performed little better than might be expected if they were simply guessing. Students who were presented the abstract symbols after the concrete examples did better than those who learned only through cups or balls, but not as well as those who learned only the abstract symbols.

The problem with the real-world examples, Dr. Kaminski said, was that they obscured the underlying math, and students were not able to transfer their knowledge to new problems.

“They tend to remember the superficial, the two trains passing in the night,” Dr. Kaminski said. “It’s really a problem of our attention getting pulled to superficial information.”