Friday, June 13, 2008

Applying Dan's assessment system, Part I

Dan Meyer breaks his courses into some 35 discrete skills and concepts, keeps separate records on students' performance on each skill, and keeps retesting students and counting their highest scores. The following two entries are some notes on things I learned while applying an adapted version of his system to my Algebra and Intermediate Algebra this year. The second entry is a dryly technical discussion of scoring.

In accordance with my Department Head's recommendation, I did not entirely replace traditional comprehensive tests with this more piecemeal system. For Algebra 1, these concept quizzes were weighted at 40% of students' grades while comprehensive tests made up the remaining 30% of the assessment grade. For Intermediate Algebra I weighted the two types of assessments at 35% each. My experiences were that...

...this system worked significantly better for Algebra 1 than for Intermediate Algebra.

In Algebra 1, I felt that pretty much everything the students really needed to know was covered by the concept quizzes – I might as well not have done chapter tests at all. For Intermediate Algebra, however the skills tended to get cumbersomely complex or impossibly many, and the supplemental chapter tests were necessary and useful.

One reason is that Intermediate Algebra, which is essentially the first 70-80% of Algebra 2, covers much more1. Another reason is that synthesis and solution of multi-step problems are inherent, irreducible goals of Algebra 2, and these skills need to be assessed, too.2

... for diagnosing and remedying deficiencies in basic skills, this system was beautiful.

At some point early in the semester I realized that a number of incoming Algebra 1 students did not know the concept of place value and could place neither decimal numbers nor fractions on a number line. Writing an assessment on placing decimals on the number line made it possible to separate out who was having trouble with this, and to know when a critical mass of students had caught up in this area. As a tool for probing missing background skills and for placing these skills clearly and definitely on the agenda this was powerful.

... writing effective assessment items was harder than I thought.

When an assessment may potentially be repeated two, three, even five or six times, what it measures had better be really important, and the assessment had better actually capture the intended skill. It is not as easy as it may sound to decide which elements of the course really are that important; which are the parts on which other understanding hinges. My list of concepts to be assessed always tended to get too long, and trimming down to the real essentials was a constant challenge. As for designing valid measurements of students' skills, I guess only experience makes it possible to figure out what kinds of problems will really show that they know what they need to know, what kinds of problems plough just deep enough without getting too involved, what kinds of misunderstandings are typical and must be caught in order to make necessary remediation possible.3

... assessments are not enough. Improvement is not automatic.

That's obvious, of course. How silly to think otherwise. Frankly, part of what I found attractive about this assessment system was the idea that with goals broken down into such small, discrete pieces, students would become empowered and motivated and take the initiative to learn what they needed to make the grade. That was actually to a significant extent the case. Tutoring hours were far more efficient due to the existence of these data, and students knew what to do to "raise their grade." However, a lot of students continued to score poorly, repeating the same mistakes, after three, four, five rounds of assessment on the same topic. Some would come during tutoring hours to retake a quiz and still make exactly the same mistakes... For weaker students especially, then, it is important to remember that the assessment data are tools for me to actually use. There is no automaticity in the translation of this very specific feedback into actual understanding.

... the transparency of the system means bad things are out there for everyone to see.

That's what we want, don't we? The direct and honest reporting involved was a major appeal of this system. However, it takes some foresight for this not to lead to discouragement. While it is pretty common practice among math teachers, any teachers, to rescale test scores so that the class average turns out okay, this could not be done in any simple way with these conceptwise assessments. The only way to improve class grades was by reteaching the material and testing again. This involved a time delay during which the grades, which were published in an online gradebook, could be quite low. This was especially true during the first month or two of school, when the grades were constituted by relatively few entries, and - well - the first months of school may not be the time you want parents to worry about what you're doing when you're a new employee. In the early stages I ended up scaling chapter tests a good deal in order to compensate for some low concept quiz scores and make the overall grades acceptable. With time, a combination of rewriting certain concept quizzes that were needlessly tricky and teaching some topics better made this less necessary. 4

In conclusion, I am definitely keeping this system for Algebra 1, probably increasing the weighting of these assessments and reducing the number and importance of comprehensive tests. For Intermediate Algebra I am keeping chapter tests, and writing a new set of piecemeal assessments to cover just the basics, so that I can have the hard data on who is really lost, but without even trying to force these assessments to cover the entire curriculum. I'll need to make sure that the first skills are very well taught and mastered before the first round of assessments: thinking a little strategically to make sure the early results are good increases buy-in, and student ownership is after all much of the point here.


Notes

1 By way of example, a comparison of the content of the chapters on exponents in the two courses: To assess mastery of this chapter for Algebra 1, I needed to check that students knew the definition of a natural power as repeated multiplication, that they could apply the power rules to simplify expressions, that they could deal with negative and zero powers, that they could complete a table of values of a simple exponential function such as 2x and plot the points to sketch a simple exponential graph. For the chapter on exponential and logarithmic functions for Intermediate Algebra, however, I needed to check whether students could do all of the above, plus convert between exponential and logarithmic form, apply the properties of logarithms, solve exponential and logarithmic equations by applying one-to-one properties, solve such equations by applying inverse properties, apply the change-of-base formula, apply the compound interest formula, identify transformations of the exponential function, understand that exponential and logarithmic functions are inverses of each other, plus a few other things that I just skipped. The number of chapters to be covered is pretty much the same for both courses, but the number of concepts and skills? Different stories. Writing broader concept tests for more advanced courses is a possibility, but the advantages of this piecewise assessment system over the usual comprehensive test system is quickly lost this way.

2 For an example of how some core skills of Intermediate Algebra are by nature multi-step and integrative, consider the case of solving a third degree polynomial equation by first finding a root by graphing, then dividing by the corresponding linear factor, then applying the quadratic formula to find the remaining roots. This task is too complex for a concept wise assessment to be very useful. I had separate assessments on 1) identifying factors given the graph of a polynomial, on 2) polynomial division and rewriting a polynomial using the results of the division process, on 3) stating and applying the factor theorem, and 4) applying the quadratic formula. I still wanted to check whether the students could put it all together.

3 As for the assessment being valid, actually capturing the important skill, here's an example of a failed attempt: I wrote one concept quiz about identifying the difference between an equation and an expression, about distinguishing the cases where you solve for a variable from the case where you can only simplify – but success on this assessment did not mean an end to confusing these two cases. Does that mean that the assessment was poorly written, or rather that this distinction just doesn't lend itself to being assessed once and for all in a little concept quiz? Is understanding equivalence, and distinguishing equations as statements that are true or false from expressions that just are, too abstract ideas to be covered this way? I don't know, but my impression is that the quiz did little to eradicate the common mistake of treating expressions as if they were equations, for example by adding or subtracting new terms in order to simplify.

4 This is at a private school, where determining the required level of mastery of each standard is to a larger extent up to the teacher, since no state testing is involved in defining the bar.

Saturday, May 24, 2008

I love inverses :)

It's sheer nerd joy, finding the inverse of an exponential or a quadratic function; confirming that entering the output of a relation into its inverse really does return the original input; finding that the graphs of a relation and its inverse really are reflections of each other in the line y = x. I think that requiring all Intermediate Algebra students to do this would be demanding a bit too much, so I offer some worksheets on inverses as extra credit opportunities, and under such conditions many students are more than willing to try. With appropriate enthusiasm, one student highlighted parts of this graph of a quadratic and its inverse in red pencil before turning it in:

I find that this work on inverses deepens students' understanding of the meaning of solving equations, and helps them appreciate the idea that the operations needed to isolate the variable are operations that undo operations previously performed on it. The students need a lot of help on the first examples, and then are quite pleased with themselves when they find they can do this initially hard bit of algebra on their own.

Following Dan Greene, I emphasize the three representations of a relation (Equation! Table! Graph!) again and again, and it is helpful to reiterate these alternative representations when working with inverses. We can find the inverse by interchanging x and y in the equation, by interchanging the values of x and y in the table, or by interchanging the coordinates of each point on the graph of a relation. Talking this way in the context of finding inverses in turn reinforces the idea of equations, tables and graphs as representations of the same information - another nice thing about working with inverses.

Worksheets:
- Exponential and logarithmic functions as inverses, Word and PDF
- Quadratics and square root relations as inverses, Word and PDF

Friday, April 25, 2008

Maybe manipulatives aren't the answer?

This NYT article suggests that
... it might be better to let the apples, oranges and locomotives stay in the real world and, in the classroom, to focus on abstract equations ... Dr. Kaminski and her colleagues Vladimir M. Sloutsky and Andrew F. Heckler ... performed a randomized, controlled experiment. ... Though the experiment tested college students, the researchers suggested that their findings might also be true for math education in elementary through high school ...

In the experiment, the college students learned a simple but unfamiliar mathematical system, essentially a set of rules. Some learned the system through purely abstract symbols, and others learned it through concrete examples like combining liquids in measuring cups and tennis balls in a container.

Then the students were tested on a different situation — what they were told was a children’s game — that used the same math. ... The students who learned the math abstractly did well with figuring out the rules of the game. Those who had learned through examples using measuring cups or tennis balls performed little better than might be expected if they were simply guessing. Students who were presented the abstract symbols after the concrete examples did better than those who learned only through cups or balls, but not as well as those who learned only the abstract symbols.

The problem with the real-world examples, Dr. Kaminski said, was that they obscured the underlying math, and students were not able to transfer their knowledge to new problems.

“They tend to remember the superficial, the two trains passing in the night,” Dr. Kaminski said. “It’s really a problem of our attention getting pulled to superficial information.”

Tuesday, March 25, 2008

Completing the Square

Toward the goal of sharing more of the humdrum, everyday business of teaching math, here are a few notes about how we do completing the square in my classes. I have no cool tricks or creative activies for this, and I would very much like to see more of yours. Nevertheless, completing the square is, inexplicably, a favorite topic of mine.

The first time I taught it I relied somewhat on the formulaic addition and subtraction of the square of half the middle coefficient, but that left the class utterly confused and frustrated. Now I rely less on this rule and more on pattern recognition and intuition. This appears to make for better retention, though the students do have trouble applying it to numerically messy cases, where a formulaic approach - if ever mastered, that is - would be safer.

We start by reviewing how to square a binomial, because while we have of course worked on multiplying binomials and using standard factoring patterns earlier, the error of squaring a binomial by squaring each term is remarkably resistant to instruction. We always need to refresh that by writing out the factors and multiplying, carefully, term by term. Indeed, any time that a review of completing the square is called for later, squaring binomials from scratch is the point I will return to, and invariably it will turn out that many students have forgotten what the squared binomial looks like. After we've done the multiplication from first principles for a handful of examples, I point out the pattern in the middle and last terms of the product and ask the students to pick up speed, which they do. I write up a few binomials where the second term is a fraction and remind the students that one advantage of fraction form over decimal form is that fractions are really easy to square.

I'll call on individual students or have the class shout out answers, and will alternate between having students suggesting problems and solving them ("J., will you give us a binomial?" "S., will you square that for us?") and I have repeatedly been surprised at how engaged the students tend to become during this exchange, since the topic, after all, isn't that inherently exciting, and we aren't doing anything particularly nifty. Part of the reason may be that it is easier than for many other topics to sense just where the students are and to tailor the next example so that it matches their readiness.

When I notice that the class is beginning to get that "now what...?" feeling, we reverse the process: I write up a perfect square trinomial and have students factor it. We keep doing this until the students again reach the point where this is too easy, and then start looking at cases where only the quadratic and linear terms are given and the students need to figure out what numbers would fit in the blank spaces in a form such as this one: Later, writing this form on the board will be sufficient to cue a large fraction of the students to what they are trying to do.

We move on to rewriting simple quadratics (where a=1 and b is an integer) in vertex form. Later I will show them that adding and subtracting the square of half of the coefficient of the linear term will give us just what we want, but at this stage we simply identify the squared binomial, multiply it out, and compare this with the original quadratic to see what we need to add or subtract. For example, to write in vertex form we will recognize that (x-4)^2 is the square term, and since expanding this gives a constant term of 16 we'll need to subtract 13 in order to ensure that we have the same quadratic that we started with: This approach seems to stick fairly well in students' memories. Many students who do not correctly add and subtract the half of the middle coefficient later (they'll insert an x in there, or halve it incorrectly, or something) will be able to rewrite simple quadratics in vertex form, and I can see from their scratch work in the margin that they're just comparing the expanded square with the original quadratic. I'm pleased with that, because the equivalence of the quadratic in its two forms is one of the big ideas I want them to take away, and the fact that we aren't dealing with different quadratics even though they do look different isn't nearly as self-evident to the young ones as it is to us.

So, that was not terribly exciting or innovative, I concede. But how do you teach completing the square?

Saturday, March 22, 2008

Joke

- What's the difference between an outgoing Physicist and one who is not?
- The outgoing Physicist looks at your shoes while talking to you.

Oof.

Thursday, March 6, 2008

Pi Day

What do you all do for Pi Day? In particular, what might be worth the while in an Algebra 1 class?

Would anyone with a high-traffic blog mind posting some version of this question, in order to cast a wider net? That would be nice of you...

Monday, February 18, 2008

Unequal Methods

Colleagues - I could use some advice. I just graded the Algebra 1 tests on Inequalities and Absolute Value, and they were quite awful. While some of that is due to the distractions of Spirit Week and me being sick a few days, there's more to it, and any reports on successful approaches to teaching inequalities in general and absolute value inequalities in particular would be most appreciated.

A first hurdle is to help students actually understand the number line graphs that they draw of solutions to inequalities. Many use middle school mnemonics about arrows pointing in the same direction as the inequality sign, and draw their graphs from these rules (and go wrong when the variable appears on the right hand sign of the inequality, of course). We've worked on listing a few actual numbers that are part of the solution, plotting these, and then drawing the graph afterward. That helps a good deal after a while.

The next hurdle is to understand the difference between AND and OR inequalities. The most effective approach so far has been a combination of the above mentioned insistence on a list of actual, specific numbers that satisfy the conditions, and lots of problem quartets like the following:
x > 2 and x < 5
x > 2 or x < 5
x > 5 and x < 2
x > 5 or x < 2
This way, we always get one inequality with no solution, one satisfied by all real numbers, and a couple plain vanilla and- and or-inequalities for the same pair of numbers. It's not magic, but it does seem to help to vary one thing at a time.

Then enter the absolute value inequalities, and what a mess they are. There are so many different ways of solving them, and of talking about them, and I've made the mistake of covering several instead of sticking to one geometric approach and one algebraic approach. Now students are, quite predictably, using messy combinations of these.

Geometrically, the absolute value of x - 2 can be understood as the distance of x - 2 from zero, or the distance of x from 2. Last semester, with the Intermediate Algebra class, I relied on the former and had the students set up inequalities such as | x - 2 | > 3 by drawing a number line, placing "x - 2" more than three units away from zero on either side of zero, reading the resulting inequalities ( "x - 2 > 3" and "x - 2 < -3" ) from their sketch, and solving from there. It didn't really stick. I am not sure whether that was due to inadequate repetition or due to this approach being conceptually confusing.

Anyhow, with the Algebra 1 group this semester, I instead belabored the geometric interpretation of | x - 2 | as the distance between x and 2. I taped a large number line under the blackboard and we checked this definition by walking back and forth: -1 is three steps from 2, and sure enough, | -1 - 2 | = 3, and so forth. In order to solve inequalities such as | x - 2 | > 3 I had two students walk three steps from 2 on the number line in either direction, and we talked about what numbers were more than 3 units away from 2. This was difficult for many students (and not only the small group that always tunes out when I use any concrete representations because they think that's too middle school). My hunch is that there's some relation between this confusion and the difficulties Mr. K's students had with the meaning of "more."* Once students did pick up the idea it seemed to stick, but many never really got it. Maybe it's harder to ask when you're confused about what the walking up and down the number line is supposed to mean than when the material is more evidently academic.

I had first hoped to rely on this geometric approach to help students remember the direction of the inequality sign for the two linear inequalities in terms of which they will rewrite their absolute value inequalities, but gave up on that and introduced the approach that follows naturally from the algebraic definition of absolute value. If | x - 2 | > 3 then either x - 2 is greater than 3 or else the opposite of x - 2 is greater than 3.

Now I wish I'd used the geometric approach only for predicting and interpreting answers and stuck religiously to the algebraic approach to setting up the inequalities - because now students are using strange combinations of the two, such as x + 2 > -3 or -x + 2 > -3. In other words, complete confusion, with neither a clear concept nor a clear method to rely on. That's pretty discouraging even before thinking about the many students who did not even acknowledge the fact that there are two solutions to absolute value problems, that a distance can be in either of two directions... So, math teachers, what do I do now?

*We tend to assume too much about students' immediate grasp of the very idea of comparing quantities, let alone the isomorphism from this ranking of magnitudes to a spatial ordering along a line. Bob Moses, with his interest in pre-mathematical concepts that must be in place in order to succeed at Algebra, would presumably have a lot to say about this.