Skip to content
assignmentAssessment and Measurement

Data-Based Decision Making

Continuing, changing, or ending an intervention is a decision — and it can be made by looking at the data.

Translated from the Portuguese original. Read the original (Português)

boltIn short

Every intervention sooner or later reaches a crossroads: keep going as is, adjust, or end it. Data-based decision making is the commitment to answer that question by looking at the person's measured performance — recorded continuously and displayed on a graph — instead of relying on the impression of the person teaching. It has three parts: a reliable measure of the behavior that actually matters; a criterion set in advance (where you want to get to, and by when); and a decision rule agreed on beforehand, which states what to do when the data fall below, above, or right at that criterion. This is not bureaucracy: in educational research, systematically tracking performance improves outcomes — and the improvement is largest precisely when the data become a graph and are read with explicit rules. In Behavior Analysis, this is not just good practice: it is an ethical requirement.

Technical definition

Data-based decision making is the systematic use of performance data — obtained through direct and repeated measurement of the target behavior, under comparable conditions — to decide whether to continue, modify, or end a teaching procedure or intervention. It is the practical form taken by two of the dimensions described by Baer, Wolf, and Risley (1968): the analytic dimension (demonstrating that the change is due to what was done) and the effective dimension (producing a change of a magnitude relevant to the person's life). It corresponds to what the educational literature calls formative evaluation or progress monitoring: evaluation carried out during the process, to correct course, rather than only at the end, to note the result. In the Ethics Code for Behavior Analysts (BACB), standards 2.17 ("Collecting and Using Data") and 2.18 ("Continual Evaluation of the Behavior-Change Intervention") make the practice mandatory: displaying data graphically and using them to decide whether to continue, modify, or end services, and taking corrective action when the data indicate the desired outcomes are not being achieved.

Why impressions aren't enough

No one needs bad faith to be mistaken about their own work. An intervention that produces an effect over several weeks simply isn't perceptible day to day — and neither is one that produces no effect at all. Memory holds on to the striking sessions, not the typical ones; a good week weighs more in recollection than the three average ones that came before it; and the expectations of whoever planned the procedure tilt the reading in its favor. That's why "it's going well" can mean, without anyone noticing, simply "I'm enjoying what I'm doing."

The problem has two sides, and both are costly. On one side, sticking too long with something that isn't working: every lost week is a week the person could have been learning with a different procedure. On the other, giving up too soon on something that was working slowly, trading a real gain for a fresh start. Without measurement, both things happen and no one finds out.

The antidote isn't to distrust professional judgment — it's to give it something better to look at. As Bushell and Baer summed up in 1994, in a chapter with an almost polemical title, "Measurably superior instruction means close, continual contact with the relevant outcome data. Revolutionary!", what separates instruction that is known to be good from instruction that is merely assumed to be good is closeness to the data. The data don't exist to prove the professional was right; they exist so that error shows up early, while there's still time to correct it.

Behavior Analysis has inherited this stance in a radical way. In Precision Teaching, Ogden Lindsley summarized the approach as "basing educational decisions on changes in continuously self-monitored performance frequencies, displayed on standard celeration charts." One of its principles is the phrase that best captures the spirit of this whole entry: "the learner knows best." If the data show the person isn't progressing, what needs to change is the procedure — not the learner.

The decision cycle

Data-based decision making isn't a single moment at the end of the process: it's a cycle that turns throughout the entire follow-up period, sometimes weekly, sometimes every session. Each full turn produces an explicit choice — and that choice becomes the input for the next turn.

Set target and criterion 1 Measure the same way 2 Put it on a graph 3 Compare against criterion 4 Decide and act 5 WHAT ARE THE DATA SAYING? today's decision is the starting point for the next measure
The cycle has no natural end: it keeps turning for as long as the follow-up lasts. Notice that step 1 comes before any data — that's where you define what will count as progress, which is why it can't be written after seeing the result. And that step 5 is a recorded choice, with a date: "we kept the procedure," "we increased support," "we changed the reinforcer." A decision that isn't written down can't be evaluated later.

Each step depends on the previous one, and it's common for the cycle to get stuck at one of them. Without an operational definition, two people measure different things and the graph turns into noise. Without reliable recording, the comparison doesn't hold up. Without a baseline, there's nothing to compare against. And without good visual analysis, a finished graph produces no decision at all — it just sits there, nice to look at, unread.

The decision rule: agreed on beforehand, applied afterward

The point that most distinguishes serious practice from an imitation of it is this: the decision rule is set at the start, together with the goal — not once the data are already on the screen. Setting the rule beforehand means answering, in writing, three questions:

  • Where do we want to get to, and by when? — the performance criterion and the deadline. From this comes the aim line: the line connecting the starting point to the goal on the planned date, showing the rate of progress needed.
  • How many measures will we look at before concluding anything? — because one bad data point is a bad day, not a failure. Materials from the MTSS Center and the National Center on Intensive Intervention (NCII) recommend gathering six to nine measures before the first formal reading.
  • What will we do for each possible result? — what counts as sufficient progress, insufficient progress, and an inconclusive result, and which action corresponds to each one.

One of the most widely used rules in school progress monitoring is the four-point rule, attributed to Lynn and Douglas Fuchs and broadly adopted in data-based individualization programs. After gathering enough measures, you look at the four most recent data points relative to the aim line:

the 4 most recent points aim line baseline median goal performance → weeks → 4 points below → change the intervention
The aim line translates the goal into a pace: it shows where performance would need to be each week to reach the target by the agreed date. At the start, the points oscillate around it — a legitimately inconclusive situation. When the four most recent measures are all below the line, the rule already said what to do: change something in the intervention. Note that the rule doesn't say what to change; it only says that staying the same has stopped being a defensible option.

Applied to this graph, the four-point rule has three outcomes and no ambiguity:

  • The four points above the line — progress exceeds what was expected. Consider raising the goal or starting to fade the supports.
  • The four points below the line — progress is insufficient. It's time to change the intervention.
  • Points on both sides — it still isn't possible to conclude anything. Keep the procedure and collect more measures.

There are other equally legitimate rules — comparing the data's trend line with the aim line, or using the median of the last three measures, for example — and in Behavior-Analytic clinical practice it's common for the criterion to take the form of a mastery criterion ("a certain percentage of independent correct responses over an agreed number of consecutive sessions"). What these forms have in common isn't the arithmetic: it's the fact that they were written beforehand, so that no one has to decide, with the graph on the screen, what would count as an acceptable result.

verified

What the research showsThe classic meta-analysis on the topic is by Lynn and Douglas Fuchs (1986), who pooled 21 controlled studies and 96 effect sizes on systematic formative evaluation in education. The weighted average effect on learning was about 0.70 standard deviations — substantial, for a procedure that doesn't change what is taught, only how what is being taught is tracked. And the most useful finding lies in the moderators the study itself identified: the effect size depended on how the data were displayed and evaluated. Secondary readings of this body of studies describe this pattern as a staircase, in approximate values — just collecting the data would yield a modest effect, around 0.26; turning it into a graph would bring it close to 0.70 (a step that nearly matches the study's overall average, since graphing was the typical case); and adding explicit decision rules for analyzing the graph would reach about 0.90. The lesson is direct: a folder full of spreadsheets teaches no one. The gain comes from looking and from deciding.

The four possible answers

Every turn of the cycle ends in one of these four decisions. They are not equivalent, and confusing them is a common source of error — especially treating a "we still can't tell" as if it were a "change something." Explore each one:

Continue as is

The data track the aim line: progress is at the expected pace. The correct decision is to not touch anything — and that is a real decision, not an absence of one. There's a real temptation to "improve" a procedure that's working by adding components; each addition, though, is one more variable and one less chance of being able to explain the effect.

A different situation is one where progress consistently exceeds what was expected. There, the criterion itself has proven modest, and the next step isn't to stand still: it's to raise the goal or start reducing supports. That is already a planned change — and it's the second case covered in the "Change" tab.

Continue and collect more data

The points are on both sides of the line, or vary too much to support any reading. The honest verdict is "we still can't tell," and that isn't failure: it's the recognition that more measures are needed.

This is the most uncomfortable decision, because it looks like inaction — and that's exactly why it needs to be built into the rule ahead of time. Without that, the pressure to "do something" leads to changing the procedure with every fluctuation, and a procedure that changes every week never actually gets tested. If variability is very high, the question to ask is about the measure and about the consistency of implementation, not about the technique.

Change the intervention

The data indicate insufficient progress. The rule says that staying the same is off the table; it doesn't say what to put in its place. That part comes from functional assessment, from the evidence literature, and from what is already known about the person.

Not every change is born from a poor result. When the data exceed the aim line consistently, the change goes upward: a more ambitious goal — which is an adjustment to the criterion, not to the intervention — or starting to fade supports, which is a procedural change like any other and deserves the same care.

One practical recommendation runs through the whole field: change one thing at a time. If the reinforcer, the level of support, and the mastery criterion all change in the same week, and performance improves, there's no way to know what to credit for the improvement — nor what to preserve when something stalls again. Before swapping the technique, it's also worth checking the more likely possibility first: was the procedure being implemented as planned?

End or move forward

The criterion has been met. Ending is a data decision like any other — and one that tends to be postponed out of insecurity, prolonging instruction that has already served its purpose.

Ending well rarely means stopping measurement. The next step is to check whether the behavior holds up without the procedure and shows up in the places and with the people that matter — which is the subject of generalization and maintenance. It's also the moment to ask the people involved whether the result made a real difference in their lives: social validity is a question the graph can't answer on its own.

Where data-based decision making slips

Talking about "data" lends an air of objectivity that doesn't always match what's actually happening. Flip the cards to see the most common ways of keeping the full appearance of the method while losing its substance:

warning

A graph doesn't choose the goalData answer two questions well: did it change? and did it change because of what we did? They don't answer the third, which is the most important one: was it worth changing? A number can climb flawlessly toward a goal the person themselves didn't choose, or one that serves the convenience of the people around them more than their own life. That's why choosing targets involves consent and assent and social validity — and why reducing expressions of identity, such as stereotypy/stimming, doesn't become a legitimate goal just because it's easy to count. Good data aimed at the wrong target is still the wrong target.

formative evaluation progress monitoring decision rule aim line mastery criterion four-point rule

Test your understanding

quiz

Quick quiz

Three questions to reinforce the concept. You'll see the explanation right away.

Further reading

  1. Cooper, J. O.; Heron, T. E.; Heward, W. L. Applied Behavior Analysis. 3rd ed. Pearson, 2020 — continuous measurement, graphic display, and decisions about teaching programs.
  2. Baer, D. M.; Wolf, M. M.; Risley, T. R. “Some current dimensions of applied behavior analysis”. Journal of Applied Behavior Analysis, 1(1), 91–97, 1968 — the analytic and effective dimensions.
  3. Behavior Analyst Certification Board (BACB). Ethics Code for Behavior Analysts, 2020 (in effect since 1/1/2022) — standards 2.17 (collecting and using data) and 2.18 (continual evaluation of the intervention).
  4. Fuchs, L. S.; Fuchs, D. “Effects of systematic formative evaluation: a meta-analysis”. Exceptional Children, 53(3), 199–208, 1986 — 21 controlled studies; average effect around 0.70, moderated by data display and evaluation rules.
  5. Lindsley, O. R. “Precision teaching: discoveries and effects”. Journal of Applied Behavior Analysis, 25(1), 51–57, 1992 — educational decisions based on continuously monitored frequencies.
  6. Bushell, D.; Baer, D. M. “Measurably superior instruction means close, continual contact with the relevant outcome data. Revolutionary!”. In: Gardner, R. III et al. (Eds.), Behavior Analysis in Education: Focus on Measurably Superior Instruction, pp. 3–10. Brooks/Cole, 1994.
  7. National Center on Intensive Intervention (NCII). Decision Rules for Analyzing Academic Progress Monitoring Data — the four-point rule, trend line, and median of the last three measures. See also MTSS Center, Better Instructional Decision Making With Progress Monitoring — the rule's three outcomes and the number of measures before the first reading.
  8. The Wing Institute. What Is the Effect of Formative Evaluation on Student Achievement? — a component-by-component reading of the Fuchs & Fuchs (1986) results: collecting, graphing, and applying decision rules.
info

This entry is educational and does not replace assessment, diagnosis or treatment by a qualified health professional. Every person is unique — for a specific case, seek individual guidance.

Looking for professional guidance?

Instituto Walden4 is a psychology and education center in Brasília, Brazil, offering evidence-based assessment and care for autism, psychotherapy and development — in person and online. Our team serves in Portuguese; get in touch and we will find the best way to help.

chat Message us on WhatsApp
chat

Conta Walden4

Entrar

Use o e-mail e a senha da sua conta.