• Analytics
  • Alerts
  • Node.js
  • Backend

Z-score anomaly detection for daily app metrics in Node.js

Z-score anomaly detection tested on 184 days of real data: why a plain rolling z-score misses drops, and a robust, weekday-aware version in Node.js.

Author
Extenify Team
Published
Reading time
14 min read
A scooter dashboard at night with an illuminated orange speedometer and a small fuel gauge

Z-score anomaly detection is the standard first answer to a question every app and extension developer runs into: which of today's numbers deserve an alert? Daily installs, weekly users, ratings and review counts all move a little every day, and an alert on every move trains your team to ignore alerts. A z-score asks a better question: how unusual is today compared with recent days?

The textbook version is three lines of code. It is also, on real data, almost blind. We ran it on 184 days of a real public daily series with the same shape as store metrics, and it flagged one day out of 184, missed every day the source reported zero, and missed a 30% drop we injected into a normal week. This guide walks through why, and builds a version that works: missing data handled as missing, the weekday pattern removed, and a robust z-score based on the median. Everything is plain Node.js with no packages.

Key takeaways

  • A z-score is (today - mean) / standard deviation over a recent window. A threshold of 3 is common.
  • On daily data with a weekly rhythm, the weekday swings inflate the standard deviation so much that a plain rolling z-score misses real drops. In our test it scored a 30% drop at -0.11.
  • Treat impossible zeros as missing data, remove the weekday pattern before scoring, and use the median and median absolute deviation (MAD) instead of the mean and standard deviation.
  • The robust, weekday-aware version caught the injected drop on its first day (-31%, score -3.4) and raised two alerts in four months of normal data.
  • Daily review counts are small numbers: test them with a Poisson probability instead of a z-score.

The test data: 184 days of real daily downloads

Synthetic data makes every detector look good, so we used a real series: daily npm downloads of webextension-polyfill, the library many cross-browser extensions use, from May 1 to October 31, 2026, read from the public npm downloads API. It has every property that makes store metrics hard:

  • A strong weekly rhythm. The median Saturday had 127,590 downloads, 38% of the median Wednesday's 331,723. Installs and active users of developer tools follow the same weekday pattern.
  • A one-day spike. June 23 had 757,726 downloads, more than double a normal Tuesday.
  • Data gaps. Eleven days came back as exactly 0, including October 13 to 15 and October 19 to 21. The react package returned zeros on the same eleven days, so these are reporting gaps in the source, not a collapse in usage. Store dashboards have the same kind of gaps and late corrections.
  • Growth. From mid-September, weekday downloads ran about 20% above the summer level.

To check that a detector catches a real problem, we also made a copy with a 30% drop on October 27 to 29, a Tuesday to Thursday, the kind of loss a broken update causes.

What a z-score measures

A z-score says how many standard deviations a value sits from the mean:

z = (value - mean) / standard deviation

For a time series you compute the mean and standard deviation over a trailing window, for example the previous 28 days, and flag today when |z| passes a threshold. Values around 3 are the usual choice, because for normally distributed data a value that far out is rare. The best-known practical write-up is the "smoothed z-score" answer to Peak signal detection in realtime timeseries data on Stack Overflow, viewed more than 390,000 times, which adds an influence parameter so that past signals do not distort the window.

Why plain z-score anomaly detection fails on daily metrics

We ran the plain version (28-day window, threshold 3) on the real series:

Test Result
Days flagged in 184 1 (the June 23 spike)
Zero-download days flagged 0 of 11. September 3 scored -2.95, October 13 scored -2.19
Injected 30% drop on October 27 Not flagged, score -0.11
Same drop at threshold 2 Still not flagged

Three effects stack up:

  1. The weekly rhythm inflates the standard deviation. A window that mixes weekdays near 330,000 with weekends near 130,000 has a huge spread. Against that spread, almost nothing on a weekday looks unusual.
  2. The mean is dragged by the weekend. The 28-day mean sits between weekday and weekend levels, so a weekday that lost 30% lands close to the mean. That is why the injected drop scored almost exactly zero.
  3. Bad points poison the window. Zeros and spikes inside the window raise the standard deviation further, which hides the next problem. A zero scored only -2.95 because earlier zeros had already widened the spread.

The lesson is not that z-scores are wrong. It is that the mean and standard deviation of a raw daily series are the wrong baseline.

Fix 1: treat impossible zeros as missing data

A product with hundreds of thousands of daily downloads or tens of thousands of users never has a true zero. A zero is a failed fetch or a reporting gap. Mark it as missing, skip it when computing the baseline, and do not score it. Send a separate "no data" alert instead, because a collector that silently stops is a problem of its own.

Fix 2: compare a day with the same weekday

The obvious way to remove the weekly rhythm is to compare Tuesday with previous Tuesdays: the median of the same weekday over the last eight weeks. We tried it, and it taught two more lessons:

  • It flagged 11 days on the real series, most of them in September as the series grew 20%. An eight-week median of the same weekday adapts slowly, so growth looks like a run of anomalies for weeks. It also flagged Saturday, July 4, 25% below a normal Saturday, which is the US Independence Day holiday: correct, but not something you want paged about.
  • It missed the injected drop. Five of the eight previous Tuesdays were reporting gaps, so October 27 did not have enough history to be scored. For October 28 the median still reflected the lower level of early September, so a 30% drop from the current level was only a 15% drop from the baseline.

Eight data points per weekday is too few, and they are too old.

Fix 3: remove the weekday pattern, then use a robust z-score

The version that works separates the two jobs:

  1. Learn the weekday pattern from the last eight weeks: for each weekday, the median of that weekday divided by the median of all days. In our series, Saturday's factor was about 0.4.
  2. Remove it by dividing each day by its weekday factor. A Saturday and a Wednesday become comparable, so the last 14 days can all serve as the baseline instead of two same-weekday points.
  3. Score with the median and MAD, not the mean and standard deviation. The NIST/SEMATECH e-Handbook of Statistical Methods gives the modified z-score as 0.6745 × (x - median) / MAD and, citing Iglewicz and Hoaglin, recommends treating an absolute value above 3.5 as a potential outlier. A spike or a gap inside the window barely moves the median, so one bad day cannot hide the next one.
  4. Require a minimum change as well as a high score, so a statistically unusual but tiny move does not wake anyone up. We used 15%.

Here is the script. It reads a JSON array of { day, value } rows, oldest first, and prints the flagged days. It needs Node.js 18 or newer.

// detect-anomalies.mjs: flag unusual days in a daily metric (installs, weekly users, downloads).
// Usage: node detect-anomalies.mjs series.json
// series.json: [{ "day": "2026-10-01", "value": 1234 }, ...], oldest first, one row per day.
import { readFile } from 'node:fs/promises';

const median = (xs) => {
  const s = [...xs].sort((a, b) => a - b);
  const mid = Math.floor(s.length / 2);
  return s.length % 2 ? s[mid] : (s[mid - 1] + s[mid]) / 2;
};
const weekday = (day) => new Date(`${day}T00:00:00Z`).getUTCDay();

// A zero your source never reports for a live product is a missing day, not a collapse.
export const markGaps = (series) =>
  series.map((p) => ({ day: p.day, value: p.value > 0 ? p.value : null }));

export function detectAnomalies(series, {
  factorDays = 56,  // history used to learn the weekday pattern (8 weeks)
  window = 14,      // recent days that define "normal" after removing the weekday pattern
  threshold = 3,    // modified z-score that counts as unusual
  minChange = 0.15, // and the change must also be at least 15% to be worth a message
} = {}) {
  const flags = [];
  for (let i = factorDays; i < series.length; i++) {
    const today = series[i];
    if (today.value === null) continue;

    // 1. Weekday factors: how each weekday compares with a typical day.
    const history = series.slice(i - factorDays, i).filter((p) => p.value !== null);
    const typical = median(history.map((p) => p.value));
    const factor = Array.from({ length: 7 }, (_, d) => {
      const values = history.filter((p) => weekday(p.day) === d).map((p) => p.value);
      return values.length >= 3 ? median(values) / typical : 1;
    });

    // 2. Remove the weekday pattern from the last `window` days and from today.
    const recent = series
      .slice(i - window, i)
      .filter((p) => p.value !== null)
      .map((p) => p.value / factor[weekday(p.day)]);
    if (recent.length < window / 2) continue; // too many missing days to judge
    const adjusted = today.value / factor[weekday(today.day)];

    // 3. Robust z-score: median and median absolute deviation instead of mean and standard deviation.
    const base = median(recent);
    const mad = median(recent.map((v) => Math.abs(v - base)));
    if (mad === 0) continue;
    const score = (0.6745 * (adjusted - base)) / mad;
    const change = (adjusted - base) / base;

    if (Math.abs(score) >= threshold && Math.abs(change) >= minChange) {
      flags.push({
        day: today.day,
        value: today.value,
        expected: Math.round(base * factor[weekday(today.day)]),
        change: `${change > 0 ? '+' : ''}${Math.round(change * 100)}%`,
        score: Number(score.toFixed(1)),
      });
    }
  }
  return flags;
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const series = markGaps(JSON.parse(await readFile(process.argv[2], 'utf8')));
  const missing = series.filter((p) => p.value === null).length;
  console.log(`${series.length} days, ${missing} missing`);
  for (const flag of detectAnomalies(series)) console.log(JSON.stringify(flag));
}

On the copy with the injected drop it printed:

184 days, 11 missing
{"day":"2026-09-20","value":167049,"expected":141113,"change":"+18%","score":8.9}
{"day":"2026-10-24","value":193089,"expected":160325,"change":"+20%","score":3}
{"day":"2026-10-27","value":297371,"expected":431132,"change":"-31%","score":-3.4}
{"day":"2026-10-29","value":315044,"expected":428617,"change":"-26%","score":-3}

On the real series without the drop, only the first two lines appear: two flags in the four months the detector could score, both weekends about 20% above normal, which is the kind of thing worth a glance and nothing more.

How the three detectors compare

Detector Flags on real data Caught the injected 30% drop
Rolling z-score, mean and standard deviation, 28 days 1 No (score -0.11)
Same weekday, median and MAD, 8 weeks 11 No (not enough history, then a lagging baseline)
Weekday pattern removed, median and MAD, 14 days 2 Yes, on the first day (score -3.4)

Know its limits

Every detector has a floor. In this series, normal day-to-day noise was about 6% of the level, so a drop has to be roughly a quarter of the level or more to pass a threshold of 3. A copy with a 20% drop went unflagged, and the middle day of the 30% drop scored -2.6 and was missed too. For a smoother metric the floor is lower. The detector also needs eight weeks of history before it can score anything, which is why it could not judge the June 23 spike: lower factorDays for new products and accept more noise until the history builds up.

Apply it to each store metric

The same idea works for most of what a store dashboard shows, with a few changes:

  • Weekly users. The Chrome Web Store's weekly users and similar metrics are already seven-day totals, so the weekday pattern is mostly gone and the series is smooth. A 14-day median and MAD is enough, and you can lower minChange to 5 to 10%. Our guide to Chrome extension weekly users dropping lists the causes to check once it fires.
  • Daily installs and downloads. Use the script as it is. Weekday patterns in installs are strong, as they were in our test data.
  • Rating. Do not score the overall average; it barely moves. Score the average of new ratings between two snapshots instead, as explained in our guide to App Store rating history.
  • Category rank. Rank is ordinal and jumps in steps. Alert on a change in position bands (top 10, top 50, top 100) rather than on a z-score.

Small counts: use a Poisson test for reviews

Many apps get a handful of reviews a day. With counts that small, the median and MAD are often zero and a z-score is meaningless. For counts of rare events, the Poisson distribution answers the real question directly: how likely is today's count if the usual daily rate is lambda?

// Probability of seeing `k` or fewer events on a day when `lambda` are expected (Poisson).
export function poissonAtMost(k, lambda) {
  let term = Math.exp(-lambda); // P(X = 0)
  let total = term;
  for (let i = 1; i <= k; i++) {
    term *= lambda / i;
    total += term;
  }
  return total;
}
// Probability of seeing `k` or more.
export const poissonAtLeast = (k, lambda) => (k === 0 ? 1 : 1 - poissonAtMost(k - 1, lambda));

console.log(poissonAtMost(0, 2).toFixed(3));  // no reviews when 2 are typical
console.log(poissonAtMost(3, 12).toFixed(4)); // 3 reviews when 12 are typical
console.log(poissonAtLeast(9, 2).toFixed(5)); // 9 one-star reviews when 2 are typical

The output was 0.135, 0.0023 and 0.00024. A day with no reviews when two are typical happens about one day in seven, so it is not news. Nine one-star reviews on a day that usually brings two has a probability of about 1 in 4,000: that is a bad release or a review bombing, and it deserves an alert the same day. Estimate lambda from the median of the last 28 days, and alert when the probability falls below 0.001.

Turn flags into alerts people trust

A detector produces flags. An alert system needs a few more rules:

  1. Alert once per episode. If three days in a row are flagged in the same direction, that is one problem. Send the first and then a daily summary until it clears.
  2. Mind the direction. A drop in installs and a rise in one-star reviews are urgent. A jump in installs is good news for a weekly report.
  3. Mark known events. Releases, store featuring, marketing pushes and public holidays explain many flags. Put them on the same timeline so whoever reads the alert sees the likely cause next to it.
  4. Keep a "no data" alert. Missing days were 6% of our series. A collector that stops looks exactly like a quiet week unless you check for it.

Our guide to setting up store change alerts your team will actually read covers channels and digests, and JSON diff in Node.js shows how to turn non-numeric changes, such as a new version or a changed permission, into alerts. To store the daily rows for any of this, see PostgreSQL time series without TimescaleDB.

Get store anomaly alerts without building the pipeline

Extenify runs this kind of daily comparison for you across the Chrome Web Store, Edge Add-ons, Firefox Add-ons, Google Play and the App Store. It records installs, ratings, reviews, versions and category rank every day, for your own apps and extensions and for competitors, and sends alerts for install swings, rating drops, fresh one-star reviews, new versions and rank jumps, against thresholds you set, to email, Slack, Discord, Telegram or Microsoft Teams. The Chrome extension metrics guide covers which numbers are worth watching in the first place. Every plan starts with a free trial; see Extenify pricing.

Frequently asked questions

What is a good z-score threshold for anomaly detection?

For alerting on daily metrics, 3 is a sensible start with a robust z-score, combined with a minimum change such as 15%. NIST, citing Iglewicz and Hoaglin, suggests 3.5 for labeling potential outliers with the modified z-score. Lower thresholds catch smaller problems and send more false alarms; pick the level your team will act on.

What is the difference between a z-score and a modified z-score?

A z-score uses the mean and standard deviation. A modified z-score uses the median and the median absolute deviation, scaled by 0.6745. The median-based version is not distorted by the outliers you are trying to find, which matters for real data with spikes and gaps.

Does z-score anomaly detection work with seasonal data?

Not directly. Weekly or yearly patterns inflate the standard deviation and hide real changes. Remove the pattern first, for example by dividing each day by its weekday factor, and score what is left.

How much history do I need?

Enough to learn the pattern and the noise. Our script uses eight weeks for weekday factors and 14 days for the baseline. With less history, use a simple percentage-change rule until enough data has built up.

How do I detect anomalies in small counts like daily reviews?

Use a Poisson probability instead of a z-score. Estimate the typical daily count, then compute how likely today's count is. A probability below about 0.001 is a reasonable alert threshold.

Why does my rolling z-score never fire?

Usually because the window mixes very different days, such as weekdays and weekends, or contains spikes and zeros that inflate the standard deviation. Remove the weekday pattern, exclude missing days and switch to the median and MAD.

Summary

Z-score anomaly detection works on daily app metrics once you stop feeding it raw data. A plain rolling z-score on 184 days of real downloads flagged a single spike and missed every reporting gap and a 30% drop. Treating zeros as missing, removing the weekday pattern and scoring with the median and MAD caught the drop on its first day with only two other flags in four months. Use a Poisson test for small counts, group flags into episodes, and keep a separate alert for missing data.

Image credits

Put every store in one dashboard tonight.

Sign in with Google, paste one store link, and your unified view is live in under a minute. Then 7 days of Pro free, no credit card required, and you never touch your app's code.

Start free trial