p-value Hacking

P-value hacking, or "p-hacking," means massaging your data or your analysis until a finding that isn't significant looks significant, which usually means getting the p-value under 0.05.

In hypothesis testing, the p-value is the chance of getting a result at least as extreme as what you observed, assuming the null hypothesis is true. The null hypothesis usually says there's no effect, or no relationship between the two things you measured.

People who p-hack usually do some of these:

1. Run a bunch of analyses with different combinations of variables, then report only the ones that came out significant.
2. Keep collecting data until the test hits significance.
3. Drop or keep outliers depending on which choice makes the result significant.
4. Transform the data until the result looks more "favorable."
5. Run a two-tailed test, then report the result as significant based on a one-tailed test.

The problem is more false positives, results that look significant but are really just chance. That makes research less reliable and harder to reproduce. Lately the scientific community has been pushing back, for example by pre-registering study designs, publishing null results, and looking at effect sizes and confidence intervals instead of p-values alone.

In a way, it's related to the Veil of Ignorance.