Glossary
Alternative hypothesis (Ha)
A statement that contradicts the null hypothesis, proposing that there is an effect, a difference, or a relationship. For regression, it typically states the population slope is not zero (or is positive/negative).
Example:
If we suspect that more hours of exercise lead to lower body fat, the alternative hypothesis would state that there is a negative linear relationship (i.e., the true slope is less than zero).
Degrees of Freedom (n-2)
A parameter that specifies the shape of the t-distribution, calculated as the sample size (n) minus the number of parameters estimated (2 for slope and intercept in simple linear regression).
Example:
For a regression analysis with 30 data points, the degrees of freedom for the t-distribution of the slope would be 30 - 2 = 28.
Fail to reject the null hypothesis (H0)
The decision made in a hypothesis test when the p-value is greater than or equal to the significance level, indicating insufficient evidence to conclude against the null hypothesis.
Example:
If a study on a new teaching method yields a p-value of 0.12 (and alpha is 0.05), we would fail to reject the null hypothesis, meaning there isn't enough evidence to claim the new method significantly improves scores.
Hypothesized population slope (β)
The true, unknown slope of the regression line for the entire population, which is the value assumed by the null hypothesis (often 0 for testing a linear relationship).
Example:
In a test to see if there's any linear relationship between temperature and ice cream sales, the hypothesized population slope (β) in the null hypothesis would be 0.
Null hypothesis (H0)
A statement of no effect, no difference, or no relationship, which is assumed to be true until evidence suggests otherwise. For regression, it typically states the population slope is zero.
Example:
In a study examining if caffeine intake affects sleep, the null hypothesis would be that there is no linear relationship between caffeine intake and hours of sleep (i.e., the true slope is zero).
Reject the null hypothesis (H0)
The decision made in a hypothesis test when the p-value is less than the significance level, indicating sufficient evidence against the null hypothesis.
Example:
If the p-value for the relationship between fertilizer and crop yield is 0.01 (and alpha is 0.05), we would reject the null hypothesis, concluding that fertilizer does have a significant linear effect on yield.
Sample slope (b)
The estimated slope of the least-squares regression line calculated from the sample data, representing the predicted change in the response variable for a one-unit increase in the explanatory variable.
Example:
If a study finds that for every additional hour of studying, exam scores increase by 3 points, then 3 is the sample slope.
Significance level (alpha)
The predetermined threshold probability (e.g., 0.05) used to decide whether to reject the null hypothesis; it represents the maximum acceptable probability of making a Type I error.
Example:
Setting a significance level of 0.05 means we are willing to accept a 5% chance of incorrectly rejecting a true null hypothesis.
Standard error of the sample slope (SEb)
A measure of the variability or precision of the sample slope estimate, indicating how much sample slopes are expected to vary from the true population slope.
Example:
A small standard error of the sample slope suggests that the calculated sample slope is a more precise estimate of the true population slope.
T-Distribution
A probability distribution used when estimating population parameters with small sample sizes or unknown population standard deviation, particularly relevant for the sampling distribution of the regression slope estimate.
Example:
When analyzing the relationship between study hours and exam scores for a small group of 20 students, the sampling distribution of the regression slope will follow a t-distribution rather than a normal distribution.
Type II error
Occurs when a hypothesis test fails to reject a false null hypothesis, meaning a real effect or relationship is missed.
Example:
In a drug trial, a Type II error would mean concluding the new drug has no effect on a disease when, in reality, it does have a beneficial effect.
p-value
The probability of observing a sample statistic as extreme as, or more extreme than, the one obtained, assuming the null hypothesis is true.
Example:
A p-value of 0.03 for a regression slope means there's a 3% chance of seeing a slope as extreme as the one observed if there truly were no linear relationship in the population.
t-score
A standardized test statistic that measures how many standard errors a sample statistic (like a regression slope) is away from the hypothesized population parameter.
Example:
If a researcher finds a t-score of 2.5 for the slope relating advertising spend to sales, it indicates the sample slope is 2.5 standard errors above the hypothesized population slope (often zero).