The irony of the random variable (r.v.) is that although it takes on an “unpredictable” value every time, it’s not exactly random if we understand the shape of its distribution. This is why descriptive statistics matters a lot—they define the boundaries of the set of values an r.v. can take, otherwise known as, again, the shape of its distribution. There isn’t just one descriptive statistics, so it sure would be nice if we had a way to systematically identify these instead of applying different formulas for each one. Well. Actually, there is. It’s called moments. *** Before we even delve…

One assumption we discussed for linear regression is the independence of error terms. In that setting, we were typically dealing with cross-sectional data, where we assumed that observations don’t influence each other. Time series data is a little special. Over time, observations are rarely ever independent. If we observe that today’s stock price is high, there’s a pretty good chance that yesterday’s price is high. In other words, values in a time series tend to be correlated with their past. We call this dependence across time autocorrelation. As if to smoothen the transition between the two, many time series models…

One of the most important assumptions for statistical models to work is the notion of consistency. This means that statisticians often drool with excitement when they find out that their data has approximately stable statistical properties, because they can finally unlock the cabinet of unused dusty models. In time series analysis (and several other disciplines), this consistency is coined stationarity. *** Stationarity is often defined in two ways: strictly and weakly. Strict stationarity requires that the joint distribution of a set of values be the same for all time points. Mathematically, given the set of values at time tt, strict…

We’ve briefly talked about how pp-values should be interpreted. It’s crucial to understand that a pp-value of 0.01 doesn’t mean that there is a 1% chance of some null hypothesis being true. Instead, it implies a 1% chance of observing data as extreme or more extreme than the current data under the condition that the null hypothesis is true. pp-values form the foundation of frequentist statistics, and rightfully so because it determines whether a certain event (alternative hypothesis) is probably true by thresholding it against a determined significance level and repeating the experiment many times. However, it’s precisely because the…

The normal distribution is one of statistics’ most precious models of reality. It’s analytically tractable, computationally simple, and provides a universal language for uncertainty. As such, it definitely deserves an in-depth exploration of its characteristics, properties, and significance. And then, we’ll explode in, Game of Thrones style, to ruin the perfect rainbow world of normality by introducing the real world we live in. The normal (Gaussian) distribution is the most famous continuous distribution with a bell-shaped p.d.f. φ\varphi, defined by only two parameters: mean μ\mu and standard deviation σ\sigma. Given a normal r.v. X∼𝒩(μ,σ2)X \sim \mathcal{N}(\mu, \sigma^2), Having two descriptive…

There was a time when I used to apply linear regression to some data and if the resulting metrics (R2R^2, RMSE, MAE, etc.) were unsatisfactory, I simply concluded that the regression wasn’t a good fit and I should probably instead look at other models like gradient boosting or neural networks. If you don’t think this was a problem, see it in this way: a baker judges whether a cake is baked properly just by looking at the clock, without ever inserting a toothpick to check inside the cake. The issue is that we tend to only use metrics to determine…