Statistics Concepts: Variables, Distributions, and Inference
Lesson 1: Variables
- Explanatory Variable – aka Independent Variable; explains variations in the response variable (x-axis). This is the predictor.
- Example: “Can quiz scores be used to predict exam scores?” (Explanatory = Quiz scores)
- Response Variable – aka Dependent Variable; its value is predicted or its variation is explained by the explanatory variable (y-axis). This is the outcome.
Lesson 2: Variable Types and Data Visualization
- Categorical vs. Quantitative Variables
- Categorical Variables = names,
Essential Causal Inference and Econometrics Techniques
Randomized Experiments and Causal Inference
Why are randomized experiments so desirable?
Randomization breaks the link between treatment assignment and confounders, making treated and untreated groups exchangeable. This guarantees unbiased estimates of causal effects (on average) because any differences in outcomes can be attributed to the treatment rather than selection.
Why might we not be able to run a randomized experiment?
They may be unethical (e.g., denying beneficial treatments), infeasible
Essential Data Science Concepts and Statistical Methods
Data Science Fundamentals
Data Science combines statistics, computer science, and domain knowledge to extract insights from data. The main goal is to uncover hidden patterns, trends, and other valuable information from large datasets to make informed, data-driven decisions. It deals with both structured (e.g., Excel tables) and unstructured (e.g., text, images) data.
The Data Science Lifecycle
- Problem Definition: Understanding the business question.
- Data Collection: Gathering data from various sources.
Statistical Sampling Distributions and Inference Exercises
Review Exercises for Sampling Distributions
8.56 Consider the data displayed in Exercise 1.20 on page 31. Construct a box-and-whisker plot and comment on the nature of the sample. Compute the sample mean and sample standard deviation.
8.57 If X1, X2, …, Xn are independent random variables having identical exponential distributions with parameter θ, show that the density function of the random variable Y = X1 + X2 + … + Xn is that of a gamma distribution with parameters α = n and β = θ.
8.58
Read MoreStatistical Concepts: T-Distribution, ANOVA, and Hypothesis Testing
Chapter 12 Key Terms
The t distribution is similar to the z distribution in that both are symmetrical, bell-shaped sampling distributions. However, the overall shape of the t distribution is strongly influenced by the sample size used to generate it. For very large samples, the t distribution approaches the z distribution, but for smaller samples, the t distribution is flatter.
A t test is a test of the null and research hypotheses used when the research design involves two samples. It tests the difference
Read MorePractical CSV and Jupyter Data Extraction with Pandas
Case 2 — Data Extraction and Transformation
Basics of CSV & Jupyter
CSV – Comma Separated Values. Commas separate columns; missing values appear as blank entries or NaN. Jupyter Notebook rules – If running a cell in the middle, run all previous cells first. Keep the kernel up-to-date with earlier variable definitions and avoid running cells out of order to prevent NameError issues.
Volatility and Simple Calculations
Volatility: Defined as values above the median; the median splits the distribution
Read More