NLP · data quality · evaluation discipline
News Classification Pipeline
A leakage-aware multiclass news classifier built around schema validation, duplicate analysis, drift checks, sparse text features and stratified cross-validation.
Problem
Text classification can look strong for the wrong reasons when duplicates, source effects or train/evaluation overlap leak label information. The project is structured around making the reported validation defensible before choosing a classifier.
What I built
The workflow validates schemas, audits duplicates and overlap, checks development/evaluation drift, removes cross-label duplicate problems, combines word and character TF-IDF with source, ranking, timestamp and missingness features, then compares linear models under stratified cross-validation.
Evaluation discipline and data-quality controls are first-class stages of the pipeline.
Engineering decisions
Where the project becomes more than a demo.
The details that matter are usually in the constraints: who is allowed to decide, what gets measured, what is kept separate, and how failure is represented.
Audit before modeling.
Duplicate, overlap, missingness and drift checks happen before model selection so suspicious signal is visible instead of buried inside a score.
Use sparse models that fit the data.
Word and character TF-IDF plus metadata form one sparse design matrix, making Linear SVM a deliberate baseline rather than reaching for a larger neural model by default.
Record fold-level evidence.
Cross-validation outputs include per-class metrics, confusion matrices, out-of-fold counts and fold diagnostics rather than one train/test number.
Evidence
What the repository actually supports.
text n-grams, metadata, time/missingness features
stratified CV with per-class and fold-level diagnostics
the complete path includes leakage cleaning before training
Limits
What I am not claiming.
- The repository does not present a single headline test score as the project's main claim.
- Model choice is intentionally conservative: linear sparse models are a strong fit for the feature space.
- The value of the project is the defensible workflow around messy text inputs, not architectural novelty.
Signal