Music Genre Classification

Mel-Spectrogram CNN, CRNN, and Audio Spectrogram Transformer (AST) for 10-Class Music Genre Recognition

---

Overview

End-to-end deep-learning pipeline for music genre classification built for the Kaggle jan-2026-dl-gen-ai-project/messy_mashup competition — a GTZAN-derived dataset where each song is pre-separated into 4 stems (bass, drums, others, vocals) and the test set consists of multi-track mashups corrupted with environmental noise.

Three models are trained, compared, and ensembled, with the best (a fine-tuned Audio Spectrogram Transformer) deployed as an interactive Gradio app:

#	Model	Approach	Val Macro-F1	Val Acc
1	GenreCNN	ResNet-style 2D-CNN trained from scratch on 3-channel log-mel spectrograms	0.8625	0.840
2	GenreCRNN	CNN feature extractor + Bidirectional GRU (1.38 M params)	0.6516	0.660
3	GenreAST	HuggingFace AST fine-tuned from MIT/ast-finetuned-audioset-10-10-0.4593 (86 M params, 29.5 M trainable)	0.9249	0.917

---

Demo

A Gradio web app wraps the deployed AST checkpoint hosted on HuggingFace Hub (chopadaansh/ast-music-genre-classifier) and returns the top-5 predicted genres for any uploaded audio clip.

cd deployment
pip install -r requirements.txt gradio
python app.py

Then open the local URL Gradio prints (default http://127.0.0.1:7860).

Note: gradio is imported in app.py but is currently missing from deployment/requirements.txt — installing it explicitly (as shown above) is required.

---

Dataset

Source: Kaggle competition jan-2026-dl-gen-ai-project/messy_mashup (GTZAN-derived).

Classes (10): blues, classical, country, disco, hiphop, jazz, metal, pop, reggae, rock

Structure:

messy_mashup/
├── genres_stems/<genre>/<song_id>/{bass,drums,others,vocals}.wav   # 100 songs/genre × 4 stems
├── mashups/song0001.wav ...                                        # ~3,020 unlabeled test mashups
├── test.csv                                                        # Kaggle submission template
└── ESC-50-master/audio/*.wav                                       # noise pool for augmentation

Train data is synthesized on-the-fly by cross-song stem mashup, additive ESC-50 noise, SpecAugment, and Mixup — no fixed train CSV. Validation set = 200 deterministically generated mashups; test set ≈ 3,020 unlabeled clips for Kaggle scoring.

---

Pipeline

Audio preprocessing

Stage	Detail
Loading	librosa.load(sr=22050, mono=True); AST branch resamples to 16 kHz
Clip length	30 s (n_samples = 661,500) — pad-or-(random)-crop
Spectrogram	melspectrogram(n_fft=2048, hop_length=512, n_mels=128) → power_to_db(ref=max)
Channels	3-channel image: [log-mel, Δ, ΔΔ] → shape (3, 128, 1293)
Normalization	Per-sample min-max → [0, 1]
AST features	HuggingFace ASTFeatureExtractor (returns pt tensors at 16 kHz)

Data augmentation (training only)

• Mashup synthesis — create_mashup() mixes stems across songs (mashup_prob=0.7)
• Additive noise from ESC-50 — noise_prob=0.7, SNR ∈ U(5, 25) dB
• SpecAugment — 2 freq masks (≤8 bands) + 2 time masks (≤15 frames), prob=0.5
• Mixup in feature space — alpha=0.3, prob=0.4
• Test-time augmentation — 3 random 30-s crops per clip, predictions averaged

Training

Hyperparameter	Model 1 (CNN)	Model 2 (CRNN)	Model 3 (AST)
Optimizer	AdamW	AdamW	AdamW
Weight decay	1e-4	1e-4	1e-4
Learning rate	3e-4	3e-4	5e-5
Scheduler	OneCycleLR (per-step)	OneCycleLR	OneCycleLR
Loss	CrossEntropy (label_smoothing=0.1, mixup-aware)	same	same
Batch size	16	16	16
Epochs	20	18	8 (early-stopped)
Mixed precision	torch.amp GradScaler	yes	yes
Early stopping	custom patience=6, min_delta=1e-3	yes	yes
Tracking	Weights & Biases	yes	yes

---

Model Architectures

Model 1 — GenreCNN (ResNet-style CNN)

Input (3, 128, 1293)
  → Conv2d(3→64, k=7, s=2) → BN → ReLU → MaxPool(3, s=2)
  → Conv2d(64→128, s=2)   → BN → ReLU → ResBlock(128)  → Dropout2d(0.10)
  → Conv2d(128→256, s=2)  → BN → ReLU → ResBlock(256)  → Dropout2d(0.15)
  → Conv2d(256→512, s=2)  → BN → ReLU → ResBlock(512)  → Dropout2d(0.20)
  → AdaptiveAvgPool2d(1) → Flatten
  → Linear(→256) → BN1d → ReLU → Dropout(0.4) → Linear(256→10)

Model 2 — GenreCRNN (CNN + Bi-GRU, 1.38 M params)

Input (3, 128, 1293)
  → 3 × [Conv2d → BN → ReLU → MaxPool(2) → Dropout2d]   # 64 → 128 → 256 ch
  → reshape → BiGRU(hidden=128, layers=2, bidirectional)
  → Linear(256→128) → BN1d → ReLU → Dropout(0.4) → Linear(128→10)

Model 3 — GenreAST (Audio Spectrogram Transformer fine-tune)

from transformers import ASTForAudioClassification

model = ASTForAudioClassification.from_pretrained(
    "MIT/ast-finetuned-audioset-10-10-0.4593",
    num_labels=10,
    ignore_mismatched_sizes=True,
)
# 86,196,490 total params  |  29,493,514 trainable

---

Results

Per-class F1 — best model (AST)

Genre	F1	Genre	F1
classical	1.000	disco	0.909
jazz	1.000	reggae	0.880
metal	0.960	country	0.815
hiphop	0.960	rock	0.762
blues	0.957
pop	0.917

Cross-model observation: rock is consistently the hardest class (recall 0.667 in AST, 0.550 in CNN, 0.250 in CRNN) — frequently confused with adjacent guitar-heavy genres. classical and metal are the easiest across all three models.

Why the ensemble?

The final Kaggle submission averages predictions from all three models (CNN + CRNN + AST). Despite the CRNN's lower standalone F1, its different architectural prior reduces variance on borderline cases.

---

Tech Stack

• Deep learning — PyTorch, torch.nn, torch.amp (mixed-precision)
• Pretrained models — HuggingFace transformers (ASTForAudioClassification, ASTFeatureExtractor)
• Audio — librosa, soundfile
• Numerics — NumPy, pandas
• Metrics — scikit-learn (f1_score, accuracy_score, classification_report)
• Experiment tracking — Weights & Biases (wandb)
• Deployment — Gradio + HuggingFace Hub

---

Project Structure

Music Classification/
├── notebooks/
│   └── main.ipynb              # full training + inference pipeline (3 models, ~140 KB)
├── deployment/
│   ├── app.py                  # Gradio app — loads AST from HF Hub
│   └── requirements.txt        # torch, transformers, numpy, librosa, soundfile (+ install gradio)
└── README.md

---

Installation & Usage

1. Clone

git clone https://github.com/<your-username>/music-genre-classification.git
cd music-genre-classification

2. Train / reproduce results

The full training pipeline lives in notebooks/main.ipynb. It expects the Kaggle dataset mounted at /kaggle/input/jan-2026-dl-gen-ai-project/messy_mashup (or adapt the CFG.data_root path) plus an active W&B login.

pip install torch torchaudio transformers librosa soundfile scikit-learn wandb pandas numpy tqdm
jupyter notebook notebooks/main.ipynb

GPU recommended. AST fine-tuning fits in ~12 GB VRAM at batch_size=16; the from-scratch CNN/CRNN are lighter.

3. Run the demo app

cd deployment
pip install -r requirements.txt gradio
python app.py

The app downloads chopadaansh/ast-music-genre-classifier from HuggingFace Hub on first launch, then accepts audio uploads (WAV, MP3, FLAC, OGG, M4A) and returns top-5 genre probabilities.

---

Future Work

• Add inline plotting (mel-spectrogram, training curves, confusion matrix) to the notebook — currently all metrics live in the W&B dashboard
• Export the AST model to ONNX for CPU-friendly inference
• Try PaSST or BEATs as additional transformer backbones
• Replace the on-the-fly mashup synthesis with a curated, reproducible split for fair benchmarking
• Extend to multi-label classification for songs spanning multiple genres
• Pin versions in deployment/requirements.txt (currently unpinned) and add gradio

---

License

Distributed under the MIT License. See LICENSE for details.

---

_{Trained with PyTorch · Tracked with Weights & Biases · Deployed with Gradio}