Data Science Project Structure
Sources:
Assuming that Python is they main language for a data science project. Assuming also to use Python venv.
The following structure is adapted from: https://github.com/CodeCutTech/data-science-template/tree/dvc-pip
.
âââ README.md
âââ pyproject.toml
âââ .gitignore
âââ LICENSE
â
âââ configs/ # Main configuration file
â âââ main.yaml # Configurations for training model
â âââ model/
| | |ââ model1.yaml
| | |ââ model2.yaml
| âââ process/ # Configuration for process
| âââ process1.yaml
â
âââ data/
â âââ final/ # Data after training
â âââ processed/ # Data after processing
| âââ raw/ # Raw data
|
âââ docs/ # Documentation
âââ models/ # Store models
âââ notebooks/ # Store notebooks
â
âââ src/ # Source code
â âââ __init__.py
â âââ process.py
â âââ __pycache__/
â âââ train_mode.py
â
âââ scripts/ # Utility scripts
â âââ download_data.sh
â âââ download_models.sh
â
âââ tests/ # Unit tests
â âââ __init__.py
â âââ test_process.py
â âââ test_training.py
Explanation with each part:
- For
pyproject.tomlsee pyproject.toml