# `hpfracc.utils` — architecture, dependencies, and maintenance

This note complements [CONTRIBUTING.md](https://github.com/dave2k77/hpfracc/blob/main/CONTRIBUTING.md), [ALGORITHMS_ARCHITECTURE.md](ALGORITHMS_ARCHITECTURE.md), [SPECIAL_ARCHITECTURE.md](SPECIAL_ARCHITECTURE.md), [ANALYTICS_ARCHITECTURE.md](ANALYTICS_ARCHITECTURE.md), [BENCHMARKS_ARCHITECTURE.md](BENCHMARKS_ARCHITECTURE.md), [VALIDATION_ARCHITECTURE.md](VALIDATION_ARCHITECTURE.md), and [SOLVERS_ARCHITECTURE.md](SOLVERS_ARCHITECTURE.md). It describes **utility modules** (errors, memory/cache, plotting), **import coupling**, and **tests**.

---

## 1. Design goals

1. **Validation-oriented helpers** without pulling numerical engines: `ErrorAnalyzer`, **`ConvergenceRateAnalyzer`** (deprecated alias `ConvergenceAnalyzer`), and `ValidationFramework` support benchmarks and analytical checks.
2. **Lightweight default import**: `import hpfracc.utils` loads **error analysis** and **memory management** only; **plotting** (Matplotlib) loads **on first use** of a plot symbol (see §3).
3. **Clear boundaries**: `utils` does **not** import `algorithms`, `core`, or `ml`; **validation** may import `utils.error_analysis`.

---

## 2. Module layout

| File | Responsibility |
|------|----------------|
| **`error_analysis.py`** | `ErrorAnalyzer` (L1/L2/L∞, MAPE, `compute_all_errors`), **`ConvergenceRateAnalyzer`** (rates, `analyze_convergence`, `analyze_convergence_behavior`; deprecated subclass **`ConvergenceAnalyzer`**), `ValidationFramework`, module helpers `compute_error_metrics`, `analyze_convergence`, `validate_solution`, `error_statistics`, etc. |
| **`memory_management.py`** | `MemoryManager` (RSS/VMS via **psutil**), `CacheManager` (size/eviction), `get_memory_usage`, `clear_cache`, `optimize_memory_usage`. |
| **`plotting.py`** | `PlotManager`, `setup_plotting_style`, `create_comparison_plot`, `plot_convergence`, `plot_error_analysis`, `save_plot` — **Matplotlib** at import time of this submodule only. |
| **`__init__.py`** | Re-exports public API; **lazy** attribute access for plotting symbols so `import hpfracc.utils` avoids Matplotlib when plot helpers are not needed. |

---

## 3. Lazy plotting surface (`__init__.py`)

These names resolve via **`__getattr__`** and import **`plotting`** only when first accessed:

`PlotManager`, `create_comparison_plot`, `plot_convergence`, `plot_error_analysis`, `save_plot`, `setup_plotting_style`.

**Prefer** for analysis-only code:

```python
from hpfracc.utils.error_analysis import ErrorAnalyzer
from hpfracc.utils.memory_management import MemoryManager
```

**Use** `from hpfracc.utils import PlotManager` (or other plot symbols) when figures are required; first access pays the Matplotlib import cost.

---

## 4. Upstream coupling

| Consumer | Typical import |
|----------|----------------|
| **`hpfracc.validation`** | `from ..utils.error_analysis import ErrorAnalyzer` (`benchmarks`, `analytical_solutions`, `convergence_tests`). |
| **Tests / examples** | Package or submodule imports; plotting tests often **`patch`** `hpfracc.utils.plotting.plt`. |

No reverse edges from **`utils`** into **`validation`** at import time.

---

## 5. Dependencies and risks

| Dependency | Where | Note |
|------------|--------|------|
| **NumPy** | All three modules | Required. |
| **psutil** | `memory_management` | Process/virtual memory stats; project already lists psutil for analytics. |
| **Matplotlib** | `plotting.py` only | Heavy; keep behind lazy package exports (§3). |
| **Optional / tests** | Torch tensors in `error_analysis._to_numpy` | Best-effort `.detach().cpu().numpy()` for metrics helpers. |

| Risk | Mitigation |
|------|------------|
| **`save_plot` path with no directory** | `os.makedirs` only when `os.path.dirname(path)` is non-empty (avoids odd edge cases on some platforms). |
| **Stale unittest paths** (`utils.plot_manager`, etc.) | Use real submodule paths: `hpfracc.utils.plotting`, `hpfracc.utils.memory_management`, `error_analysis.ConvergenceRateAnalyzer`. |

---

## 6. Tests

- **`tests/test_utils/`** — expanded and coverage-oriented tests (often patch `plt`).
- **`tests/test_utils.py`**, **`tests/utils/`** — additional smoke paths.
- **`tests_unittest/test_utils.py`** — lightweight unittest smoke aligned with the **actual** public API.

```bash
python -m pytest tests/test_utils/ tests/test_utils.py tests_unittest/test_utils.py -q
```

---

## 7. Related documentation

- [SPECIAL_ARCHITECTURE.md](SPECIAL_ARCHITECTURE.md)
- [BENCHMARKS_ARCHITECTURE.md](BENCHMARKS_ARCHITECTURE.md)
- [ANALYTICS_ARCHITECTURE.md](ANALYTICS_ARCHITECTURE.md)
- [VALIDATION_ARCHITECTURE.md](VALIDATION_ARCHITECTURE.md) (uses `ErrorAnalyzer`; distinct from **`hpfracc.validation.ValidationConvergenceAnalyzer`**)
- [SOLVERS_ARCHITECTURE.md](SOLVERS_ARCHITECTURE.md)