Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,43 @@ All notable changes to this project are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.8.0] - 2026-07-31

0.7.0 shipped optional numba acceleration that a user had no way to find or to check. This makes it
discoverable and adds one public function to inspect it. No behaviour changes and no numerical
changes.

### Added

- **`python_som.accelerated()`**, which reports whether training will use the compiled kernel:

```python
import python_som

python_som.accelerated() # True once numba is installed
```

A function rather than a constant because a constant would have to resolve at import time, which
forces the numba import on every `import python_som`. That import costs about 80 ms, and 0.7.0
deferred it deliberately; a function defers it to the caller who asked. Named for the capability
rather than the backend, so a future change of backend is not a rename.

Calling it imports numba but does not compile the kernel: `njit` is lazy, so the roughly 500 ms
compile happens on the first call that trains.

- **[Speed up training](https://andremsouza.github.io/python-som/how-to/speed-up-training/)**, a
how-to covering the four levers in order of effect: batch mode, the default distance function,
numba, and map size.

### Fixed

- **The numba acceleration is now findable.** 0.7.0's Install section listed four `pip install`
lines and numba was not among them; the only user-facing mention was one bullet in the feature
list, and the documentation site mentioned numba just once, in a caveat about MiniSom. A published
PyPI description cannot be edited, so correcting it takes a release. The README's Install section
now carries the command, its NumPy consequence, and a link to the how-to, and a packaging test
asserts that section of the built metadata rather than the description as a whole.

## [0.7.0] - 2026-07-31

Batch training is 20x to 40x faster and now beats both comparable libraries by a wide margin. The
Expand Down
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ pip install "python-som[sklearn]" # adds the scikit-learn estimator adapte
pip install "python-som[examples]" # adds matplotlib and seaborn, for the plots
```

Training can also use a compiled kernel, worth up to 2.4x on batch training. It is a separate
package rather than an extra, and it is picked up automatically once present:

```bash
pip install numba # note: numba currently requires numpy<2.5
```

Confirm it is active with `python_som.accelerated()`. See
[Speed up training](https://andremsouza.github.io/python-som/how-to/speed-up-training/).

## Quick start

```python
Expand Down Expand Up @@ -57,7 +67,7 @@ A full worked example with plots is in [examples/iris.py](https://github.com/and

* NumPy is the only runtime dependency; a fresh install is 69 MB across one package
* Batch training 23x to 31x faster than MiniSom and 26x to 94x faster than SOMPY, measured
* Optional numba acceleration: `pip install numba` and it is used automatically
* Optional compiled kernel via numba, used automatically when installed; `accelerated()` reports it
* Stepwise and batch training
* Random, random-sampling and linear (PCA) weight initialization
* Automatic selection of the map size ratio, from PCA
Expand Down
64 changes: 64 additions & 0 deletions docs/how-to/speed-up-training.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Speed up training

Four changes, ordered from largest effect to smallest. The first two cost nothing.

## 1. Use batch training

```python
som.train(data, n_iteration=100, mode="batch")
```

It is the default, it is the mode the optimization work targets, and it needs far fewer iterations:
10 per sample against 1000 for the stepwise modes. If you are passing `mode="random"` or
`mode="sequential"` for no particular reason, change it.

See [Batch vs stepwise](../explanation/batch-vs-stepwise.md) for what the modes do differently.

## 2. Keep the default distance function

The winner search has a fast path that applies only to the Euclidean distance. Supplying your own
`distance_function` falls back to one evaluation per sample, several times slower. If you need a
custom distance, see [Use a custom strategy](use-a-custom-strategy.md); if you do not, leave it
alone.

## 3. Install numba

```bash
pip install numba
```

Nothing else to do. The compiled kernel is detected and used automatically, and results are
identical to the NumPy path. Check it took effect:

```python
import python_som

python_som.accelerated() # True once numba is installed
```

Three things to expect:

- **NumPy is downgraded in that environment.** numba 0.66 requires `numpy<2.5`. Install it into a
virtual environment you are willing to hold below that.
- **The gain is uneven.** Measured on batch training: 2.4x on a 60x60 map, about 1.0x on a 20x20
or a 100x100 one. Measure your own case before accepting the NumPy constraint.
- **The first run is slower.** About 80 ms to import numba and 500 ms to compile, once per process.

## 4. Size the map for the data

Cost grows with the node count, faster than linearly. Kohonen's rule of thumb is about 50 samples
per node on average; well past that, you are paying for resolution the data does not support.

Let the constructor pick the ratio from the data rather than choosing both sides yourself:

```python
som = python_som.SOM(x=20, y=None, input_len=4, data=data)
```

The [measured timings](../explanation/comparison-with-som-libraries.md#against-minisom) show what
map size costs across a range of shapes.

## Why these and not others

[How batch training is computed](../explanation/how-batch-training-is-computed.md) covers what the
library does internally, including the approaches that were measured and rejected.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ nav:
- How-to:
- Save and load a map: how-to/save-and-load-a-map.md
- Reproduce a result: how-to/reproduce-a-result.md
- Speed up training: how-to/speed-up-training.md
- Use a custom strategy: how-to/use-a-custom-strategy.md
- Use the estimator interface: how-to/use-with-scikit-learn.md
- Reference:
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "hatchling.build"

[project]
name = "python-som"
version = "0.7.0"
version = "0.8.0"
authors = [{ name = "André Moreira Souza", email = "msouza.andre@hotmail.com" }]
description = "Python implementation of the Self-Organizing Map"
readme = "README.md"
Expand Down
2 changes: 2 additions & 0 deletions src/python_som/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

from __future__ import annotations

from ._accelerate import accelerated
from ._artifact import ArtifactError, SOMConfig, TrainingReport
from ._core._decay import (
asymptotic_decay,
Expand Down Expand Up @@ -75,6 +76,7 @@
"TrainingReport",
"WeightInit",
"WeightInitStr",
"accelerated",
"asymptotic_decay",
"bubble",
"euclidean_distance",
Expand Down
17 changes: 16 additions & 1 deletion src/python_som/_accelerate.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@

from ._core._protocols import BmuKernel

__all__ = ["bmu_kernel"]
__all__ = ["accelerated", "bmu_kernel"]


@functools.cache
Expand Down Expand Up @@ -86,3 +86,18 @@ def fused_bmu( # pragma: no cover

kernel: BmuKernel = fused_bmu # pragma: no cover only when numba is installed
return kernel # pragma: no cover


def accelerated() -> bool:
"""Whether training will use the compiled kernel.

True once numba is installed; nothing else is needed to enable it, and results are identical
either way.

**Calling this imports numba**, which takes about 80 ms the first time. It does not compile the
kernel: ``njit`` is lazy, so the roughly 500 ms compile happens on the first call that actually
trains. Both are paid once per process.

:return: True if the compiled kernel is available.
"""
return bmu_kernel() is not None
2 changes: 1 addition & 1 deletion src/python_som/_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@

__all__ = ["__version__"]

__version__ = "0.7.0"
__version__ = "0.8.0"
45 changes: 39 additions & 6 deletions tests/test_core_boundary.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@
from __future__ import annotations

import ast
import importlib
import importlib.util
import pathlib
import pkgutil
import subprocess
import sys

import numpy as np
import pytest
Expand Down Expand Up @@ -259,11 +261,9 @@ def test_nothing_that_0_3_0_exported_has_been_removed() -> None:
def test_the_public_surface_is_exactly_this() -> None:
"""Pin the whole surface, so growing it is a decision rather than an accident.

0.4.0 adds the enums and their ``Literal`` counterparts, the strategy protocols, and the
artifact types. All are additive: every existing call keeps working, and the enum members are
``str`` subclasses that compare equal to the strings they replace. ``__version__`` is
deliberately absent -- it is re-exported with the ``as`` idiom, since ``__all__`` is the public
API and a dunder is not part of it.
Everything here is additive across 0.4.0 to 0.8.0 and every existing call keeps working.
``__version__`` is deliberately absent: it is re-exported with the ``as`` idiom, and ``__all__``
is the public API where a dunder is not.
"""
assert sorted(python_som.__all__) == [
"ArtifactError",
Expand All @@ -284,6 +284,7 @@ def test_the_public_surface_is_exactly_this() -> None:
"TrainingReport",
"WeightInit",
"WeightInitStr",
"accelerated",
"asymptotic_decay",
"bubble",
"euclidean_distance",
Expand Down Expand Up @@ -431,3 +432,35 @@ def test_error_messages_never_leak_a_private_name(kwargs: dict[str, object], exp
message = str(excinfo.value)
assert "_init_" not in message, message
assert "_som" not in message, message


def test_accelerated_is_false_without_numba() -> None:
"""The default install has no numba, so the predicate must say so.

The True direction is in ``tests/test_numba_kernel.py``, which only runs where numba is present.
Skipped rather than failed if numba happens to be installed here, so the two files cannot both
be wrong in the same direction.
"""
if importlib.util.find_spec("numba") is not None:
pytest.skip("numba is installed; the True direction is asserted elsewhere")
assert python_som.accelerated() is False


def test_asking_whether_it_is_accelerated_does_not_import_numba() -> None:
"""``accelerated()`` is public now, so it is a new way to accidentally import numba eagerly.

A subprocess, because this one may already have imported it.
"""
code = (
"import sys, python_som; "
"assert python_som.accelerated() in (True, False); "
"import importlib.util; "
"expected = importlib.util.find_spec('numba') is not None; "
"assert ('numba' in sys.modules) == expected, sys.modules.keys(); "
"print('clean')"
)
result = subprocess.run( # noqa: S603
[sys.executable, "-c", code], capture_output=True, text=True, check=False
)
assert result.returncode == 0, result.stdout + result.stderr
assert "clean" in result.stdout
24 changes: 24 additions & 0 deletions tests/test_numba_kernel.py
Original file line number Diff line number Diff line change
Expand Up @@ -177,3 +177,27 @@ def test_importing_the_package_does_not_import_numba() -> None:
)
assert result.returncode == 0, result.stdout + result.stderr
assert "clean" in result.stdout


def test_accelerated_reports_true() -> None:
"""The public predicate must agree with the kernel actually being there.

The False direction is in ``tests/test_core_boundary.py``, which runs in the environment without
numba. A predicate checked in only one direction would pass while always returning the same
answer.
"""
assert python_som.accelerated() is True


def test_asking_twice_does_not_resolve_twice() -> None:
"""``bmu_kernel`` is cached, so repeated calls cost nothing after the first.

Without the cache, every ``accelerated()`` call would redefine the jitted function.
"""
bmu_kernel()
before = bmu_kernel.cache_info()
python_som.accelerated()
python_som.accelerated()
after = bmu_kernel.cache_info()
assert after.misses == before.misses
assert after.hits >= before.hits + 2
36 changes: 36 additions & 0 deletions tests/test_packaging.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,3 +101,39 @@ def test_dev_dependencies_are_pinned_exactly(pyproject: dict[str, Any]) -> None:
for spec in pyproject["project"]["optional-dependencies"][group]:
requirement = spec.split(";")[0].strip()
assert "==" in requirement, f"{group} dependency is not pinned: {spec}"


def _readme_section(name: str) -> str:
"""Return one ``## `` section of the README, which is also the PyPI description.

:param name: Heading text, without the ``## ``.
:return: The section body.
"""
lines = (PROJECT_ROOT / "README.md").read_text(encoding="utf-8").splitlines()
start = next(n for n, line in enumerate(lines) if line.strip() == f"## {name}")
end = next((n for n in range(start + 1, len(lines)) if lines[n].startswith("## ")), len(lines))
return "\n".join(lines[start:end])
Comment on lines +106 to +115


def test_the_install_section_mentions_every_installable_option() -> None:
"""The README is the PyPI description, and a published description cannot be edited.

0.7.0 shipped the numba acceleration with its only user-facing mention in the feature list, so
someone reading Install to decide what to install never saw it. Correcting that took a release.
Asserting on the Install section rather than the whole description is the point: the string was
present in 0.7.0 and still in the wrong place.
"""
section = _readme_section("Install")
for option in (
"pip install python-som",
"[cli]",
"[sklearn]",
"[examples]",
"pip install numba",
):
assert option in section, f"the Install section does not mention {option!r}"


def test_the_install_section_warns_that_numba_constrains_numpy() -> None:
"""Installing numba downgrades NumPy, which a reader has to know before running the command."""
assert "numpy<2.5" in _readme_section("Install")
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading