v0.2.0 — refresh-aware kernel learning · release notes →

Adaptive kernels,
refreshed when they drift.

Kernos is a Python library for refresh-aware hybrid continuous-discrete low-rank kernel learning. Continuous parameters learn by gradient descent. Discrete landmarks refresh only when the basis drifts — so you scale to large-batch and streaming regression without rebuilding the world.

Install kernos View on GitHub

scikit-learn compatible

PSD kernel guarantee

Streaming & cached modes

MIT licensed

quickstart.py

python 3.12

# refresh-aware hybrid kernel regression
import numpy as np
from kernos import Kernos

rng = np.random.default_rng(42)
X_train = rng.standard_normal((4000, 8))
y_train = X_train[:, 0] + 0.5 * X_train[:, 1] ** 2 + 0.1 * rng.standard_normal(4000)
X_test  = rng.standard_normal((1000, 8))

model = Kernos(
    dim=64, mbasis=256, abasis=32,
    ridge=1e-2, steps=500, seed=42,
).fit(X_train, y_train)

print(f"R² on held-out: "
      f"{model.score(X_test, y_test):.4f}")
# → R² on held-out: 0.8923

What happens inside

Global Nyström basis selected via k-means++ and whitened.
Continuous θ, R updated every step with gradient descent.
Drift-aware refresh only when the basis drifts past a threshold.
Logistic gate fuses global and local features.

Built on the shoulders of

NumPy SciPy scikit-learn HuggingFace PyPI pytest

Features

Kernel learning, without the rebuild.

Every piece of Kernos is engineered for one thing: an adaptive, low-rank kernel that learns continuously and refreshes only when it has to.

Refresh-aware basis adaptation

Landmarks, anchors, calibration, and fusion refresh only when drift exceeds a threshold — with cooldown, warmup, hysteresis, and an amortized budget.

Drop-in for scikit-learn

A familiar `fit` / `predict` / `score` surface. Plays well with `GridSearchCV`, `Pipeline`, and the rest of the sklearn ecosystem.

Mathematical guarantees

An explicit-feature kernel with PSD guarantee, a rank bound, and SPD normal equations. Calibration scalars stay bounded away from zero.

Two memory modes

Cached `O(nm)` for in-core work and streamed `O(m²)` for large data and online learning — same algorithm, different memory budget.

Residual-aware local corrections

Anchor sampling weighted by residual magnitude, k-NN sparse RBF features, and orthogonalization that keeps local features in the global nullspace.

Numerically stabilized

Eigenvalue clipping, soft spectral truncation, Cholesky with jitter fallback, and preconditioned conjugate gradient for ill-conditioned solves.

Architecture

Two parameter groups, one disciplined loop.

Kernos splits every parameter into a continuously-learned group and a discretely-refreshed group. The expensive refresh fires only when the basis drifts past a threshold you control.

The hybrid loop

Continuous descent. Discrete refresh.

Each step, the continuous parameters learn by gradient descent. Periodically, a drift-aware controller decides whether to refresh the discrete basis — gated by cooldown, warmup, hysteresis, and an amortized refresh budget.

∂

Continuous θ, R every step

Updated via gradient descent on the validation loss.

Δ

Discrete Z, A, M_g, c_g, c_l, ρ on drift

Refreshed only when basis drift exceeds a learned threshold.

Drift

frobenius · spectral

Trigger

d_t > drift_hi

Cooldown

cool = 50

Budget

amortized

Pipeline

Five stages. One coherent model.

01

Embed Φ

Project input space into a continuous embedding.
02

Basis Z, M_g

Global Nyström landmarks via k-means++ and spectral whitening.
03

Correct A, M_l

Residual-aware anchors + orthogonalized local features.
04

Fuse c_g, c_l, ρ

Trace-based calibration + logistic gate across global and local.
05

Solve w

Direct, iterative, or Jacobi ridge solve on ΦᵀΦ + λI.

rank(K) ≤

r_g + m_l

Explicit rank bound: global Nyström landmarks plus local corrective rank.

kernel PSD

K = ΦΦᵀ

Constructed as a product of explicit features. Positive semidefinite by construction.

orthogonality

Φ_gᵀ Φ_l ≈ 0

Local features live in the global nullspace, kept stable by ridge regularization.

Benchmarks

Measured against the best of the rest.

Kernos was evaluated against Ridge, Nyström, and Random Fourier Features on nine real-world regression datasets. The refresh-aware variant beats the best non-refresh baseline on average — and wins more often than it loses.

Datasets evaluated

9

real-world

Mean ΔRMSE vs best baseline

−7.99

% improvement

Win rate vs Nyström

8 / 9

across datasets

Win rate vs RFF

8 / 9

across datasets

Pareto plots — RMSE vs train time

Kernos dominates the lower-left frontier.

Kernos Baselines

Pareto plot for WineQuality — WineQuality RMSE ↓ vs time

Pareto plot for California Housing — California Housing RMSE ↓ vs time

Pareto plot for YearPredictionMSD — YearPredictionMSD RMSE ↓ vs time

Pareto plot for Kin8nm — Kin8nm RMSE ↓ vs time

Ablation impact

The refresh mechanism is doing the work.

Per-dataset ranking

Competitive on every benchmark.

Abalone

#6 / 11
CaliforniaHousing

#3 / 11
CPUActivity

#2 / 11
Elevators

#5 / 11
HouseSales

#3 / 11
Kin8nm

#4 / 11
Superconduct

#3 / 11
WineQuality

#3 / 11
YearPredictionMSD

#3 / 11

Lower rank is better. Across nine datasets, Kernos lands in the top half of eleven models on every one.

Reproduce locally:


kernos-eval --datasets WineQuality --tiers Small --n_seeds 2


kernos-analyze --results results/results.csv

API

Familiar surface.
Sophisticated internals.

Use Kernos exactly like an sklearn estimator. Tune it with GridSearchCV. Stream it with partial_fit. Drop it into a Pipeline.

kernos.Kernos — the estimator class.
kernos.Buffer — FULL or STREAM memory modes.
kernos-eval — CLI for the full evaluation suite.
kernos-analyze — produce plots, CSVs, LaTeX tables.

Read the docs Install kernos

python 3.10+

import numpy as np
from kernos import Kernos

rng = np.random.default_rng(42)
X_train = rng.standard_normal((4000, 8))
y_train = X_train[:, 0] + 0.5 * X_train[:, 1] ** 2 + 0.1 * rng.standard_normal(4000)
X_test  = rng.standard_normal((1000, 8))

model = Kernos(
    dim=64, mbasis=256, abasis=32,
    ridge=1e-2, steps=500, seed=42,
).fit(X_train, y_train)

print(fclass="hl-s">"R² on held-out: {model.score(X_test, y_test):.4f}")
class=class="hl-s">"hl-c"># → R² on held-out: 0.8923

from sklearn.model_selection import GridSearchCV
from kernos import Kernos

param_grid = {
    class="hl-s">"mbasis": [64, 128, 256],
    class="hl-s">"abasis": [16, 32, 64],
    class="hl-s">"ridge":  [1e-3, 1e-2, 1e-1],
}

search = GridSearchCV(
    Kernos(seed=42, steps=300),
    param_grid,
    cv=5,
    scoring=class="hl-s">"r2",
    n_jobs=-1,
)
search.fit(X_train, y_train)
print(search.best_params_, search.best_score_)

from kernos import Kernos, Buffer

class=class="hl-s">"hl-c"># O(m²) memory mode — same algorithm, no full ΦᵀΦ cache.
model = Kernos(
    mode=Buffer.STREAM,
    dim=32, mbasis=128, abasis=16,
    steps=2000, seed=0,
)

class=class="hl-s">"hl-c"># partial_fit drives the continuous descent; the refresh
class=class="hl-s">"hl-c"># controller still triggers when the basis drifts.
for x_batch, y_batch in stream:
    model.partial_fit(x_batch, y_batch)

print(model.predict(X_test))

Get started

Install with one command.

Kernos ships on PyPI. Python 3.10 or newer, NumPy, SciPy, and scikit-learn — that's it.

pip install kernos — production install.
pip install -e ".[dev,examples]" — for evaluation suite and plots.
MIT licensed — commercial and academic use.

Source on GitHub pip install kernos

bash

$ pip install kernos

Successfully installed kernos-0.2.0

$ python -c "import kernos; print(kernos.__version__)"

0.2.0

$ kernos-eval --datasets WineQuality --tiers Small

→ results/pareto_WineQuality.png

Tested on Python 3.10, 3.11, 3.12

Linux · macOS · Windows

Adaptive kernels, refreshed when they drift.