Quantized CNN
This notebook introduces QKerasV3, the Keras 3-compatible fork of QKeras. The package is installed as qkeras-v3, but the Python import namespace remains qkeras.
You will train a small floating-point CNN on MNIST, rewrite it as a quantized QKeras model, inspect the quantizers, automatically quantize a float model, and check model serialization.
Keras 3 note: use the standalone
keraspackage (import keras) and avoid mixing it withtf.kerasortf_kerasin the same notebook.
# Optional install for a fresh environment. Uncomment when needed.
# %pip install keras tensorflow qkeras-v3
1. Imports and reproducibility
The tutorial uses explicit imports instead of from keras.layers import *. This makes it clearer which symbols come from Keras and which come from QKeras.
import os
# Set this before importing Keras if you want a specific Keras 3 backend.
# TensorFlow is the most common backend for QKerasV3 workflows today.
os.environ.setdefault("KERAS_BACKEND", "tensorflow")
import keras
import numpy as np
from keras import layers
from keras.datasets import mnist
from keras.utils import to_categorical
from qkeras import QActivation, QConv2D, QDense, print_qstats
from qkeras.utils import load_qmodel, model_quantize, quantized_model_debug
keras.utils.set_random_seed(812)
print("Keras:", keras.__version__)
print("Backend:", keras.backend.backend())
/Users/mariuskoppel/cms/qkeras/venv/lib/python3.11/site-packages/keras/src/export/tf2onnx_lib.py:8: FutureWarning: In the future `np.object` will be defined as the corresponding NumPy scalar.
if not hasattr(np, "object"):
Keras: 3.13.0
Backend: tensorflow
2. Load and preprocess MNIST
For a quick tutorial run, the training set is limited by default. Increase train_limit or set it to None for a full training run.
def get_data(train_limit=20_000, test_limit=5_000):
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.astype("float32") / 255.0
x_test = x_test.astype("float32") / 255.0
# Add channel dimension: (N, 28, 28) -> (N, 28, 28, 1)
x_train = np.expand_dims(x_train, axis=-1)
x_test = np.expand_dims(x_test, axis=-1)
if train_limit is not None:
x_train = x_train[:train_limit]
y_train = y_train[:train_limit]
if test_limit is not None:
x_test = x_test[:test_limit]
y_test = y_test[:test_limit]
num_classes = int(np.max(y_train)) + 1
y_train = to_categorical(y_train, num_classes)
y_test = to_categorical(y_test, num_classes)
return (x_train, y_train), (x_test, y_test), num_classes
(x_train, y_train), (x_test, y_test), num_classes = get_data()
input_shape = x_train.shape[1:]
print("input_shape:", input_shape)
print("num_classes:", num_classes)
input_shape: (28, 28, 1)
num_classes: 10
3. Baseline floating-point model
This is the model we will quantize. Layer names are intentional: model_quantize can target layers by name or by layer class.
def create_float_model(input_shape, num_classes):
inputs = keras.Input(shape=input_shape, name="input")
x = layers.Conv2D(18, (3, 3), padding="same", name="conv2d_1")(inputs)
x = layers.Activation("relu", name="act_1")(x)
x = layers.Conv2D(32, (3, 3), padding="same", name="conv2d_2")(x)
x = layers.Activation("relu", name="act_2")(x)
x = layers.Flatten(name="flatten")(x)
x = layers.Dense(num_classes, name="dense")(x)
outputs = layers.Activation("softmax", name="softmax")(x)
return keras.Model(inputs=inputs, outputs=outputs, name="float_mnist_cnn")
float_model = create_float_model(input_shape, num_classes)
float_model.summary()
Model: "float_mnist_cnn"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ input (InputLayer) │ (None, 28, 28, 1) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_1 (Conv2D) │ (None, 28, 28, 18) │ 180 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ act_1 (Activation) │ (None, 28, 28, 18) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_2 (Conv2D) │ (None, 28, 28, 32) │ 5,216 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ act_2 (Activation) │ (None, 28, 28, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ flatten (Flatten) │ (None, 25088) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense (Dense) │ (None, 10) │ 250,890 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ softmax (Activation) │ (None, 10) │ 0 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 256,286 (1001.12 KB)
Trainable params: 256,286 (1001.12 KB)
Non-trainable params: 0 (0.00 B)
float_model.compile(
optimizer=keras.optimizers.Adam(learning_rate=1e-3),
loss="categorical_crossentropy",
metrics=["accuracy"],
)
float_history = float_model.fit(
x_train,
y_train,
epochs=2,
batch_size=128,
validation_data=(x_test, y_test),
verbose=2,
)
Epoch 1/2
157/157 - 4s - 27ms/step - accuracy: 0.8828 - loss: 0.4133 - val_accuracy: 0.9400 - val_loss: 0.2009
Epoch 2/2
157/157 - 4s - 23ms/step - accuracy: 0.9684 - loss: 0.1073 - val_accuracy: 0.9582 - val_loss: 0.1280
4. Manually build a quantized model
QKeras layers are drop-in replacements for Keras layers that create weights, such as Dense and Conv2D. Activations can be quantized with QActivation.
A common pattern is:
replace
Conv2DwithQConv2Dreplace
DensewithQDensereplace intermediate activations with
QActivationkeep the final
softmaxas a regular Keras activation when inference will useargmax
def create_qmodel(input_shape, num_classes):
inputs = keras.Input(shape=input_shape, name="input")
x = QConv2D(
18,
(3, 3),
padding="same",
kernel_quantizer="stochastic_ternary",
bias_quantizer="quantized_po2(4)",
name="conv2d_1",
)(inputs)
x = QActivation("quantized_relu(2)", name="act_1")(x)
x = QConv2D(
32,
(3, 3),
padding="same",
kernel_quantizer="stochastic_ternary",
bias_quantizer="quantized_po2(4)",
name="conv2d_2",
)(x)
x = QActivation("quantized_relu(3)", name="act_2")(x)
x = layers.Flatten(name="flatten")(x)
x = QDense(
num_classes,
kernel_quantizer="quantized_bits(4, 0, 1)",
bias_quantizer="quantized_bits(4)",
name="dense",
)(x)
outputs = layers.Activation("softmax", name="softmax")(x)
return keras.Model(inputs=inputs, outputs=outputs, name="qkeras_mnist_cnn")
qmodel = create_qmodel(input_shape, num_classes)
qmodel.summary()
Model: "qkeras_mnist_cnn"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ input (InputLayer) │ (None, 28, 28, 1) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_1 (QConv2D) │ (None, 28, 28, 18) │ 180 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ act_1 (QActivation) │ (None, 28, 28, 18) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_2 (QConv2D) │ (None, 28, 28, 32) │ 5,216 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ act_2 (QActivation) │ (None, 28, 28, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ flatten (Flatten) │ (None, 25088) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense (QDense) │ (None, 10) │ 250,890 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ softmax (Activation) │ (None, 10) │ 0 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 256,286 (1001.12 KB)
Trainable params: 256,286 (1001.12 KB)
Non-trainable params: 0 (0.00 B)
qmodel.compile(
optimizer=keras.optimizers.Adam(learning_rate=5e-4),
loss="categorical_crossentropy",
metrics=["accuracy"],
)
q_history = qmodel.fit(
x_train,
y_train,
epochs=3,
batch_size=128,
validation_data=(x_test, y_test),
verbose=2,
)
Epoch 1/3
157/157 - 10s - 67ms/step - accuracy: 0.8867 - loss: 0.4002 - val_accuracy: 0.9262 - val_loss: 0.2426
Epoch 2/3
157/157 - 10s - 67ms/step - accuracy: 0.9604 - loss: 0.1390 - val_accuracy: 0.9520 - val_loss: 0.1595
Epoch 3/3
157/157 - 11s - 69ms/step - accuracy: 0.9753 - loss: 0.0865 - val_accuracy: 0.9578 - val_loss: 0.1441
5. Inspect quantizers and model statistics
print_qstats gives a compact summary of quantized weights and activations.
def describe_quantizers(model):
for layer in model.layers:
if hasattr(layer, "kernel_quantizer_internal"):
print(
f"{layer.name:12s}",
"kernel=", layer.kernel_quantizer_internal,
"bias=", getattr(layer, "bias_quantizer_internal", None),
)
elif hasattr(layer, "quantizer"):
print(f"{layer.name:12s}", "activation=", layer.quantizer)
describe_quantizers(qmodel)
print()
print_qstats(qmodel)
conv2d_1 kernel= stochastic_ternary(alpha='auto_po2') bias= quantized_po2(4)
conv2d_2 kernel= stochastic_ternary(alpha='auto_po2') bias= quantized_po2(4)
dense kernel= quantized_bits(4,0,1,alpha='auto_po2') bias= quantized_bits(4,0,0)
Number of operations in model:
conv2d_1 : 127008 (smux_2_8)
conv2d_2 : 4064256 (smux_2_2)
dense : 250880 (smult_4_3)
Number of operation types in model:
smult_4_3 : 250880
smux_2_2 : 4064256
smux_2_8 : 127008
Weight profiling:
conv2d_1_weights : 162 (2-bit unit)
conv2d_1_bias : 18 (4-bit unit)
conv2d_2_weights : 5184 (2-bit unit)
conv2d_2_bias : 32 (4-bit unit)
dense_weights : tf.Tensor(250880, shape=(), dtype=int32) (4-bit unit)
dense_bias : 10 (4-bit unit)
----------------------------------------
Total Bits : 1014452
Weight sparsity:
... quantizing model
conv2d_1 : 0.3056
conv2d_2 : 0.5303
dense : 0.3652
----------------------------------------
Total Sparsity : 0.3686
6. Automatically quantize a Keras model
model_quantize converts supported Keras layers into QKeras layers. The configuration can target either specific layer names or layer classes.
The example below intentionally sets conv2d_1 differently from the default QConv2D rule to show the precedence of layer-specific configuration.
quantizer_config = {
"conv2d_1": {
"kernel_quantizer": "stochastic_binary",
"bias_quantizer": "quantized_po2(4)",
},
"QConv2D": {
"kernel_quantizer": "stochastic_ternary",
"bias_quantizer": "quantized_po2(4)",
},
"QDense": {
"kernel_quantizer": "quantized_bits(4, 0, 1)",
"bias_quantizer": "quantized_bits(4)",
},
"QActivation": {
"relu": "quantized_relu(3)",
},
"act_1": "quantized_relu(2)",
}
auto_qmodel = model_quantize(
float_model,
quantizer_config,
activation_bits=4,
transfer_weights=True,
)
auto_qmodel.summary()
describe_quantizers(auto_qmodel)
Model: "float_mnist_cnn"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩ │ input (InputLayer) │ (None, 28, 28, 1) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_1 (QConv2D) │ (None, 28, 28, 18) │ 180 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ act_1 (QActivation) │ (None, 28, 28, 18) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ conv2d_2 (QConv2D) │ (None, 28, 28, 32) │ 5,216 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ act_2 (QActivation) │ (None, 28, 28, 32) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ flatten (Flatten) │ (None, 25088) │ 0 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ dense (QDense) │ (None, 10) │ 250,890 │ ├─────────────────────────────────┼────────────────────────┼───────────────┤ │ softmax (Activation) │ (None, 10) │ 0 │ └─────────────────────────────────┴────────────────────────┴───────────────┘
Total params: 768,860 (2.93 MB)
Trainable params: 256,286 (1001.12 KB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 512,574 (1.96 MB)
conv2d_1 kernel= stochastic_binary(alpha='auto_po2') bias= quantized_po2(4)
conv2d_2 kernel= stochastic_ternary(alpha='auto_po2') bias= quantized_po2(4)
dense kernel= quantized_bits(4,0,1,alpha='auto_po2') bias= quantized_bits(4,0,0)
auto_qmodel.compile(
optimizer=keras.optimizers.Adam(learning_rate=5e-4),
loss="categorical_crossentropy",
metrics=["accuracy"],
)
auto_history = auto_qmodel.fit(
x_train,
y_train,
epochs=3,
batch_size=128,
validation_data=(x_test, y_test),
verbose=2,
)
Epoch 1/3
157/157 - 10s - 66ms/step - accuracy: 0.9688 - loss: 0.1062 - val_accuracy: 0.9586 - val_loss: 0.1399
Epoch 2/3
157/157 - 9s - 57ms/step - accuracy: 0.9783 - loss: 0.0768 - val_accuracy: 0.9642 - val_loss: 0.1161
Epoch 3/3
157/157 - 9s - 58ms/step - accuracy: 0.9812 - loss: 0.0639 - val_accuracy: 0.9644 - val_loss: 0.1201
7. Debug numeric ranges
quantized_model_debug reports observed activation, weight, and bias ranges. This is useful when choosing integer bits for fixed-point quantizers.
# Use a small sample to keep the tutorial fast.
quantized_model_debug(auto_qmodel, x_test[:256], plot=False)
8/8 ━━━━━━━━━━━━━━━━━━━━ 0s 15ms/step
input 0.0000 1.0000
conv2d_1 -0.7267 0.8517 ( -1.0000 1.0000) ( -0.0625 0.0625)
act_1 0.0000 0.7500
conv2d_2 -1.5000 1.6875 ( -1.0000 1.0000) ( -0.0625 0.0625)
act_2 0.0000 0.8750
dense -25.5781 15.0664 ( -0.8750 0.7500) ( 0.0000 0.0000)
8. Keras 3 serialization check
For QKerasV3, a good tutorial should demonstrate that the model can be saved and loaded through the qkeras helper. This catches many custom-object and config compatibility problems early.
import tempfile
from pathlib import Path
with tempfile.TemporaryDirectory() as tmpdir:
path = Path(tmpdir) / "qkeras_model.keras"
auto_qmodel.save(path)
reloaded = load_qmodel(path)
original_pred = auto_qmodel.predict(x_test[:32], verbose=0)
reloaded_pred = reloaded.predict(x_test[:32], verbose=0)
np.testing.assert_allclose(original_pred, reloaded_pred, rtol=1e-6, atol=1e-6)
print("Reloaded model predictions match.")
Reloaded model predictions match.
9. Practical guidance
Prefer string quantizers such as
"quantized_bits(4, 0, 1)"in tutorials and configs. They serialize well and make notebooks easier to read.Use explicit layer names when you plan to use
model_quantize.Quantize intermediate activations, but often keep the final
softmaxfloating point for training and useargmaxduring deployment.Start with moderate activation precision, then reduce bits after checking accuracy and numeric ranges.
In CI and notebooks, print
keras.__version__,keras.backend.backend(), and the installed QKerasV3 version to avoid confusing standalone Keras withtf.keras/tf_keras.