Keras Model Quantization
This notebook demonstrates how to automatically convert a standard, baseline Keras model into an ultra-low-power, quantized QKeras model using a quantization dictionary layout (q_dict).
Prerequisites
Ensure that qkeras, tensorflow, and keras are installed. If you are executing this inside Google Colab, uncomment and run the following setup command:
# !pip install qkeras-v3 tensorflow
from keras.datasets import mnist
from keras.layers import *
from keras.models import Model
from qkeras.estimate import print_qstats
from qkeras.utils import model_quantize, quantized_model_dump
1. Defining the Floating-Point Floating-Point Keras Model
We begin by establishing a multi-input Functional API model architecture. This functional model accepts two separate $(28, 28, 1)$ inputs, concatenates them, and passes them down a chain of standard Conv2D, Activation, MaxPooling2D, and Dense layers.
x0 = x_in0 = Input((28, 28, 1), name="input0")
x1 = x_in1 = Input((28, 28, 1), name="input1")
x = Concatenate(name="concat")([x0, x1])
x = Conv2D(128, (3, 3), strides=1, name="conv2d_0_m")(x)
x = Activation("relu", name="act0_m")(x)
x = MaxPooling2D(2, 2, name="mp_0")(x)
x = Conv2D(256, (3, 3), strides=1, name="conv2d_1_m")(x)
x = Activation("relu", name="act1_m")(x)
x = MaxPooling2D(2, 2, name="mp_1")(x)
x = Conv2D(128, (3, 3), strides=1, name="conv2d_2_m")(x)
x = Activation("relu", name="act2_m")(x)
x = MaxPooling2D(2, 2, name="mp_2")(x)
x = Flatten()(x)
x = Dense(10, name="dense")(x)
x = Activation("softmax", name="softmax")(x)
model = Model(inputs=[x_in0, x_in1], outputs=[x])
model.summary()
Model: "functional"
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ Connected to ┃ ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩ │ input0 (InputLayer) │ (None, 28, 28, 1) │ 0 │ - │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ input1 (InputLayer) │ (None, 28, 28, 1) │ 0 │ - │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ concat │ (None, 28, 28, 2) │ 0 │ input0[0][0], │ │ (Concatenate) │ │ │ input1[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ conv2d_0_m (Conv2D) │ (None, 26, 26, │ 2,432 │ concat[0][0] │ │ │ 128) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ act0_m (Activation) │ (None, 26, 26, │ 0 │ conv2d_0_m[0][0] │ │ │ 128) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ mp_0 (MaxPooling2D) │ (None, 13, 13, │ 0 │ act0_m[0][0] │ │ │ 128) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ conv2d_1_m (Conv2D) │ (None, 11, 11, │ 295,168 │ mp_0[0][0] │ │ │ 256) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ act1_m (Activation) │ (None, 11, 11, │ 0 │ conv2d_1_m[0][0] │ │ │ 256) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ mp_1 (MaxPooling2D) │ (None, 5, 5, 256) │ 0 │ act1_m[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ conv2d_2_m (Conv2D) │ (None, 3, 3, 128) │ 295,040 │ mp_1[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ act2_m (Activation) │ (None, 3, 3, 128) │ 0 │ conv2d_2_m[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ mp_2 (MaxPooling2D) │ (None, 1, 1, 128) │ 0 │ act2_m[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ flatten (Flatten) │ (None, 128) │ 0 │ mp_2[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ dense (Dense) │ (None, 10) │ 1,290 │ flatten[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ softmax │ (None, 10) │ 0 │ dense[0][0] │ │ (Activation) │ │ │ │ └─────────────────────┴───────────────────┴────────────┴───────────────────┘
Total params: 593,930 (2.27 MB)
Trainable params: 593,930 (2.27 MB)
Non-trainable params: 0 (0.00 B)
2. Setting Up the Quantization Configuration Dictionary
The dictionary q_dict provides precise target assignments for the quantization conversion:
Specific layer rules: Targets layers directly by name string keys (e.g.,
conv2d_0_m,act2_m).Global structural layer rules: Directs fallback target behavior for layer type categories (e.g., configurations for all instances of
QConv2D,QDense, orQActivation).
q_dict = {
"conv2d_0_m": {
"kernel_quantizer": "binary()",
"bias_quantizer": "quantized_bits(4,0,1)",
},
"conv2d_1_m": {
"kernel_quantizer": "ternary()",
"bias_quantizer": "quantized_bits(4,0,1)",
},
"act2_m": "quantized_relu(6,2)",
"QActivation": {"relu": "quantized_relu(4,0)"},
"QConv2D": {
"kernel_quantizer": "quantized_bits(4,0,1)",
"bias_quantizer": "quantized_bits(4,0,1)",
},
"QDense": {
"kernel_quantizer": "quantized_bits(3,0,1)",
"bias_quantizer": "quantized_bits(3,0,1)",
},
}
print("Quantization configuration map defined.")
Quantization configuration map defined.
3. Running Automatic Model Quantization Conversion
Using model_quantize, QKeras clones the architecture layout and replaces standard floating-point layers with equivalent QKeras layers (QConv2D, QDense, etc.) preconfigured with our precision bit definitions.
qmodel = model_quantize(model, q_dict, 4)
qmodel.summary()
Model: "functional"
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ Connected to ┃ ┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩ │ input0 (InputLayer) │ (None, 28, 28, 1) │ 0 │ - │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ input1 (InputLayer) │ (None, 28, 28, 1) │ 0 │ - │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ concat │ (None, 28, 28, 2) │ 0 │ input0[0][0], │ │ (Concatenate) │ │ │ input1[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ conv2d_0_m │ (None, 26, 26, │ 2,432 │ concat[0][0] │ │ (QConv2D) │ 128) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ act0_m │ (None, 26, 26, │ 0 │ conv2d_0_m[0][0] │ │ (QActivation) │ 128) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ mp_0 (MaxPooling2D) │ (None, 13, 13, │ 0 │ act0_m[0][0] │ │ │ 128) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ conv2d_1_m │ (None, 11, 11, │ 295,168 │ mp_0[0][0] │ │ (QConv2D) │ 256) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ act1_m │ (None, 11, 11, │ 0 │ conv2d_1_m[0][0] │ │ (QActivation) │ 256) │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ mp_1 (MaxPooling2D) │ (None, 5, 5, 256) │ 0 │ act1_m[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ conv2d_2_m │ (None, 3, 3, 128) │ 295,040 │ mp_1[0][0] │ │ (QConv2D) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ act2_m │ (None, 3, 3, 128) │ 0 │ conv2d_2_m[0][0] │ │ (QActivation) │ │ │ │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ mp_2 (MaxPooling2D) │ (None, 1, 1, 128) │ 0 │ act2_m[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ flatten (Flatten) │ (None, 128) │ 0 │ mp_2[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ dense (QDense) │ (None, 10) │ 1,290 │ flatten[0][0] │ ├─────────────────────┼───────────────────┼────────────┼───────────────────┤ │ softmax │ (None, 10) │ 0 │ dense[0][0] │ │ (Activation) │ │ │ │ └─────────────────────┴───────────────────┴────────────┴───────────────────┘
Total params: 593,930 (2.27 MB)
Trainable params: 593,930 (2.27 MB)
Non-trainable params: 0 (0.00 B)
4. Extracting Quantization Size and Profiling Statistics
Executing print_qstats prints a localized hardware profile dashboard breakdown indicating the allocated parameter bits, operation profiles, and precision footprint characteristics of the newly compiled model.
print_qstats(qmodel)
Number of operations in model:
conv2d_0_m : 1557504 (smux_1_8)
conv2d_1_m : 35684352 (smux_2_4)
conv2d_2_m : 2654208 (smult_4_4)
dense : 1280 (smult_3_6)
Number of operation types in model:
smult_3_6 : 1280
smult_4_4 : 2654208
smux_1_8 : 1557504
smux_2_4 : 35684352
Weight profiling:
conv2d_0_m_weights : 2304 (1-bit unit)
conv2d_0_m_bias : 128 (4-bit unit)
conv2d_1_m_weights : 294912 (2-bit unit)
conv2d_1_m_bias : 256 (4-bit unit)
conv2d_2_m_weights : 294912 (4-bit unit)
conv2d_2_m_bias : 128 (4-bit unit)
dense_weights : tf.Tensor(1280, shape=(), dtype=int32) (3-bit unit)
dense_bias : 10 (3-bit unit)
----------------------------------------
Total Bits : 1777694
Weight sparsity:
... quantizing model
conv2d_0_m : 0.0526
conv2d_1_m : 0.3756
conv2d_2_m : 0.0947
dense : 0.1643
----------------------------------------
Total Sparsity : 0.2343
5. Fetching Evaluation Samples and Layer Output Dumping
Finally, we extract a small batch of MNIST test frames and employ quantized_model_dump to generate a hardware-reproducible trace profile file. This dumps the exact fixed-point weight arrays and intermediate feature map outputs across target network layers.
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# Construct multi-input test split block match for the input layer structure
x_test_arr = [x_test[0:10, :], x_test[0:10, :]]
quantized_model_dump(
qmodel,
x_test_arr,
layers_to_dump=["input0", "input1", "act2_m", "act1_m", "act0_m"],
)
print("Layer weights and feature mapping traces dumped successfully.")
temp dir /var/folders/sd/hnbd9dm54xl_zh0fctx18ls40000gn/T/tmpbt9cyvb1
create dir /var/folders/sd/hnbd9dm54xl_zh0fctx18ls40000gn/T/tmpbt9cyvb1
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 391ms/step
writing the layer output tensor to /var/folders/sd/hnbd9dm54xl_zh0fctx18ls40000gn/T/tmpbt9cyvb1/input0.bin
writing the layer output tensor to /var/folders/sd/hnbd9dm54xl_zh0fctx18ls40000gn/T/tmpbt9cyvb1/input1.bin
writing the layer output tensor to /var/folders/sd/hnbd9dm54xl_zh0fctx18ls40000gn/T/tmpbt9cyvb1/act0_m.bin
writing the layer output tensor to /var/folders/sd/hnbd9dm54xl_zh0fctx18ls40000gn/T/tmpbt9cyvb1/act1_m.bin
writing the layer output tensor to /var/folders/sd/hnbd9dm54xl_zh0fctx18ls40000gn/T/tmpbt9cyvb1/act2_m.bin
Layer weights and feature mapping traces dumped successfully.