TensorFlow Interview Questions

A tensor is TensorFlow's core data structure: an immutable, n-dimensional array with a uniform data type (dtype) and a shape. Scalars are rank-0 tensors, vectors are rank-1, matrices are rank-2, and so on.

Key differences from a NumPy array:

• Tensors are immutable: operations return new tensors rather than mutating in place. Use tf.Variable when you need mutable state.
• Tensors can live on a GPU or TPU, not just host memory.
• Tensors integrate with automatic differentiation and graph tracing via tf.function.

import tensorflow as tf
import numpy as np

t = tf.constant([[1, 2], [3, 4]], dtype=tf.float32)
print(t.shape)   # (2, 2)
print(t.dtype)   # <dtype: 'float32'>
print(t.ndim)    # 2

# Interop is zero-copy on CPU
arr = t.numpy()          # tensor -> ndarray
back = tf.convert_to_tensor(arr)  # ndarray -> tensor

Interop note: Many TensorFlow ops accept NumPy arrays directly and NumPy functions accept tensors, because tensors implement the array interface. The conversion shares memory on CPU when possible, so it is cheap but not free on GPU (data must be copied to host).

Eager execution (the TensorFlow 2.x default) runs operations immediately, like normal Python, which makes debugging with prints and breakpoints easy. tf.function traces your Python function once and compiles it into a static dataflow graph that runs faster and can be serialized, distributed, and deployed.

import tensorflow as tf

@tf.function
def add(a, b):
    print("tracing")      # prints only during tracing
    return a + b

add(tf.constant(1), tf.constant(2))  # prints "tracing"
add(tf.constant(3), tf.constant(4))  # no print, reuses graph

Gotcha: Python side effects (printing, appending to lists, mutating globals) execute only during tracing, not on later calls. Use tf.print for graph-time printing and keep functions pure with respect to Python state.

A tf.function creates a separate ConcreteFunction (graph) for each distinct input signature, defined by the dtype and shape of tensor arguments and the identity of Python arguments. Every new signature triggers a retrace, which is expensive and, in loops, can silently destroy performance.

Common causes of excessive retracing:

• Passing Python scalars (like 1, 2, 3) instead of tensors, since each value is a new signature.
• Feeding tensors with varying shapes (variable-length batches or sequences).
• Passing new Python objects each call.

import tensorflow as tf

# Bad: Python ints cause a retrace per value
@tf.function
def f(x):
    return x * 2

for i in range(3):
    f(i)   # 3 traces

# Fix 1: pass tensors, not Python scalars
for i in range(3):
    f(tf.constant(i))   # 1 trace

# Fix 2: fix the signature for variable shapes
@tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.float32)])
def g(x):
    return tf.reduce_sum(x)

Diagnostics: enable tf.data.experimental.enable_debug_mode() or set @tf.function(experimental_get_tracing_count) style checks, and watch for repeated tracing warnings. An input_signature with None dimensions makes the graph shape-polymorphic so variable batch sizes reuse one graph.

Practical rule: keep the number of distinct signatures small and bounded; if you see retracing every iteration, your loop is running in eager-equivalent cost with graph overhead on top.

tf.constant creates an immutable tensor whose value is fixed at creation. tf.Variable creates a mutable tensor whose value can be updated in place with assign, assign_add, and similar methods. Trainable model parameters (weights, biases) are always variables.

import tensorflow as tf

c = tf.constant([1.0, 2.0])
# c.assign(...) would fail: constants are immutable

w = tf.Variable([1.0, 2.0])
w.assign([3.0, 4.0])       # replace value
w.assign_add([1.0, 1.0])   # in-place add -> [4.0, 5.0]
print(w.trainable)         # True by default

Key points:

• Variables are tracked automatically by GradientTape for gradient computation.
• Set trainable=False for state you update manually but do not optimize (for example, batch-norm moving averages or step counters).
• Constants get baked into the graph, so large constants bloat serialized models; prefer variables or inputs for big data.

tf.GradientTape records operations onto a tape during the forward pass, then applies reverse-mode automatic differentiation to compute gradients of a target with respect to sources. By default it watches all trainable tf.Variable objects touched inside the context.

import tensorflow as tf

x = tf.Variable(3.0)
with tf.GradientTape() as tape:
    y = x ** 2 + 2 * x        # y = x^2 + 2x
dy_dx = tape.gradient(y, x)    # dy/dx = 2x + 2 = 8.0
print(dy_dx.numpy())           # 8.0

Important behaviors:

• The tape is consumed after one gradient call. Use persistent=True to compute multiple gradients.
• Constants are not watched by default; call tape.watch(t) to differentiate with respect to a non-variable tensor.
• Only ops executed inside the with block are recorded.

x = tf.constant(3.0)
with tf.GradientTape(persistent=True) as tape:
    tape.watch(x)
    y = x ** 2
    z = x ** 3
print(tape.gradient(y, x).numpy())  # 6.0
print(tape.gradient(z, x).numpy())  # 27.0
del tape

Higher-order gradients come from nesting tapes: an outer tape differentiates the gradient produced by an inner tape.

Broadcasting lets ops combine tensors of different shapes without copying data, by virtually stretching dimensions. TensorFlow follows the same rules as NumPy: align shapes from the trailing (rightmost) dimension and, for each dimension, they are compatible if they are equal or one of them is 1.

import tensorflow as tf

a = tf.constant([[1], [2], [3]])   # shape (3, 1)
b = tf.constant([10, 20, 30])      # shape (3,)
print((a + b).shape)               # (3, 3)

Here a is (3, 1) and b is treated as (1, 3); both stretch to (3, 3).

Step-by-step for (3, 1) and (3,):

• Right-align: (3, 1) and (_, 3) where the missing left dim is treated as 1, giving (1, 3).
• Dimension 0: 3 vs 1 -> stretch to 3.
• Dimension 1: 1 vs 3 -> stretch to 3.
• Result: (3, 3).

Common mistakes:

• Shapes like (4,) and (3,) are incompatible (neither is 1 and they are unequal) and raise an error.
• Silent broadcasting can hide bugs; a (N, 1) label accidentally broadcasting against (N, N) predictions produces a wrong loss without any error.

Use tf.broadcast_to to make broadcasting explicit and tf.reshape or tf.expand_dims to add size-1 dimensions deliberately.

Standard tensors are dense and rectangular. Ragged and sparse tensors handle two different irregular-data cases efficiently.

RaggedTensor: rows of variable length, such as tokenized sentences or user session sequences. It avoids padding waste while still supporting batched ops.

import tensorflow as tf

rt = tf.ragged.constant([[1, 2, 3], [4], [5, 6]])
print(rt.shape)          # (3, None)
print(rt.row_lengths())  # [3, 1, 2]
print((rt + 1).to_list())  # [[2, 3, 4], [5], [6, 7]]

SparseTensor: mostly zeros stored as indices, values, and dense_shape, ideal for one-hot categorical features, bag-of-words, or huge embedding lookups.

st = tf.sparse.SparseTensor(
    indices=[[0, 0], [1, 2]],
    values=[10, 20],
    dense_shape=[3, 4],
)
dense = tf.sparse.to_dense(st)   # zeros everywhere except the two entries

Rule of thumb: ragged for variable length, sparse for mostly empty. Both convert to dense when a downstream op requires it, but converting a huge sparse tensor to dense can blow up memory.

TensorFlow places ops on the GPU automatically when a compatible GPU is visible, falling back to CPU otherwise. You override placement explicitly with tf.device.

import tensorflow as tf

print(tf.config.list_physical_devices('GPU'))

with tf.device('/CPU:0'):
    a = tf.constant([1.0, 2.0])

with tf.device('/GPU:0'):
    b = tf.constant([3.0, 4.0])
    c = b * 2            # runs on GPU if available

Key points:

• Enable tf.debugging.set_log_device_placement(True) to log where each op runs.
• Ops that lack a GPU kernel automatically fall back to CPU, which can cause hidden host-device copies that slow training.
• Use tf.config.set_visible_devices to restrict which GPUs a process sees, and set_memory_growth to avoid grabbing all GPU memory at startup.

gpus = tf.config.list_physical_devices('GPU')
if gpus:
    tf.config.experimental.set_memory_growth(gpus[0], True)

Interop caution: moving data between CPU and GPU (for example, calling .numpy() inside a training loop) forces a copy and a synchronization, so keep hot-path computation on one device.

Reproducibility requires controlling every source of randomness: weight initialization, data shuffling, dropout, and augmentation. TensorFlow has both a global seed and operation-level seeds, and results depend on both.

import tensorflow as tf

tf.random.set_seed(42)          # global seed
print(tf.random.uniform([2]))   # deterministic given the global seed

How the two seeds interact:

• With only a global seed, each op still derives its own deterministic stream, so repeated identical programs match.
• An op-level seed (tf.random.uniform(..., seed=1)) makes that specific op deterministic regardless of surrounding ops.
• Setting neither gives fully nondeterministic results.

For full determinism, also:

import numpy as np, random, os
os.environ['PYTHONHASHSEED'] = '0'
random.seed(42)
np.random.seed(42)
tf.random.set_seed(42)
tf.config.experimental.enable_op_determinism()  # deterministic GPU kernels

Remaining gotchas:

• Some GPU kernels (certain reductions, atomics) are nondeterministic unless enable_op_determinism() is set, at a speed cost.
• tf.data pipelines must set shuffle(..., seed=...) and avoid num_parallel_calls nondeterminism by setting deterministic=True.
• Use the stateless generators (tf.random.stateless_uniform) when you need results that depend only on an explicit seed tensor, which is ideal for reproducible per-example augmentation.

Keras offers three ways to build models, each trading simplicity for flexibility.

Sequential: a linear stack where each layer feeds the next.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

seq = keras.Sequential([
    layers.Input(shape=(784,)),
    layers.Dense(128, activation="relu"),
    layers.Dense(10, activation="softmax"),
])

Functional: you connect layers as callables on tensors, so branching and merging are natural.

inputs = keras.Input(shape=(784,))
x = layers.Dense(128, activation="relu")(inputs)
outputs = layers.Dense(10, activation="softmax")(x)
func = keras.Model(inputs, outputs)

Subclassed: you write imperative logic in call, useful for research and dynamic graphs.

class MyModel(keras.Model):
    def __init__(self):
        super().__init__()
        self.d1 = layers.Dense(128, activation="relu")
        self.d2 = layers.Dense(10, activation="softmax")

    def call(self, x):
        return self.d2(self.d1(x))

Rule of thumb: reach for Sequential for prototypes, Functional for most production models because it stays serializable and inspectable, and subclassing only when you need imperative control flow.

These four methods form the core Keras training workflow.

• compile configures the model for training: it attaches the optimizer, loss function, and metrics. Nothing trains yet; it just wires up the objective.
• fit runs the actual training loop over batches for a number of epochs, updating weights via backpropagation.
• evaluate runs a forward pass over a dataset and returns the loss and metrics with no weight updates.
• predict returns raw model outputs (probabilities, logits, regressions) for inference.

model.compile(
    optimizer=keras.optimizers.Adam(1e-3),
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)

history = model.fit(
    x_train, y_train,
    validation_data=(x_val, y_val),
    epochs=10, batch_size=32,
)

test_loss, test_acc = model.evaluate(x_test, y_test)
probs = model.predict(x_new)

Key distinctions:

• fit returns a History object whose .history dict holds per-epoch loss and metric values.
• evaluate uses the same loss and metrics set in compile, but in inference mode (dropout off, BatchNorm using running stats).
• predict does not need labels and never touches the loss.

Callbacks are objects passed to fit that hook into the training loop at defined points (epoch start/end, batch start/end) to observe or alter behavior without rewriting the loop.

Three you should know cold:

• EarlyStopping halts training when a monitored metric stops improving, avoiding wasted epochs and overfitting.
• ModelCheckpoint saves the model (or just weights) periodically, typically the best so far.
• TensorBoard logs scalars, histograms, and graphs for visualization.

callbacks = [
    keras.callbacks.EarlyStopping(
        monitor="val_loss", patience=3, restore_best_weights=True
    ),
    keras.callbacks.ModelCheckpoint(
        "best.keras", monitor="val_loss", save_best_only=True
    ),
    keras.callbacks.TensorBoard(log_dir="./logs"),
]

model.fit(x_train, y_train,
          validation_data=(x_val, y_val),
          epochs=50, callbacks=callbacks)

Important flags:

• restore_best_weights=True rolls the model back to the epoch with the best monitored value, not the last one.
• patience is how many epochs of no improvement to tolerate before stopping.
• save_best_only=True overwrites the checkpoint only when the metric improves.

You can also write a custom callback by subclassing keras.callbacks.Callback and overriding methods like on_epoch_end.

Subclass keras.layers.Layer and implement three methods:

• __init__ stores configuration (units, activation) but should not create weights that depend on input shape.
• build(input_shape) creates weights lazily once the input shape is known, using self.add_weight.
• call(inputs) defines the forward computation.

import tensorflow as tf
from tensorflow import keras

class DenseWithScale(keras.layers.Layer):
    def __init__(self, units, **kwargs):
        super().__init__(**kwargs)
        self.units = units

    def build(self, input_shape):
        self.w = self.add_weight(
            shape=(input_shape[-1], self.units),
            initializer="glorot_uniform", trainable=True, name="w",
        )
        self.b = self.add_weight(
            shape=(self.units,), initializer="zeros",
            trainable=True, name="b",
        )

    def call(self, inputs):
        return tf.matmul(inputs, self.w) + self.b

    def get_config(self):
        config = super().get_config()
        config.update({"units": self.units})
        return config

Why build instead of __init__? It defers weight creation until the layer sees real input, so you do not hardcode the input dimension. Keras calls build automatically on the first forward pass.

Why get_config? It makes the layer serializable, so a saved model can be reloaded with keras.models.load_model when you pass custom_objects.

Use self.add_weight(trainable=False) for non-trained state like running statistics.

A custom loss is any callable taking (y_true, y_pred) and returning a scalar (or per-sample tensor Keras reduces).

import tensorflow as tf
from tensorflow import keras

def huber_loss(y_true, y_pred, delta=1.0):
    error = y_true - y_pred
    abs_error = tf.abs(error)
    quadratic = tf.minimum(abs_error, delta)
    linear = abs_error - quadratic
    return 0.5 * quadratic**2 + delta * linear

For a loss with hyperparameters or state, subclass keras.losses.Loss so it serializes cleanly.

class HuberLoss(keras.losses.Loss):
    def __init__(self, delta=1.0, **kwargs):
        super().__init__(**kwargs)
        self.delta = delta

    def call(self, y_true, y_pred):
        error = y_true - y_pred
        abs_error = tf.abs(error)
        quadratic = tf.minimum(abs_error, self.delta)
        linear = abs_error - quadratic
        return 0.5 * quadratic**2 + self.delta * linear

    def get_config(self):
        return {**super().get_config(), "delta": self.delta}

Custom metrics differ from losses: they accumulate state across batches. Subclass keras.metrics.Metric and use add_weight plus update_state, result, and reset_state.

class F1Score(keras.metrics.Metric):
    def __init__(self, name="f1", **kwargs):
        super().__init__(name=name, **kwargs)
        self.tp = self.add_weight(name="tp", initializer="zeros")
        self.fp = self.add_weight(name="fp", initializer="zeros")
        self.fn = self.add_weight(name="fn", initializer="zeros")

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_pred = tf.cast(y_pred > 0.5, tf.float32)
        y_true = tf.cast(y_true, tf.float32)
        self.tp.assign_add(tf.reduce_sum(y_true * y_pred))
        self.fp.assign_add(tf.reduce_sum((1 - y_true) * y_pred))
        self.fn.assign_add(tf.reduce_sum(y_true * (1 - y_pred)))

    def result(self):
        precision = self.tp / (self.tp + self.fp + 1e-7)
        recall = self.tp / (self.tp + self.fn + 1e-7)
        return 2 * precision * recall / (precision + recall + 1e-7)

    def reset_state(self):
        for v in self.variables:
            v.assign(0.0)

Key point: metrics are stateful and averaged over an epoch, losses are computed per batch to drive gradients.

Transfer learning reuses a network pretrained on a large dataset as a feature extractor for a new, usually smaller, task. The standard recipe has two phases.

Phase 1: feature extraction. Freeze the pretrained backbone and train only a fresh head.

import tensorflow as tf
from tensorflow import keras

base = keras.applications.ResNet50(
    include_top=False, weights="imagenet",
    input_shape=(224, 224, 3), pooling="avg",
)
base.trainable = False  # freeze all backbone weights

inputs = keras.Input((224, 224, 3))
x = keras.applications.resnet50.preprocess_input(inputs)
x = base(x, training=False)  # keep BatchNorm in inference mode
outputs = keras.layers.Dense(5, activation="softmax")(x)
model = keras.Model(inputs, outputs)

model.compile(optimizer=keras.optimizers.Adam(1e-3),
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(train_ds, epochs=10)

Phase 2: fine-tuning. Unfreeze part or all of the backbone and continue training with a much lower learning rate so you do not destroy the pretrained weights.

base.trainable = True
# optionally keep early layers frozen
for layer in base.layers[:100]:
    layer.trainable = False

model.compile(optimizer=keras.optimizers.Adam(1e-5),  # 10-100x smaller
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(train_ds, epochs=10)

Critical details:

• Pass training=False when calling the base so frozen BatchNorm layers use running statistics rather than updating them.
• You must recompile after changing trainable for the change to take effect in the training step.
• Fine-tune with a small learning rate; large steps wipe out useful pretrained features.

Keras 3 has two main serialization targets, plus a weights-only path.

The .keras file is a single zip archive holding the full model. It is the recommended format for saving and reloading inside Keras.

model.save("model.keras")
reloaded = keras.models.load_model("model.keras")

For custom layers or losses, either register them or pass custom_objects:

@keras.saving.register_keras_serializable()
class DenseWithScale(keras.layers.Layer):
    ...

reloaded = keras.models.load_model(
    "model.keras",
    custom_objects={"HuberLoss": HuberLoss},
)

SavedModel via export produces a language-neutral inference graph for deployment. It drops the Python training state and keeps only serving signatures.

model.export("saved_model_dir")          # for TF Serving / TFLite
infer = tf.saved_model.load("saved_model_dir")

When to use which:

• Iterating and resuming training in Python: .keras.
• Deploying to TF Serving, mobile, or another runtime: export to SavedModel.
• Sharing weights across code that rebuilds the architecture: save_weights.

When fit is too rigid (multiple optimizers, custom gradient manipulation, GANs, RL), write the loop yourself with tf.GradientTape.

import tensorflow as tf
from tensorflow import keras

model = keras.Sequential([
    keras.layers.Dense(64, activation="relu"),
    keras.layers.Dense(10),
])
optimizer = keras.optimizers.Adam(1e-3)
loss_fn = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
train_acc = keras.metrics.SparseCategoricalAccuracy()

@tf.function  # compile to a graph for speed
def train_step(x, y):
    with tf.GradientTape() as tape:
        logits = model(x, training=True)   # training=True for dropout/BN
        loss = loss_fn(y, logits)
    grads = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))
    train_acc.update_state(y, logits)
    return loss

for epoch in range(epochs):
    train_acc.reset_state()
    for x_batch, y_batch in train_ds:
        loss = train_step(x_batch, y_batch)
    print(f"epoch {epoch}: acc={train_acc.result():.3f}")

What each piece does:

• GradientTape records operations on watched tensors (trainable variables by default) so you can compute gradients of the loss.
• tape.gradient(loss, vars) does reverse-mode autodiff.
• optimizer.apply_gradients applies the update rule to the variables.
• @tf.function traces the Python function into a static graph, giving large speedups; keep it side-effect free.
• training=True in the forward call activates dropout and BatchNorm updates.

Gradient tricks: clip with tf.clip_by_global_norm(grads, 1.0) before applying, or use tape.watch(tensor) to differentiate w.r.t. a non-variable. Use persistent=True if you need multiple gradient calls from one tape.

Regularization fights overfitting by constraining model capacity or adding noise. Keras exposes several complementary tools.

from tensorflow import keras
from tensorflow.keras import layers, regularizers

model = keras.Sequential([
    layers.Dense(256, activation="relu",
                 kernel_regularizer=regularizers.l2(1e-4)),
    layers.Dropout(0.5),
    layers.Dense(128, activation="relu",
                 kernel_regularizer=regularizers.l2(1e-4)),
    layers.Dropout(0.3),
    layers.Dense(10, activation="softmax"),
])

Key nuances:

• Dropout is active only in training. In inference Keras scales activations so expected values match; you never manually toggle it.
• L2 penalty vs decoupled weight decay are not identical for adaptive optimizers. With Adam, adding an L2 kernel_regularizer is not the same as true weight decay. Use keras.optimizers.AdamW, which decouples decay from the gradient update, for the correct behavior.

optimizer = keras.optimizers.AdamW(
    learning_rate=1e-3, weight_decay=1e-4,
)

• Do not stack aggressive dropout on top of BatchNorm carelessly. They can interact poorly; a common pattern is BatchNorm in conv blocks and dropout in the dense head.
• Regularization strength is a hyperparameter: too much underfits, too little overfits. Tune with validation curves.

Rule of thumb: start with L2/AdamW weight decay plus modest dropout (0.2 to 0.5), add early stopping, and only then reach for heavier schemes.

tf.data.Dataset is TensorFlow's API for building input pipelines that stream data into training loops efficiently. Instead of loading everything into memory, you describe a pipeline of transformations that execute lazily and in C++, overlapping with GPU compute.

Why prefer it over a Python generator:

• Performance: transformations run in the TensorFlow runtime, not the Python interpreter, so they escape the Global Interpreter Lock and parallelize across cores.
• Prefetching: data for step N+1 is prepared while step N trains, keeping the accelerator busy.
• Portability: the same pipeline plugs into model.fit, custom loops, and distribution strategies.
• Composability: map, batch, shuffle, repeat, filter chain cleanly.

import tensorflow as tf

dataset = (
    tf.data.Dataset.from_tensor_slices((features, labels))
    .shuffle(buffer_size=10000)
    .map(preprocess, num_parallel_calls=tf.data.AUTOTUNE)
    .batch(32)
    .prefetch(tf.data.AUTOTUNE)
)

model.fit(dataset, epochs=10)

For large corpora on disk, tf.data.TFRecordDataset reads sharded TFRecord files, and Dataset.list_files plus interleave reads many shards concurrently.

A slow input pipeline starves the GPU, so these three tools remove that bottleneck.

prefetch(buffer_size) overlaps preprocessing on the CPU with model execution on the GPU. While the accelerator trains on batch N, the CPU prepares batch N+1. Always place it last in the pipeline.

cache() stores the results of expensive upstream transformations after the first epoch, either in memory or in a file. Everything before the cache runs once; everything after runs every epoch. Place cache after costly deterministic ops (decode, resize) but before random augmentation so you still get fresh randomness each epoch.

tf.data.AUTOTUNE lets the runtime dynamically tune buffer sizes and parallelism at runtime based on available CPU and memory, instead of you hardcoding magic numbers.

dataset = (
    tf.data.Dataset.from_tensor_slices(paths)
    .map(load_and_decode, num_parallel_calls=tf.data.AUTOTUNE)
    .cache()                       # cache decoded images (deterministic)
    .map(random_augment, num_parallel_calls=tf.data.AUTOTUNE)
    .batch(64)
    .prefetch(tf.data.AUTOTUNE)    # overlap CPU prep with GPU compute
)

Order matters: a common mistake is caching after random augmentation, which freezes the augmentations to the first epoch's values.

Dataset.interleave maps a function that produces a dataset over each element, then interleaves the results from multiple source datasets concurrently. Its main use is parallel reading of many files so I/O latency from one file overlaps with reading another.

Compared to map, which produces one output element per input, interleave produces a whole sub-dataset per input and pulls from several of them at once.

files = tf.data.Dataset.list_files("gs://bucket/train-*.tfrecord")

dataset = files.interleave(
    lambda path: tf.data.TFRecordDataset(path),
    cycle_length=8,                       # read 8 files concurrently
    num_parallel_calls=tf.data.AUTOTUNE,
    deterministic=False,                  # allow out-of-order for speed
)

Key knobs:

• cycle_length controls how many input elements (files) are processed concurrently. Higher values raise parallelism and memory use.
• block_length sets how many consecutive elements are drawn from each open sub-dataset before rotating.
• deterministic=False lets the runtime emit elements as soon as they are ready, trading reproducible ordering for throughput.

When to reach for it: reading from sharded TFRecords, remote object stores where per-file latency is high, or any case where a single reader cannot saturate bandwidth. For pure element-wise transforms, plain map with num_parallel_calls is the right tool instead.

Mixed precision runs most operations in 16-bit floats (float16 on GPU, bfloat16 on TPU) while keeping numerically sensitive parts in float32. On modern GPUs with Tensor Cores this cuts memory roughly in half and can speed training 2 to 3 times.

The policy: set a global policy so layers compute in 16-bit but keep their variables (weights) in float32 for stable updates.

import tensorflow as tf

tf.keras.mixed_precision.set_global_policy("mixed_float16")

model = build_model()
# IMPORTANT: final layer outputs float32 for numerical stability
outputs = tf.keras.layers.Dense(10)(x)
outputs = tf.keras.layers.Activation("softmax", dtype="float32")(outputs)

Loss scaling is the critical piece. float16 has a small dynamic range, so tiny gradients underflow to zero. A loss scaler multiplies the loss by a large factor before backprop and unscales the gradients afterward. Keras applies this automatically when you compile with the default optimizer wrapper:

optimizer = tf.keras.optimizers.Adam(1e-3)
# Keras wraps it with LossScaleOptimizer under mixed_float16 policy
model.compile(optimizer=optimizer, loss="categorical_crossentropy")

Gotchas:

• Keep the softmax/output layer in float32 to avoid unstable probabilities.
• Mixed precision helps most on GPUs with Tensor Cores (Volta and newer); older cards see little gain.
• On TPU use mixed_bfloat16, which has float32's range and needs no loss scaling.

tf.distribute.Strategy is TensorFlow's API for distributing training across multiple devices with minimal code change. Two common ones:

MirroredStrategy does synchronous data parallelism across multiple GPUs on a single machine. Each GPU holds a full replica of the model, processes a different slice of the batch, and gradients are combined with all-reduce before each weight update, so replicas stay identical.

strategy = tf.distribute.MirroredStrategy()
print("Replicas:", strategy.num_replicas_in_sync)

with strategy.scope():
    model = build_model()
    model.compile(optimizer="adam", loss="mse")

# Scale the global batch size by replica count
model.fit(dataset, epochs=10)

MultiWorkerMirroredStrategy extends the same synchronous all-reduce approach across multiple machines. Each worker is configured through the TF_CONFIG environment variable describing the cluster and this worker's index.

# TF_CONFIG example (set per worker before starting):
# {"cluster": {"worker": ["host1:12345", "host2:12345"]},
#  "task": {"type": "worker", "index": 0}}
strategy = tf.distribute.MultiWorkerMirroredStrategy()

Key points to raise:

• Everything that creates variables (model, optimizer, metrics) must be built inside strategy.scope().
• The batch size you pass is the global batch size; it is split across replicas, so scale it up and often the learning rate too.
• Multi-worker requires each worker to save checkpoints to avoid conflicts and to handle fault tolerance; only the chief writes the final model.
• For huge embedding tables, ParameterServerStrategy (async) may suit better than mirrored all-reduce.

The standard path is the SavedModel format served by TensorFlow Serving, a high-performance C++ server with gRPC and REST endpoints.

Step 1: export a SavedModel.

model.export("saved_model/my_model/1")   # note the version subdirectory

The trailing 1 is a version number; TF Serving watches the directory and can hot-swap versions without downtime.

Step 2: run TensorFlow Serving (typically via Docker):

docker run -p 8501:8501 \
  --mount type=bind,source=$(pwd)/saved_model/my_model,target=/models/my_model \
  -e MODEL_NAME=my_model tensorflow/serving

Step 3: send a REST request:

import requests, json
resp = requests.post(
    "http://localhost:8501/v1/models/my_model:predict",
    data=json.dumps({"instances": inputs.tolist()}),
)
predictions = resp.json()["predictions"]

What TF Serving gives you:

• Dynamic batching of concurrent requests for higher throughput.
• Version management with canary and A/B rollout across versions.
• gRPC for low-latency, high-volume traffic; REST for convenience.

For on-device or edge deployment, convert to TFLite / LiteRT instead (covered separately). For full MLOps pipelines, TFX wraps training, validation, and serving.

For phones, wearables, and microcontrollers you convert to TFLite (now branded LiteRT), a compact runtime optimized for low latency and small binary size.

Convert the SavedModel:

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_saved_model("saved_model/my_model/1")
converter.optimizations = [tf.lite.Optimize.DEFAULT]   # enables quantization
tflite_model = converter.convert()

with open("model.tflite", "wb") as f:
    f.write(tflite_model)

Quantization is the main lever for shrinking and accelerating models:

• Dynamic range quantization: weights to int8, activations computed in float. Roughly 4x smaller, easy default.
• Full integer quantization: weights and activations to int8 using a representative dataset; needed for integer-only accelerators and Edge TPUs.
• Float16 quantization: halves size, good for GPU delegates.

# Full integer quantization with a representative dataset
def rep_data():
    for sample in calibration_samples.take(100):
        yield [tf.cast(sample, tf.float32)]

converter.representative_dataset = rep_data
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]

Running inference uses the Interpreter, and on device you attach delegates (GPU, NNAPI on Android, Core ML on Apple) to offload to hardware accelerators.

TensorFlow 2.x runs eagerly by default, so you can debug with normal Python tools until you wrap code in @tf.function, where graph tracing changes the rules.

1. Shape errors. Most bugs are dimension mismatches. Inspect and assert shapes explicitly:

print(x.shape, x.dtype)             # eager: works like NumPy
tf.debugging.assert_shapes([(x, ("batch", 128)), (w, (128, 10))])

A frequent culprit is a missing or extra batch dimension, or broadcasting that silently succeeds then fails downstream.

2. Printing inside graphs. A plain Python print runs once at trace time, not every step. To print actual runtime values inside a @tf.function, use tf.print, which becomes a graph op that executes each call:

@tf.function
def step(x):
    tf.print("x mean:", tf.reduce_mean(x))   # prints every execution
    print("traced once")                     # prints only during tracing
    return model(x)

3. Force eager execution to debug. Temporarily run functions eagerly so breakpoints and print behave normally:

tf.config.run_functions_eagerly(True)   # disable graph mode globally
# ... reproduce the bug, step through with pdb ...
tf.config.run_functions_eagerly(False)  # re-enable for speed

You can also pass model.compile(..., run_eagerly=True) to debug a fit loop.

4. NaNs and Infs. Enable the numeric checker to pinpoint where they first appear:

tf.debugging.enable_check_numerics()

5. Retracing warnings. Repeated retracing (from passing Python ints or varying shapes) silently slows training; use input_signature on tf.function to lock shapes.

Framework trade-offs. The two have converged, but differences remain:

Rule of thumb: PyTorch for research flexibility and rapid iteration; TensorFlow when you need a battle-tested serving and edge story. Both support XLA compilation and mixed precision.

Handling GPU OOM. When training crashes with an out-of-memory error, work through these levers:

• Reduce batch size, then recover the effective batch with gradient accumulation (accumulate grads over several micro-batches before one update).
• Mixed precision (mixed_float16) roughly halves activation memory.
• Gradient checkpointing trades compute for memory by recomputing activations during the backward pass instead of storing them.
• Limit or grow GPU memory so TensorFlow does not grab all of it up front:

gpus = tf.config.list_physical_devices("GPU")
for gpu in gpus:
    tf.config.experimental.set_memory_growth(gpu, True)

• Shrink the model (fewer layers, smaller hidden size) or shard it with model parallelism across GPUs.
• Trim the input pipeline: smaller image resolution, or lower cache/shuffle buffers that themselves consume host memory.
• Free references: avoid accumulating tensors in Python lists across steps, which keeps them alive and leaks memory.

Diagnosing: watch nvidia-smi, and note that memory that spikes then crashes usually points to a too-large batch or an unbounded buffer, while a slow climb points to a leak from retained tensors.