TensorFlow — Common Interview Questions and Answers

1. What is TensorFlow?

TensorFlow is Google’s open-source deep learning framework (2015). TF 2.x is eager-by-default with Keras as the official high-level API. Strong in production, mobile/edge (TFLite), browser (TF.js), and TPU training. Mindshare for new research has shifted to PyTorch, but TF remains widely deployed and well-supported.

2. What changed between TF 1.x and TF 2.x?

TF 1.x is legacy. New code should be TF 2.x with tf.keras. If you see tf.Session, tf.placeholder, feed_dict — that’s 1.x.

3. What’s the difference between tf.Tensor and tf.Variable?

• tf.Tensor — immutable n-dimensional array. Outputs of computations.
• tf.Variable — mutable tensor. Used for model parameters (weights, biases). Can be modified with .assign() / .assign_add(). Trainable by default.

w = tf.Variable(tf.zeros([10, 5]))
# in-place update allowed
w.assign_add(tf.ones([10, 5]))

4. What is Keras and how does it relate to TF?

Keras is the official high-level API for TF (since TF 2.0). Provides:

• keras.Model, keras.layers.* — composable building blocks.
• model.compile, model.fit, model.evaluate, model.predict — training loop hidden.
• Sequential, Functional, and Subclassing APIs for defining models.

Keras 3 (2023+) is multi-backend — same Keras code runs on TF, JAX, or PyTorch by setting KERAS_BACKEND env var.

5. What are the three ways to build a Keras model?

1. Sequential — linear stack, simplest:

   python Wrap Copy

   model = keras.Sequential([
       layers.Dense(128, activation="relu"),
       layers.Dense(10),
   ])

2. Functional — DAG with multi-input/multi-output support:

   python Wrap Copy

   inputs = keras.Input(shape=(784,))
   x = layers.Dense(128, activation="relu")(inputs)
   outputs = layers.Dense(10)(x)
   model = keras.Model(inputs, outputs)

3. Subclassing — define __init__ and call, exactly like a PyTorch nn.Module. Full flexibility, and you give up .summary() shape inference until the model is built.

Most production code uses Sequential or Functional; subclassing is for custom control flow.

6. What’s tf.GradientTape?

TF’s autograd mechanism. Context manager that records operations on watched tensors so you can compute gradients later:

x = tf.Variable(2.0)
with tf.GradientTape() as tape:
    y = x ** 3 + 2 * x
dy_dx = tape.gradient(y, x)             # 14

Variables are watched automatically. Tensors need tape.watch(tensor). By default, a tape is consumed after one .gradient() call — use persistent=True to call it multiple times.

7. What does @tf.function do?

Traces a Python function into a TF computation graph that can be optimized and run faster on GPU/TPU. First call traces; subsequent calls run the compiled graph.

@tf.function
def train_step(xb, yb):
    with tf.GradientTape() as tape:
        loss = loss_fn(yb, model(xb, training=True))
    grads = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))
    return loss

Gotcha: Python side effects (print, mutations) only run during tracing, not on every call. Use tf.print for runtime printing.

Gotcha: Different input shapes/dtypes trigger retracing. If you call with many different shapes, performance suffers — set input_signature to lock the trace.

8. What’s the difference between eager and graph mode?

• Eager (TF 2 default) — ops run immediately, Python control flow works, easy to debug.
• Graph — ops are compiled into a TF graph; the graph runs as a unit. Faster, optimizable, but harder to debug.

In TF 2.x you write eager code, then opt into graph mode for tight loops via @tf.function. Best of both worlds: develop in eager, deploy / hot-path in graph.

9. What’s tf.data.Dataset?

TF’s input pipeline API. Composable, lazy, parallelized. Designed to overlap data prep with GPU/TPU compute.

ds = tf.data.Dataset.from_tensor_slices((x, y))
ds = (
    ds.shuffle(10_000)
      .map(augment, num_parallel_calls=tf.data.AUTOTUNE)
      .batch(64)
      .prefetch(tf.data.AUTOTUNE)
)

for xb, yb in ds:
    train_step(xb, yb)

Key helpers: .shuffle, .batch, .map, .prefetch, .cache, .interleave. AUTOTUNE lets TF pick parallelism dynamically.

10. How do you save and load a Keras model?

Two main formats:

# Recommended: TF SavedModel (a directory)
model.save("path/to/model")
loaded = keras.models.load_model("path/to/model")

# Or .keras file (single file, also recommended)
model.save("model.keras")

# Just weights
model.save_weights("weights.h5")
model.load_weights("weights.h5")

The full save includes architecture, weights, optimizer state, training config. For deployment, SavedModel is the canonical format (TF Serving consumes it).

11. How do you train with a custom loop?

optimizer = keras.optimizers.AdamW(1e-3)
loss_fn = keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = keras.metrics.SparseCategoricalAccuracy()

@tf.function
def train_step(xb, yb):
    with tf.GradientTape() as tape:
        logits = model(xb, training=True)
        loss = loss_fn(yb, logits)
    grads = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))
    metric.update_state(yb, logits)
    return loss

for epoch in range(EPOCHS):
    metric.reset_state()
    for xb, yb in train_ds:
        train_step(xb, yb)
    print(f"Epoch {epoch}: acc={metric.result():.3f}")

Note training=True passed to model() — Keras layers need it explicitly in custom loops (Dropout, BatchNorm depend on it).

12. What’s tf.distribute.Strategy?

TF’s distributed training API. Different strategies for different setups:

• MirroredStrategy — multi-GPU on one machine (data parallel).
• MultiWorkerMirroredStrategy — multi-GPU across machines.
• TPUStrategy — TPU pods.
• ParameterServerStrategy — async parameter server (rare now).

strategy = tf.distribute.MirroredStrategy()
with strategy.scope():
    model = build_model()
    model.compile(...)

# automatically distributed
model.fit(...)

Wrap model creation in strategy.scope(); the rest of the training code is unchanged.

13. Where does TF deploy that PyTorch struggles to?

This is the real reason TF is still in production, and it is one answer rather than three products.

conv = tf.lite.TFLiteConverter.from_saved_model("model")
conv.optimizations = [tf.lite.Optimize.DEFAULT]   # int8 / fp16
open("model.tflite", "wb").write(conv.convert())

docker run -p 8501:8501 -v /path/to/model:/models/m   -e MODEL_NAME=m tensorflow/serving

TF Serving gives versioning, A/B and hot-reload out of the box. TFLite’s quantisation plus hardware delegates (NNAPI, GPU, Hexagon) is the strongest part of the story, and on-device is where TF still clearly leads.

The caveat that keeps this current: for LLM serving, vLLM and TensorRT-LLM have replaced TF Serving, and PyTorch has ExecuTorch for on-device. The gap is narrowing everywhere except the long tail of embedded hardware.

14. How do you do transfer learning in TF?

base = keras.applications.MobileNetV2(
    input_shape=(224, 224, 3),
    # drop the original classification head
    include_top=False,
    weights="imagenet",
)
base.trainable = False           # freeze

model = keras.Sequential([
    base,
    layers.GlobalAveragePooling2D(),
    layers.Dense(NUM_CLASSES),
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy")
model.fit(...)

# Then unfreeze for fine-tuning
base.trainable = True
model.compile(optimizer=keras.optimizers.Adam(1e-5), loss=...)  # lower LR
model.fit(...)

Standard pattern: freeze base, train head; then unfreeze and fine-tune end-to-end with a tiny LR.

15. What are common TF gotchas?

• @tf.function Python side effects — print(), mutations only run during tracing. Use tf.print.
• Retracing on every call — Python ints/lists trigger new traces; use tensors or input_signature.
• Creating tf.Variable inside @tf.function — raises; create variables outside.
• training=True/False in custom loops — must pass explicitly for Dropout/BatchNorm.
• TF 1.x code in tutorials — anything with tf.Session / tf.placeholder is legacy.
• GPU memory allocation — TF grabs all GPU memory by default. Limit with tf.config.experimental.set_memory_growth.
• Mixing TF and Keras APIs — stick to tf.keras.*, not standalone keras package (different versions exist).

16. What’s the difference between TF and JAX?

Both are Google projects; both compile to XLA. Differences:

• TF — object-oriented Keras, eager-by-default, mature ecosystem, production tooling.
• JAX — functional, pure functions transformed by jit / grad / vmap / pmap. Closer to NumPy. Used heavily in DeepMind / research.

JAX is faster for research-grade code and TPU workloads but has a smaller ecosystem. TF is the more “batteries-included” choice for production.

Keras 3 supports JAX as a backend — you can write Keras code and run it on JAX for the speed/TPU benefits.

17. When would you pick TF over PyTorch?

• TPU training — Google’s accelerators; TF/JAX are canonical here.
• Mobile / edge deployment — TFLite is more mature than PyTorch Mobile / ExecutorTorch.
• Browser inference — TF.js is best-in-class.
• Existing TFX pipelines — don’t migrate working production infra.
• Team already has TF expertise — switching costs matter.

For most other cases (especially LLMs, research, HF transformers), PyTorch is the default. See PyTorch vs TensorFlow for the full comparison.