Neural Networks with TensorFlow and PyTorch: Understand the Basics

A neural network combines simple numerical operations to learn a mapping from inputs to outputs. TensorFlow and PyTorch help define these operations, calculate gradients and run training. They are tools, not different kinds of intelligence.

The parts of a small network

A tensor is an array of numbers with a shape. A dense layer combines inputs with weights and biases. An activation adds a non-linear step. A loss measures a chosen kind of error. An optimiser updates parameters using gradients.

For four flower measurements, a small classifier could use four inputs, eight hidden units and three output scores. The output scores are not automatically calibrated probabilities. The loss function must match the output and label format.

Understand a training step

1. Read a batch of training examples.
2. Run the network to get predictions.
3. Calculate loss against the labels.
4. Calculate gradients.
5. Update parameters.
6. Evaluate on separate data without updating parameters.

An epoch is one pass through the training set. A batch is the group processed in one step. More epochs do not always improve new-data results: a model can overfit.

Run the idea without a GPU

This complete small example uses scikit-learn's multilayer perceptron. It lets you practise a neural classifier before installing a large deep-learning framework. Save it as neural_network.py after installing scikit-learn.

from sklearn.datasets import load_iris
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X,y = load_iris(return_X_y=True)
X_train,X_test,y_train,y_test = train_test_split(
    X,y,test_size=0.25,stratify=y,random_state=42)
model = make_pipeline(StandardScaler(),MLPClassifier(
    hidden_layer_sizes=(8,),solver="lbfgs",max_iter=2000,random_state=42))
model.fit(X_train,y_train)
print("Held-out accuracy:",round(accuracy_score(y_test,model.predict(X_test)),3))

Run python neural_network.py. It prints accuracy on a held-out Iris split. Compare it with the simpler model in the supervised-learning lesson. A neural network is not automatically a better choice for small tables.

The same layer idea in PyTorch

This optional shape example requires a compatible PyTorch installation from its official instructions. It is not a complete training script and does not download a dataset.

import torch
from torch import nn
network = nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 3))
example_batch = torch.zeros(2, 4)
print(network(example_batch).shape)  # two rows, three scores each

The same idea in TensorFlow

This separate optional example requires a compatible TensorFlow installation. You do not need both frameworks to learn the idea.

import tensorflow as tf
network = tf.keras.Sequential([
    tf.keras.Input(shape=(4,)),
    tf.keras.layers.Dense(8, activation="relu"),
    tf.keras.layers.Dense(3)
])
print(network(tf.zeros((2, 4))).shape)

Both examples produce a shape of two by three. PyTorch commonly exposes a training loop directly; TensorFlow with Keras offers a high-level fit workflow as well as custom loops. Both support many styles. Avoid choosing from outdated claims that one is only for research and the other only for production.

Common mistakes

Check tensor shapes, label types and loss expectations. Standardise suitable numeric inputs using training data. Keep validation separate. Use evaluation mode where your framework requires it for layers such as dropout. A random seed helps repeat an experiment but does not guarantee identical results across every device.

Practice

Explain why the last layer has three outputs. What changes for a one-number regression task? Choose one framework for further study and follow its official beginner training tutorial rather than installing every tool at once.

Official references

Reviewed on 25 October 2026. Use documentation for your installed versions.

• PyTorch beginner workflow
• TensorFlow beginner workflow
• Small neural networks with scikit-learn

All AI & Machine Learning tutorials