Epochs

Definition: An epoch refers to one complete pass through the entire dataset during training. The model will see every data point once per epoch.

Typical Range: **10 to 1000**, depending on the dataset size and complexity.

• When increased: Allows the model more time to learn patterns but may overfit if too high.
• When decreased: Training is faster but the model might underfit, missing important patterns.

Batch Size

Definition: The number of samples processed before the model is updated. A batch size of 32 means the model processes 32 samples before updating its weights.

Typical Range: **8 to 256**.

• When increased: Training is faster but may overlook finer details.
• When decreased: More precise learning but slower training and potentially noisier updates.

Dropout Rate

Definition: A regularization technique to prevent overfitting by randomly "dropping out" neurons during training. A dropout rate of 0.2 means 20% of neurons are ignored during each update.

Typical Range: **0.1 to 0.5**.

• When increased: Reduces overfitting but may hinder learning complex patterns.
• When decreased: Allows model to learn more but may overfit.

Learning Rate

Definition: Controls how much the model adjusts its weights during training. A smaller learning rate means smaller adjustments.

Typical Range: **0.0001 to 0.1**.

• When increased: Model learns faster but may miss optimal values.
• When decreased: More precise learning but slower convergence.

Activation Function

Definition: Determines the output of a neural network layer. It introduces non-linearities.

Common Choices: **ReLU**, **Sigmoid**, **Tanh**.

• ReLU: Effective for deep networks, avoids vanishing gradient.
• Sigmoid: Good for binary classification, outputs between 0 and 1.
• Tanh: Outputs between -1 and 1, zero-centered.

Hidden Layers

Definition: Layers between input and output layers. More layers can capture more complex patterns.

Typical Range: **1 to 5**.

• When increased: Can model complex relationships but may overfit or require more data.
• When decreased: Simpler model, faster training, may miss complex patterns.

What is Overfitting?

Overfitting occurs when a model learns the training data too well, including noise, and performs poorly on new, unseen data. Regularization techniques like dropout help prevent this.