Epochs

Definition: An epoch refers to one complete pass through the entire dataset during training. In this case, the dataset will be passed through the neural network 100 times.

Typical Range: Epochs can range from **10 to 1000** depending on the dataset size and complexity. Lower values (e.g., **10-50**) are usually used for smaller datasets or faster training, while higher values (e.g., **100-1000**) are used for more complex data.

• When increased: Training for more epochs gives the model more time to learn, but too many can cause overfitting, where the model becomes too specialized on the training data.
• When decreased: Fewer epochs will make training faster, but the model might not learn as much, potentially missing important patterns.

Batch Size

Definition: The batch size determines how many samples are processed at once before updating the model's internal parameters. Here, the batch size is 8, meaning the model processes 8 entries at a time.

Typical Range: Batch sizes typically range from **8 to 256**. Smaller values (e.g., **8-32**) make training slower but more precise, while larger values (e.g., **64-256**) allow faster training but may miss some finer patterns in the data.

• When increased: A larger batch size allows faster training but may overlook finer details in the data.
• When decreased: A smaller batch size means more precise learning, but it takes longer to train and can result in noisier updates.

Dropout Rate

Definition: The dropout rate is a regularization method to prevent overfitting by randomly "dropping" a percentage of neurons during training. A 40% dropout rate means that 40% of neurons are turned off during each training step.

Typical Range: The dropout rate typically ranges from **0.2 to 0.5**. Lower values (e.g., **0.2-0.3**) allow more neurons to remain active, while higher values (e.g., **0.4-0.5**) drop more neurons, reducing the risk of overfitting.

• When increased: A higher dropout rate reduces the risk of overfitting but may limit the model's ability to learn complex patterns.
• When decreased: A lower dropout rate allows the model to learn more from the data but increases the risk of overfitting, where the model may perform poorly on new data.

What is Overfitting?

Overfitting happens when a model becomes too good at learning from the training data, memorizing it rather than understanding general patterns. As a result, the model performs well on the data it was trained on, but it struggles to make accurate predictions when given new, unseen data. This is like studying only past exam questions and failing to answer questions that are slightly different in a real exam.