feat: Add sklearn to requirements.txt and include transfer learning and early stopping in theory section

requirements.txt

@@ -2,4 +2,5 @@ pandas
 seaborn
 jupyter
 torch
-matplotlib
+matplotlib
+sklearn

src/_quarto.yml

@@ -76,6 +76,10 @@ website:
               text: "Underfitting"
             - href: theory/hyperparameter_tuning.qmd
               text: "Hyperparameter Tuning"
+            - href: theory/transfer_learning.qmd
+              text: "Transfer Learning"
+            - href: theory/early_stopping.qmd
+              text: "Early Stopping"
 
           - href: theory/perplexity_in_ai.qmd
             text: "Perplexity and Quantization"

src/theory/early_stopping.qmd

@@ -0,0 +1,63 @@
+# Early Stopping in AI Training: Maximizing Efficiency for Better Results
+
+Early stopping is a powerful technique used during the training of artificial intelligence (AI) models that helps prevent overfitting and enhances model performance. This method involves monitoring the model's performance on a validation set and terminating the training process when it ceases to improve, or starts deteriorating. In this article, we will explore the concept of early stopping in AI training, its benefits, how to implement it effectively using Python, and demonstrate its impact with visualizations.
+
+## Understanding Overfitting and Early Stopping
+
+Overfitting occurs when a model learns the training data too well, capturing noise or random fluctuations instead of generalizing patterns in the data. This can lead to poor performance on new, unseen data. By implementing early stopping, we can mitigate overfitting and improve our AI models' ability to generalize.
+
+```python
+import numpy as np
+from sklearn.metrics import mean_squared_error
+
+# Generating a simple dataset with noise
+np.random.seed(0)
+X = np.linspace(-1, 1, 20).reshape(-1, 1)
+y = np.sin(X).ravel() + np.random.normal(scale=0.3, size=len(X))
+```
+
+## Implementing Early Stopping in Python
+
+To implement early stopping during AI training using popular libraries like TensorFlow and Keras, we can use callbacks provided by these frameworks. Here's an example of how to set up an early stopping mechanism:
+
+```python
+from tensorflow.keras import Sequential
+from tensorflow.keras.layers import Dense
+from tensorflow.keras.callbacks import EarlyStopping
+
+# Define a simple model architecture
+model = Sequential([Dense(1, input_shape=(1,))])
+model.compile(optimizer='adam', loss='mse')
+
+# Set up early stopping callback
+early_stopper = EarlyStopping(monitor='val_loss', patience=5)
+
+# Train the model with early stopping
+history = model.fit(X, y, epochs=100, validation_split=0.2, callbacks=[early_stopper])
+```
+
+## Visualizing Early Stopping Effectiveness
+
+To demonstrate how effective early stopping can be in preventing overfitting and improving AI model performance, let's plot the training and validation loss during the training process using Matplotlib:
+
+```python
+import matplotlib.pyplot as plt
+
+# Plotting training and validation losses
+plt.figure(figsize=(12, 6))
+pltenas = history.history['val_loss']
+train_losses = history.history['loss']
+epochs = range(len(train_losses))
+
+plt.plot(epochs, train_losses, 'b', label='Training loss')
+plt.plot(epochs, val_losses, 'r', label='Validation loss')
+plt.title('Training and validation losses over time')
+plt.legend()
+plt.show()
+```
+
+## Conclusion
+
+Early stopping is a valuable technique in AI training that can help prevent overfitting and enhance model performance by terminating the training process when it ceases to improve or starts deteriorating. By using callbacks like EarlyStopping, we can implement this technique effectively with popular libraries such as TensorFlow and Keras. The visualization of training and validation losses demonstrated above further emphasizes how early stopping contributes to more efficient AI model development.
+
+By incorporating early stopping into your machine learning workflows, you'll be taking a significant step towards creating robust and highly-performing models that are better suited for real-world applications.

src/theory/transfer_learning.qmd

@@ -0,0 +1,69 @@
+
+# Transfer Learning: Techniques and Applications
+
+## Introduction
+
+Transfer learning is a powerful technique in machine learning that allows us to leverage knowledge from one problem domain to improve learning efficiency or performance on another, but related, domain. This approach has revolutionized the field of deep learning by enabling models to achieve state-of-the-art results with less data and computational resources.
+
+## What is Transfer Learning?
+
+At its core, transfer learning involves two main components: a source task (domain A) where abundant labeled data exists, and a target task (domain B) that has limited or noisy data available. The goal of transfer learning is to utilize the knowledge gained from solving domain A's problem to benefit performance on domain B's problem.
+
+## Techniques for Transfer Learning
+
+There are several techniques used in transfer learning, which can be broadly classified into two categories: **fine-tuning** and **feature extraction**.
+
+### Fine-Tuning
+
+Fine-tuning involves taking a pre-trained model (usually trained on a large dataset) and continuing the training process to adapt it for the target task. The most common approach is to replace the final layer(s) of the neural network with new layers tailored to the target problem, while keeping the earlier layers fixed.
+
+Here's an example using Keras/TensorFlow:
+
+```python
+from tensorflow import keras
+
+# Load pre-trained model (assuming ResNet50 trained on ImageNet)
+pretrained_model = keras.applications.ResNet50(weights="imagenet", include_top=False, input_shape=(224, 224, 3))
+
+# Freeze layers up to the desired depth (e.g., 17th layer)
+pretrained_model.layers[:18].trainable = False
+
+# Add new classification head for target task
+x = keras.layers.GlobalAveragePooling2D()(pretrained_model.output)
+x = keras.layers.Dense(units=num_classes, activation="softmax")(x)
+final_model = keras.models.Model(inputs=pretrained_model.input, outputs=x)
+```
+
+### Feature Extraction
+
+In feature extraction-based transfer learning, the pre-trained model is used as a fixed feature extractor, and its output serves as input to another classifier trained from scratch for the target task. This approach does not modify the original network architecture but instead utilizes learned features directly.
+
+Here's an example using Keras/TensorFlow:
+
+```python
+from tensorflow import keras
+
+# Load pre-trained model (ResNet50) as a feature extractor
+pretrained_model = keras.applications.ResNet50(weights="imagenet", include_top=False, input_shape=(224, 224, 3))
+
+# Extract features from the pre-trained model
+features = pretrained_model.output
+
+# Flatten and add new classification head for target task
+x = keras.layers.GlobalAveragePooling2D()(features)
+x = kerasinas.layers.Dense(units=num_classes, activation="softmax")(x)
+final_model = keras.models.Model(inputs=pretrained_model.input, outputs=x)
+```
+
+## Benefits of Transfer Learning
+
+Transfer learning offers several advantages:
+
+1. **Reduced Data Requirement**: By leveraging pre-existing models and knowledge from large datasets (e.g., ImageNet), transfer learning allows us to achieve high performance even with limited labeled data in the target domain.
+2. **Faster Convergence**: Since a portion of the model is already learned, training times are significantly reduced compared to building an entirely new network.
+3. **Improved Performance**: Transfer learning can lead to better generalization and accuracy by utilizing knowledge from related tasks or domains.
+
+## Conclusion
+
+Transfer learning has transformed machine learning applications across various fields such as computer vision, natural language processing, and speech recognition. By understanding the techniques of fine-tuning and feature extraction, developers can effectively apply transfer learning to their problems, saving time and resources while achieving impressive results.
+