{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8835c846",
   "metadata": {},
   "outputs": [],
   "source": [
    "    stratify=y if len(np.unique(y)) > 1 and np.min(np.bincount(y.astype(int))) >= 2 else None\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "OzkAq0IIFAfj",
   "metadata": {
    "id": "OzkAq0IIFAfj"
   },
   "source": [
    "## Q1. Multiple Linear Regression from Scratch (Normal Equation)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "TNttBpMvFAfj",
   "metadata": {
    "id": "TNttBpMvFAfj"
   },
   "source": [
    "**a.** Load the dataset using NumPy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wcrZHsDtFAfj",
   "metadata": {
    "id": "wcrZHsDtFAfj"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error, r2_score\n",
    "\n",
    "data = np.array([\n",
    "    [1000, 2, 35],\n",
    "    [1200, 2, 40],\n",
    "    [1500, 3, 50],\n",
    "    [1800, 3, 60],\n",
    "    [2200, 4, 75],\n",
    "    [2500, 4, 85],\n",
    "    [2800, 5, 100]\n",
    "], dtype=float)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "LebVpASRFAfj",
   "metadata": {
    "id": "LebVpASRFAfj"
   },
   "source": [
    "**b.** Separate Independent variables (X) and Dependent variable (Y)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f42Oh4vQFAfk",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "f42Oh4vQFAfk",
    "outputId": "38950135-0b9a-4469-818c-8b4ac8d2ca42"
   },
   "outputs": [],
   "source": [
    "X = data[:, :2]\n",
    "Y = data[:, 2]\n",
    "\n",
    "print(\"X =\\n\", X)\n",
    "print(\"Y =\", Y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cO5D4sESFAfk",
   "metadata": {
    "id": "cO5D4sESFAfk"
   },
   "source": [
    "**c.** Add the intercept column manually (x0 to account for β0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "lTo8CeBjFAfk",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lTo8CeBjFAfk",
    "outputId": "ef97eb9e-82b5-45bf-ce2e-9b6e06473fa7"
   },
   "outputs": [],
   "source": [
    "X_b = np.c_[np.ones((X.shape[0], 1)), X]\n",
    "print(X_b)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6-yFNES1FAfk",
   "metadata": {
    "id": "6-yFNES1FAfk"
   },
   "source": [
    "**d.** Compute the regression coefficients using the Normal Equation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "Y9xtk7o0FAfk",
   "metadata": {
    "id": "Y9xtk7o0FAfk"
   },
   "outputs": [],
   "source": [
    "theta = np.linalg.inv(X_b.T @ X_b) @ X_b.T @ Y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1hM9UdGRFAfk",
   "metadata": {
    "id": "1hM9UdGRFAfk"
   },
   "source": [
    "**e.** Print all coefficients."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "JhNw4hunFAfk",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "JhNw4hunFAfk",
    "outputId": "2b16b105-98f7-449d-d71a-762d7f170c6f"
   },
   "outputs": [],
   "source": [
    "print(\"Intercept (beta0) =\", theta[0])\n",
    "print(\"Coefficient for House Size (beta1) =\", theta[1])\n",
    "print(\"Coefficient for Bedrooms (beta2) =\", theta[2])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "zUbJmw9jFAfk",
   "metadata": {
    "id": "zUbJmw9jFAfk"
   },
   "source": [
    "**f.** Predict the price of a house having 3 bedrooms and 2000 sq. ft size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "buBpW0tyFAfk",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "buBpW0tyFAfk",
    "outputId": "5aebbfd8-a10d-4010-c4a8-169c190e97fc"
   },
   "outputs": [],
   "source": [
    "x_new = np.array([1, 2000, 3])\n",
    "predicted_price = x_new @ theta\n",
    "print(\"Predicted Price (Lakhs) =\", predicted_price)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Tc62xWMlFAfk",
   "metadata": {
    "id": "Tc62xWMlFAfk"
   },
   "source": [
    "**g.** Compute MSE and R2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "MqptaIszFAfk",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "MqptaIszFAfk",
    "outputId": "38fb5457-7ca2-4d57-b0c2-acd8fb524576"
   },
   "outputs": [],
   "source": [
    "Y_pred_normal_eq = X_b @ theta\n",
    "\n",
    "mse_normal_eq = mean_squared_error(Y, Y_pred_normal_eq)\n",
    "r2_normal_eq = r2_score(Y, Y_pred_normal_eq)\n",
    "\n",
    "print(\"MSE (Normal Equation) =\", mse_normal_eq)\n",
    "print(\"R2 (Normal Equation) =\", r2_normal_eq)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "LCB5vBkkFAfk",
   "metadata": {
    "id": "LCB5vBkkFAfk"
   },
   "source": [
    "## Q2. Multiple Linear Regression using scikit-learn (Same Dataset)\n",
    "Use the same tiny dataset and train a Multiple Linear Regression model using scikit-learn."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "jvGBmSbtFAfk",
   "metadata": {
    "id": "jvGBmSbtFAfk"
   },
   "source": [
    "**a.** Train a Multiple Linear Regression model using Scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "H2Bu7qTxFAfk",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "H2Bu7qTxFAfk",
    "outputId": "c37feb88-13a0-443c-942a-3851158b8673"
   },
   "outputs": [],
   "source": [
    "sklearn_model = LinearRegression()\n",
    "sklearn_model.fit(X, Y)\n",
    "\n",
    "print(\"Intercept (sklearn) =\", sklearn_model.intercept_)\n",
    "print(\"Coefficients (sklearn) =\", sklearn_model.coef_)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pCfk59JJFAfl",
   "metadata": {
    "id": "pCfk59JJFAfl"
   },
   "source": [
    "**b.** Predict the price of a house having 3 bedrooms and 2000 sq. ft size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "lpEhBDHNFAfl",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lpEhBDHNFAfl",
    "outputId": "11bfb96e-fb21-4292-87f5-82fabdccfccf"
   },
   "outputs": [],
   "source": [
    "predicted_price_sklearn = sklearn_model.predict([[2000, 3]])\n",
    "print(\"Predicted Price (Lakhs, sklearn) =\", predicted_price_sklearn[0])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "iAN3L5_BFAfl",
   "metadata": {
    "id": "iAN3L5_BFAfl"
   },
   "source": [
    "**c.** Compare MSE obtained using Q1 and Q2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5M-qygkFAfl",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "b5M-qygkFAfl",
    "outputId": "9e8b1dc8-3615-47f9-ea82-d59991103eb9"
   },
   "outputs": [],
   "source": [
    "Y_pred_sklearn = sklearn_model.predict(X)\n",
    "mse_sklearn = mean_squared_error(Y, Y_pred_sklearn)\n",
    "\n",
    "print(\"MSE (Q1 Normal Equation) =\", mse_normal_eq)\n",
    "print(\"MSE (Q2 scikit-learn)    =\", mse_sklearn)\n",
    "print(\"Difference =\", abs(mse_normal_eq - mse_sklearn))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "F25ykMM5FAfl",
   "metadata": {
    "id": "F25ykMM5FAfl"
   },
   "source": [
    "## Q3. House Price Prediction on a Realistic Dataset\n",
    "Consider the same house price prediction problem with a more realistic dataset (`House_Price.csv`), which includes features such as house area in square feet, number of bedrooms, number of bathrooms, age of the house, distance from city center, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7Y8qcvGzFAfl",
   "metadata": {
    "id": "7Y8qcvGzFAfl"
   },
   "source": [
    "**a.** Load the dataset using Pandas and display the first five records."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "AXTixaB_FAfl",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 304
    },
    "id": "AXTixaB_FAfl",
    "outputId": "9aff87f7-378c-460a-bb5f-84f7ca4ce960"
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"House_Price.csv\")\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Pgtb50OMFAfl",
   "metadata": {
    "id": "Pgtb50OMFAfl"
   },
   "source": [
    "**b.** Preprocess the dataset:\n",
    "- Check for missing values.\n",
    "- Apply One-Hot Encoding to the Location feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eA81x8znFAfl",
   "metadata": {
    "id": "eA81x8znFAfl"
   },
   "outputs": [],
   "source": [
    "print(\"Missing values per column:\\n\", df.isnull().sum())\n",
    "\n",
    "df_encoded = pd.get_dummies(df, columns=['Location'], drop_first=True)\n",
    "df_encoded.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fvlnlV6LFAfl",
   "metadata": {
    "id": "fvlnlV6LFAfl"
   },
   "source": [
    "**c.** Separate the independent variables (X) and dependent variable (Price_Lakhs)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ypNrxHeyFAfl",
   "metadata": {
    "id": "ypNrxHeyFAfl"
   },
   "outputs": [],
   "source": [
    "X_house = df_encoded.drop('Price_Lakhs', axis=1)\n",
    "y_house = df_encoded['Price_Lakhs']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7R-8Fi8RFAfl",
   "metadata": {
    "id": "7R-8Fi8RFAfl"
   },
   "source": [
    "**d.** Split the dataset into 80% training and 20% testing datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "l_jDnU88FAfl",
   "metadata": {
    "id": "l_jDnU88FAfl"
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X_house, y_house, test_size=0.2, random_state=0\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "RKgLEh_GFAfl",
   "metadata": {
    "id": "RKgLEh_GFAfl"
   },
   "source": [
    "**e.** Train a Multiple Linear Regression model using the training dataset (scikit-learn)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "uWkF5ageFAfl",
   "metadata": {
    "id": "uWkF5ageFAfl"
   },
   "outputs": [],
   "source": [
    "house_model = LinearRegression()\n",
    "house_model.fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "JrnESvIHFAfl",
   "metadata": {
    "id": "JrnESvIHFAfl"
   },
   "source": [
    "**f.** Predict the house prices for the test dataset and evaluate the model using MSE and R2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "kLana3iSFAfl",
   "metadata": {
    "id": "kLana3iSFAfl"
   },
   "outputs": [],
   "source": [
    "y_pred_house = house_model.predict(X_test)\n",
    "\n",
    "mse_full = mean_squared_error(y_test, y_pred_house)\n",
    "r2_full = r2_score(y_test, y_pred_house)\n",
    "\n",
    "print(\"MSE (all features) =\", mse_full)\n",
    "print(\"R2 (all features)  =\", r2_full)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "q6e-LUbpFAfl",
   "metadata": {
    "id": "q6e-LUbpFAfl"
   },
   "source": [
    "**g.** Plot a correlation heatmap for the dataset. Based on the correlation values, identify the features that have the strongest relationship with the target variable (Price_Lakhs). Create a reduced feature set using these highly correlated features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "E7JecFzzFAfl",
   "metadata": {
    "id": "E7JecFzzFAfl"
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(10, 8))\n",
    "sns.heatmap(df_encoded.corr(), annot=True, fmt=\".2f\")\n",
    "plt.title(\"Correlation Heatmap\")\n",
    "plt.show()\n",
    "\n",
    "target_corr = df_encoded.corr()['Price_Lakhs'].drop('Price_Lakhs').sort_values(key=abs, ascending=False)\n",
    "print(\"Correlation with Price_Lakhs:\\n\", target_corr)\n",
    "\n",
    "threshold = 0.5\n",
    "reduced_features = target_corr[abs(target_corr) > threshold].index.tolist()\n",
    "print(\"\\nSelected (highly correlated) features:\", reduced_features)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pTG9j-pxFAfm",
   "metadata": {
    "id": "pTG9j-pxFAfm"
   },
   "source": [
    "**h.** Retrain the Multiple Linear Regression model using the reduced feature set and evaluate it using MSE and R2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4aakZ4bvFAfm",
   "metadata": {
    "id": "4aakZ4bvFAfm"
   },
   "outputs": [],
   "source": [
    "X_train_reduced = X_train[reduced_features]\n",
    "X_test_reduced = X_test[reduced_features]\n",
    "\n",
    "reduced_model = LinearRegression()\n",
    "reduced_model.fit(X_train_reduced, y_train)\n",
    "\n",
    "y_pred_reduced = reduced_model.predict(X_test_reduced)\n",
    "\n",
    "mse_reduced = mean_squared_error(y_test, y_pred_reduced)\n",
    "r2_reduced = r2_score(y_test, y_pred_reduced)\n",
    "\n",
    "print(\"MSE (reduced features) =\", mse_reduced)\n",
    "print(\"R2 (reduced features)  =\", r2_reduced)\n",
    "\n",
    "print(\"\\nComparison\")\n",
    "print(\"Full feature set    - MSE:\", mse_full, \"& R2:\", r2_full)\n",
    "print(\"Reduced feature set - MSE:\", mse_reduced, \"& R2:\", r2_reduced)\n",
    "\n",
    "if mse_reduced < mse_full:\n",
    "    print(\"\\nFeature selection IMPROVED model performance (lower MSE).\")\n",
    "else:\n",
    "    print(\"\\nFeature selection REDUCED model performance (higher MSE).\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "PJb5fAAKeXQY",
   "metadata": {
    "id": "PJb5fAAKeXQY"
   },
   "source": [
    "## Q1. Logistic Regression from Scratch (Obesity Prediction)\n",
    "Build a classification model to predict whether a person is obese or not, using Age and Weight (Y = 1 indicates obese)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ZbAYD-NaeXQY",
   "metadata": {
    "id": "ZbAYD-NaeXQY"
   },
   "source": [
    "**a.** Load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "V_NMURB9eXQZ",
   "metadata": {
    "id": "V_NMURB9eXQZ"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "data = np.array([\n",
    "    [12, 56, 1],\n",
    "    [45, 65, 0],\n",
    "    [30, 79, 1],\n",
    "    [23, 98, 1],\n",
    "    [17, 45, 0],\n",
    "    [32, 58, 0],\n",
    "    [33, 81, 1],\n",
    "    [34, 50, 0],\n",
    "    [21, 49, 0]\n",
    "], dtype=float)\n",
    "\n",
    "X = data[:, :2]\n",
    "Y = data[:, 2]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "FC9h9v4SeXQZ",
   "metadata": {
    "id": "FC9h9v4SeXQZ"
   },
   "source": [
    "**b.** Build a Logistic Regression model for the above task (without using the scikit-learn function). Print all the model parameters after training for 10 epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "oPVm845KeXQa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "oPVm845KeXQa",
    "outputId": "ffc507b8-bab5-44a2-cfa5-5c5d730dcd70"
   },
   "outputs": [],
   "source": [
    "X_mean = X.mean(axis=0)\n",
    "X_std = X.std(axis=0)\n",
    "X_norm = (X - X_mean) / X_std\n",
    "\n",
    "X_b = np.c_[np.ones((X_norm.shape[0], 1)), X_norm]\n",
    "\n",
    "def sigmoid(z):\n",
    "    return 1 / (1 + np.exp(-z))\n",
    "\n",
    "weights = np.zeros(X_b.shape[1])\n",
    "lr = 0.5\n",
    "epochs = 10\n",
    "m = len(Y)\n",
    "\n",
    "for epoch in range(epochs):\n",
    "    z = X_b @ weights\n",
    "    predictions = sigmoid(z)\n",
    "    gradient = (X_b.T @ (predictions - Y)) / m\n",
    "    weights -= lr * gradient\n",
    "    print(f\"Epoch {epoch + 1}: weights = {weights}\")\n",
    "\n",
    "print(\"\\nFinal model parameters after 10 epochs:\")\n",
    "print(\"Intercept (beta0) =\", weights[0])\n",
    "print(\"Coefficient : Age (beta1)    =\", weights[1])\n",
    "print(\"Coefficient : Weight (beta2) =\", weights[2])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "kkjDtDXPeXQa",
   "metadata": {
    "id": "kkjDtDXPeXQa"
   },
   "source": [
    "**c.** Test the model performance with the test dataset: (20, 55, 0), (30, 77, 1), (45, 61, 0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "Zfc0R37qeXQa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Zfc0R37qeXQa",
    "outputId": "d02785b6-664d-42cf-d106-61321e090c10"
   },
   "outputs": [],
   "source": [
    "test_data = np.array([\n",
    "    [20, 55, 0],\n",
    "    [30, 77, 1],\n",
    "    [45, 61, 0]\n",
    "], dtype=float)\n",
    "\n",
    "X_test_raw = test_data[:, :2]\n",
    "Y_test = test_data[:, 2]\n",
    "\n",
    "X_test_norm = (X_test_raw - X_mean) / X_std\n",
    "X_test_b = np.c_[np.ones((X_test_norm.shape[0], 1)), X_test_norm]\n",
    "\n",
    "test_probs = sigmoid(X_test_b @ weights)\n",
    "test_preds = (test_probs >= 0.5).astype(int)\n",
    "\n",
    "for i in range(len(test_data)):\n",
    "    print(f\"Age={int(X_test_raw[i,0])}, Weight={int(X_test_raw[i,1])} : \"\n",
    "          f\"Probability={test_probs[i]:.4f}, Predicted={test_preds[i]}, Actual={int(Y_test[i])}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cSeDl99JeXQa",
   "metadata": {
    "id": "cSeDl99JeXQa"
   },
   "source": [
    "**d.** Count the correct predictions made by your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "taUsNh7beXQa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "taUsNh7beXQa",
    "outputId": "9d3ea066-568b-4fa0-ffa4-58b8eabe8595"
   },
   "outputs": [],
   "source": [
    "correct_predictions = np.sum(test_preds == Y_test)\n",
    "print(f\"Correct predictions: {correct_predictions} out of {len(Y_test)}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "V6XF7QvweXQb",
   "metadata": {
    "id": "V6XF7QvweXQb"
   },
   "source": [
    "## Q2. Diabetes Prediction using Logistic Regression\n",
    "Use the Diabetes Prediction Dataset (`diabetes_prediction_dataset.csv`), which includes `gender`, `age`, `hypertension`, `heart_disease`, `smoking_history`, `bmi`, `HbA1c_level`, `blood_glucose_level`, and `diabetes` (target)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Wc4T0Od7eXQb",
   "metadata": {
    "id": "Wc4T0Od7eXQb"
   },
   "source": [
    "**a.** Load the dataset using pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "-klIsKFteXQb",
   "metadata": {
    "id": "-klIsKFteXQb"
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"diabetes_prediction_dataset.csv\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "IDSRDfK5eXQb",
   "metadata": {
    "id": "IDSRDfK5eXQb"
   },
   "source": [
    "**b.** Display the first five records and check for missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "-wbNYd1ZeXQb",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 414
    },
    "id": "-wbNYd1ZeXQb",
    "outputId": "4000c78e-0c11-4897-af3d-dbf2e0eaca66"
   },
   "outputs": [],
   "source": [
    "display(df.head())\n",
    "print(\"\\nMissing values per column:\\n\", df.isnull().sum())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vSSIJXfQeXQb",
   "metadata": {
    "id": "vSSIJXfQeXQb"
   },
   "source": [
    "**c.** Perform suitable preprocessing:\n",
    "- Convert categorical features into numerical values using One-Hot Encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "SbrTS8b2eXQb",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 244
    },
    "id": "SbrTS8b2eXQb",
    "outputId": "67eb9950-7d6c-469d-b2a9-2e44471c3058"
   },
   "outputs": [],
   "source": [
    "df_encoded = pd.get_dummies(df, columns=['gender', 'smoking_history'], drop_first=True)\n",
    "df_encoded.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "XSgqT_tjeXQb",
   "metadata": {
    "id": "XSgqT_tjeXQb"
   },
   "source": [
    "**d.** Separate input features (X) and target variable (y)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "skaL6GApeXQb",
   "metadata": {
    "id": "skaL6GApeXQb"
   },
   "outputs": [],
   "source": [
    "X_diab = df_encoded.drop('diabetes', axis=1)\n",
    "y_diab = df_encoded['diabetes']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "IZyQV3ZXeXQb",
   "metadata": {
    "id": "IZyQV3ZXeXQb"
   },
   "source": [
    "**e.** Split the dataset into 80% training and 20% testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ErveVMrYeXQb",
   "metadata": {
    "id": "ErveVMrYeXQb"
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X_diab, y_diab, test_size=0.2, random_state=0\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "XN93Y7hmeXQb",
   "metadata": {
    "id": "XN93Y7hmeXQb"
   },
   "source": [
    "**f.** Train a Logistic Regression classifier using Scikit-Learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9-rKc6JeeXQb",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 80
    },
    "id": "9-rKc6JeeXQb",
    "outputId": "bc210b76-5b45-4f39-8c2b-b3b86c9a1261"
   },
   "outputs": [],
   "source": [
    "log_reg = LogisticRegression(max_iter=1000)\n",
    "log_reg.fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "DC2oPKc_eXQc",
   "metadata": {
    "id": "DC2oPKc_eXQc"
   },
   "source": [
    "**g.** Predict the class labels for the test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "LtfYCA_6eXQc",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "LtfYCA_6eXQc",
    "outputId": "c03ef1a6-2605-4d77-b423-132dc5cbb4ee"
   },
   "outputs": [],
   "source": [
    "y_pred = log_reg.predict(X_test)\n",
    "print(y_pred[:20])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "DBQXCJ5aeXQc",
   "metadata": {
    "id": "DBQXCJ5aeXQc"
   },
   "source": [
    "**h.** Evaluate the model using Accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "zVDFRHiJeXQc",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "zVDFRHiJeXQc",
    "outputId": "97f59fbe-9a4c-4c75-c043-9704bcdcc831"
   },
   "outputs": [],
   "source": [
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "print(\"Accuracy =\", accuracy)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5zYPC0ZVeXQc",
   "metadata": {
    "id": "5zYPC0ZVeXQc"
   },
   "source": [
    "**i.** Predict diabetes for the following new patients:\n",
    "\n",
    "Age\tGender\tHypertension\tHeart Disease\tSmoking\tBMI\tHbA1c\tGlucose\n",
    "45\tFemale\t0\t0\tnever\t29.5\t6.1\t145\n",
    "30\tMale\t0\t0\tNo Info\t22.4\t5.2\t92\n",
    "62\tMale\t1\t1\tformer\t34.2\t7.5\t190\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hvibAet5eXQc",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "hvibAet5eXQc",
    "outputId": "b89a0bcf-6fd1-4ca0-ab20-9a2f3219d451"
   },
   "outputs": [],
   "source": [
    "new_patients = pd.DataFrame([\n",
    "    {'gender': 'Female', 'age': 45, 'hypertension': 0, 'heart_disease': 0,\n",
    "     'smoking_history': 'never', 'bmi': 29.5, 'HbA1c_level': 6.1, 'blood_glucose_level': 145},\n",
    "    {'gender': 'Male', 'age': 30, 'hypertension': 0, 'heart_disease': 0,\n",
    "     'smoking_history': 'No Info', 'bmi': 22.4, 'HbA1c_level': 5.2, 'blood_glucose_level': 92},\n",
    "    {'gender': 'Male', 'age': 62, 'hypertension': 1, 'heart_disease': 1,\n",
    "     'smoking_history': 'former', 'bmi': 34.2, 'HbA1c_level': 7.5, 'blood_glucose_level': 190}\n",
    "])\n",
    "\n",
    "new_patients_encoded = pd.get_dummies(new_patients, columns=['gender', 'smoking_history'], drop_first=True)\n",
    "new_patients_encoded = new_patients_encoded.reindex(columns=X_diab.columns, fill_value=0)\n",
    "\n",
    "new_predictions = log_reg.predict(new_patients_encoded)\n",
    "new_probabilities = log_reg.predict_proba(new_patients_encoded)[:, 1]\n",
    "\n",
    "for i in range(len(new_patients)):\n",
    "    print(f\"Patient {i + 1}: Predicted diabetes = {new_predictions[i]} \"\n",
    "          f\"(probability = {new_probabilities[i]:.4f})\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "H9wRxPTpE0ov",
   "metadata": {
    "id": "H9wRxPTpE0ov"
   },
   "source": [
    "## Breast Cancer Classification using KNN\n",
    "Use the Breast Cancer Wisconsin Dataset available in Scikit-Learn to build a KNN classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "BJGgIlDjE0ov",
   "metadata": {
    "id": "BJGgIlDjE0ov"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix, classification_report\n",
    "\n",
    "data = load_breast_cancer()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2J9g6EsQE0ow",
   "metadata": {
    "id": "2J9g6EsQE0ow"
   },
   "source": [
    "**a.** Load the dataset and answer the following:\n",
    "- Number of samples\n",
    "- Number of features\n",
    "- Number of classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "UqwOKWm9E0ow",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UqwOKWm9E0ow",
    "outputId": "d39bb669-a5db-4748-a5b6-fe20864fe308"
   },
   "outputs": [],
   "source": [
    "n_samples, n_features = data.data.shape\n",
    "n_classes = len(data.target_names)\n",
    "\n",
    "print(\"Number of samples  =\", n_samples)\n",
    "print(\"Number of features =\", n_features)\n",
    "print(\"Number of classes  =\", n_classes)\n",
    "print(\"Class names        =\", list(data.target_names))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "idWO5_cnE0ow",
   "metadata": {
    "id": "idWO5_cnE0ow"
   },
   "source": [
    "**b.** Check whether there are missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "isv7PUJQE0ow",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "isv7PUJQE0ow",
    "outputId": "81bfd1f2-e722-4b84-adb2-0fdc78a260ec"
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame(data.data, columns=data.feature_names)\n",
    "df['target'] = data.target\n",
    "\n",
    "print(\"Missing values per column:\\n\", df.isnull().sum().sum(), \"total missing values\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ggkbYPeE0ox",
   "metadata": {
    "id": "0ggkbYPeE0ox"
   },
   "source": [
    "**c.** Display the first five rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3N5Ija_aE0ox",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 270
    },
    "id": "3N5Ija_aE0ox",
    "outputId": "0bb25ac0-5ee4-4080-8d49-10cda21afdda"
   },
   "outputs": [],
   "source": [
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "MWSKuWGBE0ox",
   "metadata": {
    "id": "MWSKuWGBE0ox"
   },
   "source": [
    "**d.** Perform preprocessing:\n",
    "- Split the dataset (80:20).\n",
    "- Standardize the features using `StandardScaler`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "LwSfoITDE0ox",
   "metadata": {
    "id": "LwSfoITDE0ox"
   },
   "outputs": [],
   "source": [
    "X = data.data\n",
    "y = data.target\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=0\n",
    ")\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X_train_scaled = scaler.fit_transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ldvi3GlPE0ox",
   "metadata": {
    "id": "ldvi3GlPE0ox"
   },
   "source": [
    "**e.** Train a KNN classifier using K=3, K=5 and K=7 (Euclidean distance, `p=2`, `metric='minkowski'`). Report the results for each K-value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f1Ev-TjTE0ox",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "f1Ev-TjTE0ox",
    "outputId": "6327a1a6-9d28-4d34-c9c6-cd151db8b610"
   },
   "outputs": [],
   "source": [
    "k_values = [3, 5, 7]\n",
    "euclidean_results = {}\n",
    "\n",
    "for k in k_values:\n",
    "    knn = KNeighborsClassifier(n_neighbors=k, p=2, metric='minkowski')\n",
    "    knn.fit(X_train_scaled, y_train)\n",
    "    y_pred = knn.predict(X_test_scaled)\n",
    "\n",
    "    acc = accuracy_score(y_test, y_pred)\n",
    "    euclidean_results[k] = acc\n",
    "\n",
    "    print(f\"\\n K = {k} (Euclidean distance) \")\n",
    "    print(\"Accuracy =\", acc)\n",
    "    print(\"Confusion Matrix:\\n\", confusion_matrix(y_test, y_pred))\n",
    "    print(\"Classification Report:\\n\", classification_report(y_test, y_pred, target_names=data.target_names))\n",
    "\n",
    "print(\"\\nSummary (Euclidean distance):\")\n",
    "for k, acc in euclidean_results.items():\n",
    "    print(f\"K = {k} -> Accuracy = {acc:.4f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "VOlH8CSIE0ox",
   "metadata": {
    "id": "VOlH8CSIE0ox"
   },
   "source": [
    "**f.** Change the distance metric to Manhattan Distance and observe the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "R7fVetxlE0oy",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "R7fVetxlE0oy",
    "outputId": "4ec20865-6706-415c-bcb2-ec4fc855493f"
   },
   "outputs": [],
   "source": [
    "manhattan_results = {}\n",
    "\n",
    "for k in k_values:\n",
    "    knn_manhattan = KNeighborsClassifier(n_neighbors=k, p=1, metric='minkowski')\n",
    "    knn_manhattan.fit(X_train_scaled, y_train)\n",
    "    y_pred_manhattan = knn_manhattan.predict(X_test_scaled)\n",
    "\n",
    "    acc_manhattan = accuracy_score(y_test, y_pred_manhattan)\n",
    "    manhattan_results[k] = acc_manhattan\n",
    "\n",
    "    print(f\"\\nK = {k} (Manhattan distance)\")\n",
    "    print(\"Accuracy =\", acc_manhattan)\n",
    "    print(\"Confusion Matrix:\\n\", confusion_matrix(y_test, y_pred_manhattan))\n",
    "\n",
    "print(\"\\nComparison of Euclidean vs Manhattan distance:\")\n",
    "print(f\"{'K':<5}{'Euclidean Acc':<18}{'Manhattan Acc':<18}\")\n",
    "for k in k_values:\n",
    "    print(f\"{k:<5}{euclidean_results[k]:<18.4f}{manhattan_results[k]:<18.4f}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8a9TpKxN3rO",
   "metadata": {
    "id": "a8a9TpKxN3rO"
   },
   "source": [
    "## Q1. Categorical Naïve Bayes on the PlayTennis Dataset\n",
    "Implement a Naïve Bayes Classifier (`CategoricalNB`) on the `PlayTennis.csv` dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mARDr00rN3rO",
   "metadata": {
    "id": "mARDr00rN3rO"
   },
   "source": [
    "**a.** Load the PlayTennis dataset from a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aMbkhTilN3rP",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "aMbkhTilN3rP",
    "outputId": "5258bdac-4527-4136-9795-7e02a01875c3"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.naive_bayes import CategoricalNB, GaussianNB\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "\n",
    "df = pd.read_csv(\"PlayTennis.csv\")\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1HGmh7EfN3rP",
   "metadata": {
    "id": "1HGmh7EfN3rP"
   },
   "source": [
    "**b.** Encode categorical attributes. `CategoricalNB` is designed for categorical features, but it expects each category to be represented as an integer. The numbers are just labels, not numerical quantities — `CategoricalNB` treats them as distinct categories rather than assuming that 2 is greater than 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0MVc7OhNN3rP",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "0MVc7OhNN3rP",
    "outputId": "3af21cc2-73ef-4fd9-81e7-930d3557d956"
   },
   "outputs": [],
   "source": [
    "encoder = LabelEncoder()\n",
    "df_encoded = df.copy()\n",
    "\n",
    "for column in df_encoded.columns:\n",
    "    df_encoded[column] = encoder.fit_transform(df_encoded[column])\n",
    "\n",
    "df_encoded.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3X8MGOqN3rP",
   "metadata": {
    "id": "d3X8MGOqN3rP"
   },
   "source": [
    "**c.** Split the dataset into 70% training and 30% testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "U4PBgFfAN3rQ",
   "metadata": {
    "id": "U4PBgFfAN3rQ"
   },
   "outputs": [],
   "source": [
    "X_tennis = df_encoded.drop('play', axis=1)\n",
    "y_tennis = df_encoded['play']\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X_tennis, y_tennis, test_size=0.3, random_state=0\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "JBT6u5-6N3rQ",
   "metadata": {
    "id": "JBT6u5-6N3rQ"
   },
   "source": [
    "**d.** Train a Categorical Naïve Bayes classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "g8wEOYwNN3rQ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 80
    },
    "id": "g8wEOYwNN3rQ",
    "outputId": "810e4700-6103-47b3-a152-afd0608eaf0e"
   },
   "outputs": [],
   "source": [
    "cat_nb = CategoricalNB()\n",
    "cat_nb.fit(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27zMWdniN3rQ",
   "metadata": {
    "id": "27zMWdniN3rQ"
   },
   "source": [
    "**e.** Predict the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "oipUa9WMN3rQ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "oipUa9WMN3rQ",
    "outputId": "ef1c87bc-1a22-4d27-f86d-299481c39ec6"
   },
   "outputs": [],
   "source": [
    "y_pred_tennis = cat_nb.predict(X_test)\n",
    "print(y_pred_tennis)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "nlcwdoS0N3rQ",
   "metadata": {
    "id": "nlcwdoS0N3rQ"
   },
   "source": [
    "**f.** Evaluate the model using Accuracy and Confusion Matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "nCmLUxtiN3rQ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nCmLUxtiN3rQ",
    "outputId": "fe5648e8-59ea-45ac-d85c-92adda5609a8"
   },
   "outputs": [],
   "source": [
    "accuracy_tennis = accuracy_score(y_test, y_pred_tennis)\n",
    "cm_tennis = confusion_matrix(y_test, y_pred_tennis)\n",
    "\n",
    "print(\"Accuracy =\", accuracy_tennis)\n",
    "print(\"Confusion Matrix:\\n\", cm_tennis)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bYrmR6k4N3rQ",
   "metadata": {
    "id": "bYrmR6k4N3rQ"
   },
   "source": [
    "## Q2. Gaussian Naïve Bayes on the Breast Cancer Dataset\n",
    "Implement a Naïve Bayes Classifier (`GaussianNB`) on the Breast Cancer Wisconsin dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tGtn1jnxN3rR",
   "metadata": {
    "id": "tGtn1jnxN3rR"
   },
   "source": [
    "**a.** Load the Breast Cancer Wisconsin Dataset available in scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "-25k_8duN3rR",
   "metadata": {
    "id": "-25k_8duN3rR"
   },
   "outputs": [],
   "source": [
    "data = load_breast_cancer()\n",
    "X_cancer = data.data\n",
    "y_cancer = data.target\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0sNTrt2HN3rR",
   "metadata": {
    "id": "0sNTrt2HN3rR"
   },
   "source": [
    "**b.** Split the dataset into 70% training and 30% testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0jJF0g2BN3rR",
   "metadata": {
    "id": "0jJF0g2BN3rR"
   },
   "outputs": [],
   "source": [
    "X_train_c, X_test_c, y_train_c, y_test_c = train_test_split(\n",
    "    X_cancer, y_cancer, test_size=0.3, random_state=0\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "n-IJG_nQN3rR",
   "metadata": {
    "id": "n-IJG_nQN3rR"
   },
   "source": [
    "**c.** Train a Gaussian Naïve Bayes classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "pUC0V2oHN3rS",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 80
    },
    "id": "pUC0V2oHN3rS",
    "outputId": "5d4c3485-e46c-40d7-b186-1fdf53c4ab3f"
   },
   "outputs": [],
   "source": [
    "gauss_nb = GaussianNB()\n",
    "gauss_nb.fit(X_train_c, y_train_c)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ClbcpJdYN3rS",
   "metadata": {
    "id": "ClbcpJdYN3rS"
   },
   "source": [
    "**d.** Predict the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8X5qfSh4N3rS",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8X5qfSh4N3rS",
    "outputId": "3a366c14-e5bf-4774-a8ce-706596b43dce"
   },
   "outputs": [],
   "source": [
    "y_pred_cancer = gauss_nb.predict(X_test_c)\n",
    "print(y_pred_cancer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "YTmKCK38N3rS",
   "metadata": {
    "id": "YTmKCK38N3rS"
   },
   "source": [
    "**e.** Evaluate the model using Accuracy and Confusion Matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "vcHhfInlN3rS",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vcHhfInlN3rS",
    "outputId": "71108484-e7c4-48e1-b64b-f32b19c80745"
   },
   "outputs": [],
   "source": [
    "accuracy_cancer = accuracy_score(y_test_c, y_pred_cancer)\n",
    "cm_cancer = confusion_matrix(y_test_c, y_pred_cancer)\n",
    "\n",
    "print(\"Accuracy =\", accuracy_cancer)\n",
    "print(\"Confusion Matrix:\\n\", cm_cancer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "kwNZYkb4N3rS",
   "metadata": {
    "id": "kwNZYkb4N3rS"
   },
   "source": [
    "**f.** Display the predicted class (Malignant or Benign) for the first five test samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "Oo6lrAxJN3rS",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Oo6lrAxJN3rS",
    "outputId": "90e800df-9f73-483d-ded5-23c7a30ba5a8"
   },
   "outputs": [],
   "source": [
    "for i in range(5):\n",
    "    predicted_label = data.target_names[y_pred_cancer[i]]\n",
    "    print(f\"Sample {i + 1}: Predicted class = {predicted_label.capitalize()}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rqYogH0MOXz_",
   "metadata": {
    "id": "rqYogH0MOXz_"
   },
   "source": [
    "## Q1. Decision Tree Classifier on the PlayTennis Dataset\n",
    "Implement a Decision Tree Classifier (scikit-learn) on the `PlayTennis.csv` dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ie-gvTZeOX0A",
   "metadata": {
    "id": "ie-gvTZeOX0A"
   },
   "source": [
    "**a.** Load the PlayTennis dataset from a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "vYBs7ZE5OX0A",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "vYBs7ZE5OX0A",
    "outputId": "d55e31d5-c0ba-414a-d4dc-01f0140de340"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier, plot_tree\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "\n",
    "df = pd.read_csv(\"PlayTennis.csv\")\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "p1mafiakOX0A",
   "metadata": {
    "id": "p1mafiakOX0A"
   },
   "source": [
    "**b.** Use the entire dataset except the last row as training and the last row for testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "_6rLzn1HOX0A",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 457
    },
    "id": "_6rLzn1HOX0A",
    "outputId": "5e242dba-ec70-4803-a8e6-226f721a1b3f"
   },
   "outputs": [],
   "source": [
    "encoder = LabelEncoder()\n",
    "df_encoded = df.copy()\n",
    "\n",
    "for column in df_encoded.columns:\n",
    "    df_encoded[column] = encoder.fit_transform(df_encoded[column])\n",
    "\n",
    "X_tennis = df_encoded.drop('play', axis=1)\n",
    "y_tennis = df_encoded['play']\n",
    "\n",
    "X_train = X_tennis.iloc[:-1]\n",
    "y_train = y_tennis.iloc[:-1]\n",
    "X_test = X_tennis.iloc[[-1]]\n",
    "y_test = y_tennis.iloc[[-1]]\n",
    "\n",
    "X_train\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2YcupDtiOX0B",
   "metadata": {
    "id": "2YcupDtiOX0B"
   },
   "source": [
    "**c.** Fit a decision tree model and draw the tree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "u0gdYtFPOX0B",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 653
    },
    "id": "u0gdYtFPOX0B",
    "outputId": "c58438c3-a778-474d-e21e-afccda5b41c6"
   },
   "outputs": [],
   "source": [
    "dt_tennis = DecisionTreeClassifier(criterion='entropy', random_state=0)\n",
    "dt_tennis.fit(X_train, y_train)\n",
    "\n",
    "plt.figure(figsize=(14, 8))\n",
    "plot_tree(dt_tennis, feature_names=X_tennis.columns, class_names=['no', 'yes'], filled=True)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "EeeWbTcyOX0B",
   "metadata": {
    "id": "EeeWbTcyOX0B"
   },
   "source": [
    "**d.** Test the model with the testing sample and report the answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dF_S7AuBOX0B",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dF_S7AuBOX0B",
    "outputId": "5ddff370-2352-4f13-e47b-c8bd607311e4"
   },
   "outputs": [],
   "source": [
    "y_pred_tennis = dt_tennis.predict(X_test)\n",
    "\n",
    "predicted_label = encoder.fit(df['play']).inverse_transform(y_pred_tennis)\n",
    "actual_label = encoder.inverse_transform(y_test)\n",
    "\n",
    "print(\"Test sample:\\n\", df.iloc[[-1]])\n",
    "print(\"\\nPredicted class =\", predicted_label[0])\n",
    "print(\"Actual class    =\", actual_label[0])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vgCD8aw4OX0B",
   "metadata": {
    "id": "vgCD8aw4OX0B"
   },
   "source": [
    "## Q2. Decision Tree Classifier on the Breast Cancer Dataset\n",
    "Implement a Decision Tree Classifier on the Breast Cancer Wisconsin dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "YtmCn07YOX0B",
   "metadata": {
    "id": "YtmCn07YOX0B"
   },
   "source": [
    "**a.** Load the Breast Cancer Wisconsin Dataset available in scikit-learn."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ug9gQ53OX0B",
   "metadata": {
    "id": "9ug9gQ53OX0B"
   },
   "outputs": [],
   "source": [
    "data = load_breast_cancer()\n",
    "X_cancer = data.data\n",
    "y_cancer = data.target\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b-xCenEsOX0B",
   "metadata": {
    "id": "b-xCenEsOX0B"
   },
   "source": [
    "**b.** Do encoding of variables if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "lKVjiS6bOX0B",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "lKVjiS6bOX0B",
    "outputId": "e78123d8-2673-45df-f16d-50418c6f8e3e"
   },
   "outputs": [],
   "source": [
    "print(\"Feature dtypes:\", X_cancer.dtype)\n",
    "print(\"Target classes:\", data.target_names)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "xgur4PQfOX0B",
   "metadata": {
    "id": "xgur4PQfOX0B"
   },
   "source": [
    "**c.** Split the dataset into 70% training and 30% testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2FzRVUEoOX0C",
   "metadata": {
    "id": "2FzRVUEoOX0C"
   },
   "outputs": [],
   "source": [
    "X_train_c, X_test_c, y_train_c, y_test_c = train_test_split(\n",
    "    X_cancer, y_cancer, test_size=0.3, random_state=0\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "SHghbPIcOX0C",
   "metadata": {
    "id": "SHghbPIcOX0C"
   },
   "source": [
    "**d.** Train a Decision Tree Classifier (scikit-learn)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "i4JwaVVVOX0C",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 80
    },
    "id": "i4JwaVVVOX0C",
    "outputId": "a1eca70a-c533-48ef-f78e-ff013ca6d578"
   },
   "outputs": [],
   "source": [
    "dt_cancer = DecisionTreeClassifier(criterion='entropy', random_state=0)\n",
    "dt_cancer.fit(X_train_c, y_train_c)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bZVq8P71OX0C",
   "metadata": {
    "id": "bZVq8P71OX0C"
   },
   "source": [
    "**e.** Predict the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "G8wuD_FvOX0C",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "G8wuD_FvOX0C",
    "outputId": "10ff2ef4-aa13-4d48-c8d5-421c16062300"
   },
   "outputs": [],
   "source": [
    "y_pred_cancer = dt_cancer.predict(X_test_c)\n",
    "print(y_pred_cancer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59OHO5tOOX0C",
   "metadata": {
    "id": "59OHO5tOOX0C"
   },
   "source": [
    "**f.** Evaluate the model using Accuracy and Confusion Matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8QwFGBITOX0C",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8QwFGBITOX0C",
    "outputId": "4a46145e-fb4b-452a-809a-82f84e9d9411"
   },
   "outputs": [],
   "source": [
    "accuracy_cancer = accuracy_score(y_test_c, y_pred_cancer)\n",
    "cm_cancer = confusion_matrix(y_test_c, y_pred_cancer)\n",
    "\n",
    "print(\"Accuracy =\", accuracy_cancer)\n",
    "print(\"Confusion Matrix:\\n\", cm_cancer)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "HvtvMHgbeVOE",
   "metadata": {
    "id": "HvtvMHgbeVOE"
   },
   "source": [
    "## Q1. Single-Layer Perceptron for the AND Logic Gate\n",
    "Implement a single-layer perceptron to learn the AND logic gate (no scikit-learn function). Start with random initial weights, train the perceptron, print the weights after training, and test for the inputs (1,1) and (1,0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sxfsv5z4eVOF",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sxfsv5z4eVOF",
    "outputId": "65983555-9333-4261-92f2-150068ae2ae9"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "X = np.array([\n",
    "    [0, 0],\n",
    "    [0, 1],\n",
    "    [1, 0],\n",
    "    [1, 1]\n",
    "])\n",
    "\n",
    "Y = np.array([0, 0, 0, 1])\n",
    "\n",
    "np.random.seed(1)\n",
    "weights = np.random.uniform(-1, 1, size=2)\n",
    "bias = np.random.uniform(-1, 1)\n",
    "\n",
    "print(\"Initial weights =\", weights)\n",
    "print(\"Initial bias    =\", bias)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4y5A7LVzeVOF",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "4y5A7LVzeVOF",
    "outputId": "edb02fae-ea7f-4714-949d-c9d3b7ffdd02"
   },
   "outputs": [],
   "source": [
    "def step_function(z):\n",
    "    return 1 if z >= 0 else 0\n",
    "\n",
    "\n",
    "def train_perceptron(X, Y, weights, bias, lr=0.1, epochs=20):\n",
    "    for epoch in range(epochs):\n",
    "        for i in range(len(X)):\n",
    "            z = np.dot(X[i], weights) + bias\n",
    "            prediction = step_function(z)\n",
    "            error = Y[i] - prediction\n",
    "            weights = weights + lr * error * X[i]\n",
    "            bias = bias + lr * error\n",
    "    return weights, bias\n",
    "\n",
    "\n",
    "weights, bias = train_perceptron(X, Y, weights, bias)\n",
    "\n",
    "print(\"Weights after training =\", weights)\n",
    "print(\"Bias after training    =\", bias)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "mtIQoTfdeVOF",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "mtIQoTfdeVOF",
    "outputId": "f6b19d4a-6103-4936-933a-8a0eca4f8b60"
   },
   "outputs": [],
   "source": [
    "def predict(x, weights, bias):\n",
    "    z = np.dot(x, weights) + bias\n",
    "    return step_function(z)\n",
    "\n",
    "\n",
    "test_inputs = [(1, 1), (1, 0)]\n",
    "\n",
    "for x in test_inputs:\n",
    "    result = predict(np.array(x), weights, bias)\n",
    "    print(f\"Input {x} -> Output {result}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "uzsfFYhReVOG",
   "metadata": {
    "id": "uzsfFYhReVOG"
   },
   "source": [
    "## Q2. Multilayer Perceptron for Wine Quality Prediction\n",
    "Implement a Multilayer Perceptron (MLP) model for predicting wine quality based on physicochemical properties such as `fixed_acidity`, `volatile_acidity`, `citric_acid`, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "JR_fJ5CaeVOG",
   "metadata": {
    "id": "JR_fJ5CaeVOG"
   },
   "source": [
    "**a.** Load the Wine dataset (`wine.csv`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "AtTwXNAEeVOG",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "AtTwXNAEeVOG",
    "outputId": "27c48cbf-d543-49cf-881d-4a75c2bac1f2"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix\n",
    "\n",
    "df = pd.read_csv(\"wine.csv\")\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "jBuYZoHyeVOG",
   "metadata": {
    "id": "jBuYZoHyeVOG"
   },
   "source": [
    "**b.** Split the dataset into 80% training and 20% testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ninifKHdeVOG",
   "metadata": {
    "id": "ninifKHdeVOG"
   },
   "outputs": [],
   "source": [
    "X_wine = df.drop('quality', axis=1)\n",
    "y_wine = df['quality']\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X_wine, y_wine, test_size=0.2, random_state=0\n",
    ")\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X_train_scaled = scaler.fit_transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "YIk2mCPseVOG",
   "metadata": {
    "id": "YIk2mCPseVOG"
   },
   "source": [
    "**c.** Train a MLPClassifier (scikit-learn)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6Ei0chFPeVOG",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 80
    },
    "id": "6Ei0chFPeVOG",
    "outputId": "c271ccec-d966-4883-9307-4f479ba272c0"
   },
   "outputs": [],
   "source": [
    "mlp = MLPClassifier(hidden_layer_sizes=(64, 32), max_iter=1000, random_state=0)\n",
    "mlp.fit(X_train_scaled, y_train)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "zWberPJOeVOH",
   "metadata": {
    "id": "zWberPJOeVOH"
   },
   "source": [
    "**d.** Predict the test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "kCPzjIu9eVOH",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kCPzjIu9eVOH",
    "outputId": "4d54ecdb-bfc4-4c9a-e6ad-6575c898da09"
   },
   "outputs": [],
   "source": [
    "y_pred = mlp.predict(X_test_scaled)\n",
    "print(y_pred)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "-LL6C97WeVOH",
   "metadata": {
    "id": "-LL6C97WeVOH"
   },
   "source": [
    "**e.** Evaluate the model using Accuracy and Confusion Matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1iWG-O-reVOH",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1iWG-O-reVOH",
    "outputId": "ad166697-2a7e-4ba1-b7bc-0efc928f313f"
   },
   "outputs": [],
   "source": [
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "cm = confusion_matrix(y_test, y_pred)\n",
    "\n",
    "print(\"Accuracy =\", accuracy)\n",
    "print(\"Confusion Matrix:\\n\", cm)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fed5f723",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.cluster import KMeans, AgglomerativeClustering\n",
    "from sklearn.metrics import rand_score, silhouette_score\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c45f76a7",
   "metadata": {},
   "source": [
    "## Q1. K-Means vs Hierarchical Agglomerative Clustering on Synthetic 2-D Datasets\n",
    "Implement and compare K-Means Clustering and Hierarchical Agglomerative Clustering on three synthetic 2-D datasets and visualize the resulting clusters. For each dataset:\n",
    "\n",
    "**a.** Load the dataset.  \n",
    "**b.** Plot the original data points using a scatter plot.  \n",
    "**c.** Apply K-Means and Hierarchical Agglomerative Clustering with the given k value.  \n",
    "**d.** Plot the clusters obtained from each algorithm (separate plot for each method).  \n",
    "**e.** For the K-Means output, display the cluster centers.  \n",
    "**f.** Observe which algorithm produced more clearly separated clusters visually.  \n",
    "**g.** Calculate and display the Rand Index and Silhouette Score for both algorithms.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "683236ec",
   "metadata": {},
   "source": [
    "### Dataset 1 (ClusteringData1.csv)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e99a722",
   "metadata": {},
   "source": [
    "**a.** Load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d19d9534",
   "metadata": {},
   "outputs": [],
   "source": [
    "df1 = pd.read_csv(\"ClusteringData1.csv\")\n",
    "X1 = df1[['x', 'y']].values\n",
    "y_true1 = df1['target'].values\n",
    "\n",
    "df1.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c773708",
   "metadata": {},
   "outputs": [],
   "source": [
    "**b.** Plot the original data points using a scatter plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96e9b8f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(6, 5))\n",
    "plt.scatter(X1[:, 0], X1[:, 1], c=y_true1, cmap='viridis', s=20)\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Dataset 1 — Original Data\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6ae2c88",
   "metadata": {},
   "source": [
    "**c.** Apply K-Means and Hierarchical Agglomerative Clustering with the given k value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a03e801f",
   "metadata": {},
   "outputs": [],
   "source": [
    "k1 = len(np.unique(y_true1))\n",
    "print(\"k =\", k1)\n",
    "\n",
    "kmeans1 = KMeans(n_clusters=k1, random_state=0, n_init=10)\n",
    "kmeans_labels1 = kmeans1.fit_predict(X1)\n",
    "\n",
    "agglo1 = AgglomerativeClustering(n_clusters=k1)\n",
    "agglo_labels1 = agglo1.fit_predict(X1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "442a814b",
   "metadata": {},
   "source": [
    "**d.** Plot the clusters obtained from each algorithm (separate plot for each method)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7cd054b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(12, 5))\n",
    "\n",
    "axes[0].scatter(X1[:, 0], X1[:, 1], c=kmeans_labels1, cmap='viridis', s=20)\n",
    "axes[0].set_title(\"Dataset 1 — K-Means Clusters\")\n",
    "axes[0].set_xlabel(\"x\")\n",
    "axes[0].set_ylabel(\"y\")\n",
    "\n",
    "axes[1].scatter(X1[:, 0], X1[:, 1], c=agglo_labels1, cmap='viridis', s=20)\n",
    "axes[1].set_title(\"Dataset 1 — Hierarchical Agglomerative Clusters\")\n",
    "axes[1].set_xlabel(\"x\")\n",
    "axes[1].set_ylabel(\"y\")\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73419792",
   "metadata": {},
   "source": [
    "**e.** For the K-Means output, display the cluster centers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7dfa406",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"K-Means cluster centers:\\n\", kmeans1.cluster_centers_)\n",
    "\n",
    "plt.figure(figsize=(6, 5))\n",
    "plt.scatter(X1[:, 0], X1[:, 1], c=kmeans_labels1, cmap='viridis', s=20)\n",
    "plt.scatter(kmeans1.cluster_centers_[:, 0], kmeans1.cluster_centers_[:, 1],\n",
    "            c='red', marker='X', s=200, label='Cluster Centers')\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Dataset 1 — K-Means Cluster Centers\")\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c56a5d3",
   "metadata": {},
   "source": [
    "**f.** Observe which algorithm produced more clearly separated clusters visually."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88bd2d7f",
   "metadata": {},
   "source": [
    "*(Compare the two subplots in part (d) for Dataset 1 — note here which algorithm shows tighter, more clearly separated clusters and which shows more overlap.)*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76e0b738",
   "metadata": {},
   "source": [
    "**g.** Calculate and display the Rand Index and Silhouette Score for both algorithms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9258bdda",
   "metadata": {},
   "outputs": [],
   "source": [
    "rand_kmeans1 = rand_score(y_true1, kmeans_labels1)\n",
    "rand_agglo1 = rand_score(y_true1, agglo_labels1)\n",
    "\n",
    "sil_kmeans1 = silhouette_score(X1, kmeans_labels1)\n",
    "sil_agglo1 = silhouette_score(X1, agglo_labels1)\n",
    "\n",
    "print(\"K-Means      -> Rand Index: {:.4f}, Silhouette Score: {:.4f}\".format(rand_kmeans1, sil_kmeans1))\n",
    "print(\"Hierarchical -> Rand Index: {:.4f}, Silhouette Score: {:.4f}\".format(rand_agglo1, sil_agglo1))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa0f4025",
   "metadata": {},
   "source": [
    "### Dataset 2 (ClusteringData2.csv)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a6600e4",
   "metadata": {},
   "source": [
    "**a.** Load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70e784a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "df2 = pd.read_csv(\"ClusteringData2.csv\")\n",
    "X2 = df2[['x', 'y']].values\n",
    "y_true2 = df2['target'].values\n",
    "\n",
    "df2.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b67a88bc",
   "metadata": {},
   "source": [
    "**b.** Plot the original data points using a scatter plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fbf58004",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(6, 5))\n",
    "plt.scatter(X2[:, 0], X2[:, 1], c=y_true2, cmap='viridis', s=20)\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Dataset 2 — Original Data\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa9d4503",
   "metadata": {},
   "source": [
    "**c.** Apply K-Means and Hierarchical Agglomerative Clustering with the given k value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a661b21b",
   "metadata": {},
   "outputs": [],
   "source": [
    "k2 = len(np.unique(y_true2))\n",
    "print(\"k =\", k2)\n",
    "\n",
    "kmeans2 = KMeans(n_clusters=k2, random_state=0, n_init=10)\n",
    "kmeans_labels2 = kmeans2.fit_predict(X2)\n",
    "\n",
    "agglo2 = AgglomerativeClustering(n_clusters=k2)\n",
    "agglo_labels2 = agglo2.fit_predict(X2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb0225d3",
   "metadata": {},
   "source": [
    "**d.** Plot the clusters obtained from each algorithm (separate plot for each method)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1d42d50",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(12, 5))\n",
    "\n",
    "axes[0].scatter(X2[:, 0], X2[:, 1], c=kmeans_labels2, cmap='viridis', s=20)\n",
    "axes[0].set_title(\"Dataset 2 — K-Means Clusters\")\n",
    "axes[0].set_xlabel(\"x\")\n",
    "axes[0].set_ylabel(\"y\")\n",
    "\n",
    "axes[1].scatter(X2[:, 0], X2[:, 1], c=agglo_labels2, cmap='viridis', s=20)\n",
    "axes[1].set_title(\"Dataset 2 — Hierarchical Agglomerative Clusters\")\n",
    "axes[1].set_xlabel(\"x\")\n",
    "axes[1].set_ylabel(\"y\")\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "373ba1d6",
   "metadata": {},
   "source": [
    "**e.** For the K-Means output, display the cluster centers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfe3ac35",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"K-Means cluster centers:\\n\", kmeans2.cluster_centers_)\n",
    "\n",
    "plt.figure(figsize=(6, 5))\n",
    "plt.scatter(X2[:, 0], X2[:, 1], c=kmeans_labels2, cmap='viridis', s=20)\n",
    "plt.scatter(kmeans2.cluster_centers_[:, 0], kmeans2.cluster_centers_[:, 1],\n",
    "            c='red', marker='X', s=200, label='Cluster Centers')\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Dataset 2 — K-Means Cluster Centers\")\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3212edd",
   "metadata": {},
   "source": [
    "**f.** Observe which algorithm produced more clearly separated clusters visually."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbf4c9ce",
   "metadata": {},
   "source": [
    "*(Compare the two subplots in part (d) for Dataset 2 — note here which algorithm shows tighter, more clearly separated clusters and which shows more overlap.)*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8856a41e",
   "metadata": {},
   "source": [
    "**g.** Calculate and display the Rand Index and Silhouette Score for both algorithms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b605f678",
   "metadata": {},
   "outputs": [],
   "source": [
    "rand_kmeans2 = rand_score(y_true2, kmeans_labels2)\n",
    "rand_agglo2 = rand_score(y_true2, agglo_labels2)\n",
    "\n",
    "sil_kmeans2 = silhouette_score(X2, kmeans_labels2)\n",
    "sil_agglo2 = silhouette_score(X2, agglo_labels2)\n",
    "\n",
    "print(\"K-Means      -> Rand Index: {:.4f}, Silhouette Score: {:.4f}\".format(rand_kmeans2, sil_kmeans2))\n",
    "print(\"Hierarchical -> Rand Index: {:.4f}, Silhouette Score: {:.4f}\".format(rand_agglo2, sil_agglo2))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72ba36ad",
   "metadata": {},
   "source": [
    "### Dataset 3 (ClusteringData3.csv)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b25a1382",
   "metadata": {},
   "source": [
    "**a.** Load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16dce425",
   "metadata": {},
   "outputs": [],
   "source": [
    "df3 = pd.read_csv(\"ClusteringData3.csv\")\n",
    "X3 = df3[['x', 'y']].values\n",
    "y_true3 = df3['target'].values\n",
    "\n",
    "df3.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1981000",
   "metadata": {},
   "source": [
    "**b.** Plot the original data points using a scatter plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7caf94f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(6, 5))\n",
    "plt.scatter(X3[:, 0], X3[:, 1], c=y_true3, cmap='viridis', s=20)\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Dataset 3 — Original Data\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b1d78b0",
   "metadata": {},
   "source": [
    "**c.** Apply K-Means and Hierarchical Agglomerative Clustering with the given k value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8f00edb",
   "metadata": {},
   "outputs": [],
   "source": [
    "k3 = len(np.unique(y_true3))\n",
    "print(\"k =\", k3)\n",
    "\n",
    "kmeans3 = KMeans(n_clusters=k3, random_state=0, n_init=10)\n",
    "kmeans_labels3 = kmeans3.fit_predict(X3)\n",
    "\n",
    "agglo3 = AgglomerativeClustering(n_clusters=k3)\n",
    "agglo_labels3 = agglo3.fit_predict(X3)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d2c4b5e",
   "metadata": {},
   "source": [
    "**d.** Plot the clusters obtained from each algorithm (separate plot for each method)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d23b6f4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(12, 5))\n",
    "\n",
    "axes[0].scatter(X3[:, 0], X3[:, 1], c=kmeans_labels3, cmap='viridis', s=20)\n",
    "axes[0].set_title(\"Dataset 3 — K-Means Clusters\")\n",
    "axes[0].set_xlabel(\"x\")\n",
    "axes[0].set_ylabel(\"y\")\n",
    "\n",
    "axes[1].scatter(X3[:, 0], X3[:, 1], c=agglo_labels3, cmap='viridis', s=20)\n",
    "axes[1].set_title(\"Dataset 3 — Hierarchical Agglomerative Clusters\")\n",
    "axes[1].set_xlabel(\"x\")\n",
    "axes[1].set_ylabel(\"y\")\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9eec757",
   "metadata": {},
   "source": [
    "**e.** For the K-Means output, display the cluster centers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db960c92",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"K-Means cluster centers:\\n\", kmeans3.cluster_centers_)\n",
    "\n",
    "plt.figure(figsize=(6, 5))\n",
    "plt.scatter(X3[:, 0], X3[:, 1], c=kmeans_labels3, cmap='viridis', s=20)\n",
    "plt.scatter(kmeans3.cluster_centers_[:, 0], kmeans3.cluster_centers_[:, 1],\n",
    "            c='red', marker='X', s=200, label='Cluster Centers')\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.title(\"Dataset 3 — K-Means Cluster Centers\")\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf7e0ba7",
   "metadata": {},
   "source": [
    "**f.** Observe which algorithm produced more clearly separated clusters visually."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f8dc623",
   "metadata": {},
   "source": [
    "*(Compare the two subplots in part (d) for Dataset 3 — note here which algorithm shows tighter, more clearly separated clusters and which shows more overlap.)*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43f1e59c",
   "metadata": {},
   "source": [
    "**g.** Calculate and display the Rand Index and Silhouette Score for both algorithms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc7ffe5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "rand_kmeans3 = rand_score(y_true3, kmeans_labels3)\n",
    "rand_agglo3 = rand_score(y_true3, agglo_labels3)\n",
    "\n",
    "sil_kmeans3 = silhouette_score(X3, kmeans_labels3)\n",
    "sil_agglo3 = silhouette_score(X3, agglo_labels3)\n",
    "\n",
    "print(\"K-Means      -> Rand Index: {:.4f}, Silhouette Score: {:.4f}\".format(rand_kmeans3, sil_kmeans3))\n",
    "print(\"Hierarchical -> Rand Index: {:.4f}, Silhouette Score: {:.4f}\".format(rand_agglo3, sil_agglo3))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47cc4fb0",
   "metadata": {},
   "source": [
    "## Q2. Customer Segmentation on the Mall Customers Dataset\n",
    "A shopping mall wants to group customers based on their Annual Income and Spending Score. Apply K-Means and Hierarchical Agglomerative Clustering to identify customer segments and compare the two methods."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a0a94f0",
   "metadata": {},
   "source": [
    "**a.** Load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26b3529f",
   "metadata": {},
   "outputs": [],
   "source": [
    "mall_df = pd.read_csv(\"Mall_Customers.csv\")\n",
    "X_mall = mall_df[['Annual Income (k$)', 'Spending Score (1-100)']].values\n",
    "\n",
    "mall_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee562316",
   "metadata": {},
   "source": [
    "**b.** Apply K-Means and Hierarchical Agglomerative Clustering with the given k value from the dataset. The best k is selected using the Silhouette Score across a range of candidate values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34cbe14b",
   "metadata": {},
   "outputs": [],
   "source": [
    "silhouette_scores = {}\n",
    "\n",
    "for k in range(2, 11):\n",
    "    kmeans_trial = KMeans(n_clusters=k, random_state=0, n_init=10)\n",
    "    trial_labels = kmeans_trial.fit_predict(X_mall)\n",
    "    silhouette_scores[k] = silhouette_score(X_mall, trial_labels)\n",
    "\n",
    "for k, score in silhouette_scores.items():\n",
    "    print(f\"k = {k} -> Silhouette Score = {score:.4f}\")\n",
    "\n",
    "best_k = max(silhouette_scores, key=silhouette_scores.get)\n",
    "print(\"\\nSelected k =\", best_k)\n",
    "\n",
    "kmeans_mall = KMeans(n_clusters=best_k, random_state=0, n_init=10)\n",
    "kmeans_mall_labels = kmeans_mall.fit_predict(X_mall)\n",
    "\n",
    "agglo_mall = AgglomerativeClustering(n_clusters=best_k)\n",
    "agglo_mall_labels = agglo_mall.fit_predict(X_mall)\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(12, 5))\n",
    "\n",
    "axes[0].scatter(X_mall[:, 0], X_mall[:, 1], c=kmeans_mall_labels, cmap='viridis', s=30)\n",
    "axes[0].scatter(kmeans_mall.cluster_centers_[:, 0], kmeans_mall.cluster_centers_[:, 1],\n",
    "                c='red', marker='X', s=200, label='Cluster Centers')\n",
    "axes[0].set_title(\"Mall Customers — K-Means Clusters\")\n",
    "axes[0].set_xlabel(\"Annual Income (k$)\")\n",
    "axes[0].set_ylabel(\"Spending Score (1-100)\")\n",
    "axes[0].legend()\n",
    "\n",
    "axes[1].scatter(X_mall[:, 0], X_mall[:, 1], c=agglo_mall_labels, cmap='viridis', s=30)\n",
    "axes[1].set_title(\"Mall Customers — Hierarchical Agglomerative Clusters\")\n",
    "axes[1].set_xlabel(\"Annual Income (k$)\")\n",
    "axes[1].set_ylabel(\"Spending Score (1-100)\")\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c30ff6cd",
   "metadata": {},
   "source": [
    "**c.** Calculate and display the Rand Index and Silhouette Score for both algorithms. Since this dataset has no ground-truth cluster labels, the Rand Index here measures the agreement between the two clustering methods themselves rather than against a true label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a0228c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "rand_between_methods = rand_score(kmeans_mall_labels, agglo_mall_labels)\n",
    "\n",
    "sil_kmeans_mall = silhouette_score(X_mall, kmeans_mall_labels)\n",
    "sil_agglo_mall = silhouette_score(X_mall, agglo_mall_labels)\n",
    "\n",
    "print(\"Rand Index (K-Means vs Hierarchical agreement) =\", rand_between_methods)\n",
    "print(\"Silhouette Score (K-Means)      =\", sil_kmeans_mall)\n",
    "print(\"Silhouette Score (Hierarchical) =\", sil_agglo_mall)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39218d23",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import io\n",
    "from sklearn.datasets import make_blobs, make_circles\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.metrics import (accuracy_score, precision_score, recall_score,\n",
    "                              f1_score, confusion_matrix, ConfusionMatrixDisplay)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe3c8b52",
   "metadata": {},
   "source": [
    "## Q1. SVM on Synthetic Datasets\n",
    "Generate a linearly separable dataset (`make_blobs`) and a non-linearly separable concentric-circle dataset (`make_circles`). Apply Linear SVM and RBF-kernel SVM on each, and visualize and compare their decision boundaries and support vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06cf0afe",
   "metadata": {},
   "source": [
    "**a.** Generate the two synthetic 2-D datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f1309d63",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_blobs, y_blobs = make_blobs(\n",
    "    n_samples=200,\n",
    "    centers=2,\n",
    "    n_features=2,\n",
    "    cluster_std=1.0,\n",
    "    random_state=42\n",
    ")\n",
    "\n",
    "plt.figure(figsize=(7, 5))\n",
    "plt.scatter(X_blobs[:, 0], X_blobs[:, 1], c=y_blobs, edgecolor='k')\n",
    "plt.xlabel(\"Feature 1\")\n",
    "plt.ylabel(\"Feature 2\")\n",
    "plt.title(\"Linearly Separable 2-D Dataset\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e923a82e",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_circles, y_circles = make_circles(\n",
    "    n_samples=200,\n",
    "    factor=0.4,\n",
    "    noise=0.08,\n",
    "    random_state=42\n",
    ")\n",
    "\n",
    "plt.figure(figsize=(7, 5))\n",
    "plt.scatter(X_circles[:, 0], X_circles[:, 1], c=y_circles, edgecolor='k')\n",
    "plt.xlabel(\"Feature 1\")\n",
    "plt.ylabel(\"Feature 2\")\n",
    "plt.title(\"Non-Linearly Separable 2-D Dataset\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25c58768",
   "metadata": {},
   "source": [
    "**b.** Apply Linear SVM and RBF-kernel SVM on both datasets. Visualize and compare their decision boundaries and support vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "170fe298",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_decision_boundary(model, X, y, title, ax):\n",
    "    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n",
    "    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n",
    "    xx, yy = np.meshgrid(np.linspace(x_min, x_max, 300), np.linspace(y_min, y_max, 300))\n",
    "\n",
    "    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "    Z = Z.reshape(xx.shape)\n",
    "\n",
    "    ax.contourf(xx, yy, Z, alpha=0.2, cmap='coolwarm')\n",
    "    ax.scatter(X[:, 0], X[:, 1], c=y, cmap='coolwarm', edgecolor='k', s=25)\n",
    "    ax.scatter(model.support_vectors_[:, 0], model.support_vectors_[:, 1],\n",
    "               facecolors='none', edgecolors='black', s=100, linewidths=1.5,\n",
    "               label='Support Vectors')\n",
    "    ax.set_title(title)\n",
    "    ax.set_xlabel(\"Feature 1\")\n",
    "    ax.set_ylabel(\"Feature 2\")\n",
    "    ax.legend()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08645d27",
   "metadata": {},
   "outputs": [],
   "source": [
    "linear_svm_blobs = SVC(kernel='linear', C=1.0)\n",
    "linear_svm_blobs.fit(X_blobs, y_blobs)\n",
    "\n",
    "rbf_svm_blobs = SVC(kernel='rbf', C=1.0)\n",
    "rbf_svm_blobs.fit(X_blobs, y_blobs)\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(13, 5))\n",
    "plot_decision_boundary(linear_svm_blobs, X_blobs, y_blobs, \"Blobs  Linear SVM\", axes[0])\n",
    "plot_decision_boundary(rbf_svm_blobs, X_blobs, y_blobs, \"Blobs  RBF SVM\", axes[1])\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e02f5c95",
   "metadata": {},
   "outputs": [],
   "source": [
    "linear_svm_circles = SVC(kernel='linear', C=1.0)\n",
    "linear_svm_circles.fit(X_circles, y_circles)\n",
    "\n",
    "rbf_svm_circles = SVC(kernel='rbf', C=1.0)\n",
    "rbf_svm_circles.fit(X_circles, y_circles)\n",
    "\n",
    "fig, axes = plt.subplots(1, 2, figsize=(13, 5))\n",
    "plot_decision_boundary(linear_svm_circles, X_circles, y_circles, \"Circles  Linear SVM\", axes[0])\n",
    "plot_decision_boundary(rbf_svm_circles, X_circles, y_circles, \"Circles  RBF SVM\", axes[1])\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95379213",
   "metadata": {},
   "source": [
    "On the linearly separable **blobs** dataset, both the Linear and RBF kernels draw a clean, almost identical boundary with very few support vectors. On the **circles** dataset, the Linear SVM cannot separate the classes (a straight line cannot isolate a ring from its center), while the RBF kernel wraps a circular boundary around the inner class and separates the two classes correctly  this is the classic case for using a non-linear kernel."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "128d3365",
   "metadata": {},
   "source": [
    "## Q2. SVM on the Heart Disease Dataset\n",
    "Use a publicly available Heart Disease dataset containing patient attributes and a binary target indicating the presence/absence of heart disease ([UCI Heart Disease dataset](https://raw.githubusercontent.com/mrdbourke/zero-to-mastery-ml/master/data/heart-disease.csv), 303 patients, 13 attributes)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f59bd42",
   "metadata": {},
   "source": [
    "**a.** Perform basic preprocessing: handle missing values, separate features and target, encode categorical variables where required, and standardize the numerical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15d77708",
   "metadata": {},
   "outputs": [],
   "source": [
    "heart_csv = \"\"\"age,sex,cp,trestbps,chol,fbs,restecg,thalach,exang,oldpeak,slope,ca,thal,target\n",
    "63,1,3,145,233,1,0,150,0,2.3,0,0,1,1\n",
    "57,0,1,130,236,0,0,174,0,0.0,1,1,2,0\n",
    "\"\"\"\n",
    "df = pd.read_csv(io.StringIO(heart_csv))\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66e020b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Missing values per column:\\n\", heart_df.isnull().sum())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39e27d2b",
   "metadata": {},
   "outputs": [],
   "source": [
    "categorical_cols = ['cp', 'restecg', 'slope', 'thal']\n",
    "heart_encoded = pd.get_dummies(heart_df, columns=categorical_cols, drop_first=True)\n",
    "\n",
    "X_heart = heart_encoded.drop('target', axis=1)\n",
    "y_heart = heart_encoded['target']\n",
    "\n",
    "X_heart.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f0ac16aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "X_heart_scaled = scaler.fit_transform(X_heart)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e489b28f",
   "metadata": {},
   "source": [
    "**b.** Divide the dataset into training and testing sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1bd69cbf",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X_heart_scaled, y_heart, test_size=0.2, random_state=0\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f274011d",
   "metadata": {},
   "source": [
    "**c.** Train Linear SVM, Polynomial-kernel SVM, and RBF-kernel SVM models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "348925b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "svm_models = {\n",
    "    \"Linear SVM\": SVC(kernel='linear', C=1.0),\n",
    "    \"Polynomial SVM\": SVC(kernel='poly', degree=3, C=1.0),\n",
    "    \"RBF SVM\": SVC(kernel='rbf', C=1.0)\n",
    "}\n",
    "\n",
    "predictions = {}\n",
    "\n",
    "for name, model in svm_models.items():\n",
    "    model.fit(X_train, y_train)\n",
    "    predictions[name] = model.predict(X_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22770ca2",
   "metadata": {},
   "source": [
    "**d.** Calculate Accuracy, Precision, Recall, and F1-score for each model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36d92350",
   "metadata": {},
   "outputs": [],
   "source": [
    "results = []\n",
    "\n",
    "for name, y_pred in predictions.items():\n",
    "    results.append({\n",
    "        \"Model\": name,\n",
    "        \"Accuracy\": accuracy_score(y_test, y_pred),\n",
    "        \"Precision\": precision_score(y_test, y_pred),\n",
    "        \"Recall\": recall_score(y_test, y_pred),\n",
    "        \"F1-score\": f1_score(y_test, y_pred)\n",
    "    })\n",
    "\n",
    "results_df = pd.DataFrame(results)\n",
    "results_df\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa1a599b",
   "metadata": {},
   "source": [
    "**e.** Display the confusion matrix for each model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bed4374d",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 3, figsize=(16, 5))\n",
    "\n",
    "for ax, (name, y_pred) in zip(axes, predictions.items()):\n",
    "    cm = confusion_matrix(y_test, y_pred)\n",
    "    disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=['No Disease', 'Disease'])\n",
    "    disp.plot(ax=ax, colorbar=False)\n",
    "    ax.set_title(name)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c8a63a2",
   "metadata": {},
   "source": [
    "**f.** Observe the effect of the kernel function and the C parameter on SVM classification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ec33f3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "C_values = [0.01, 0.1, 1, 10, 100]\n",
    "kernel_C_results = []\n",
    "\n",
    "for kernel in ['linear', 'poly', 'rbf']:\n",
    "    for C in C_values:\n",
    "        model = SVC(kernel=kernel, C=C, degree=3)\n",
    "        model.fit(X_train, y_train)\n",
    "        y_pred = model.predict(X_test)\n",
    "        kernel_C_results.append({\n",
    "            \"Kernel\": kernel,\n",
    "            \"C\": C,\n",
    "            \"Accuracy\": accuracy_score(y_test, y_pred)\n",
    "        })\n",
    "\n",
    "kernel_C_df = pd.DataFrame(kernel_C_results)\n",
    "pivot_table = kernel_C_df.pivot(index='C', columns='Kernel', values='Accuracy')\n",
    "pivot_table\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6458a2b9",
   "metadata": {},
   "source": [
    "A small **C** allows more misclassified training points (a wider, softer margin), which tends to under-fit; a large **C** forces the model to classify every training point correctly (a narrow margin), which can over-fit and reduce test accuracy once C grows too large. The kernel choice matters as much as C: the linear kernel does well when the classes are close to linearly separable, while the RBF and polynomial kernels can model more complex, curved boundaries but are more sensitive to the C value and can overfit at high C on a dataset this size."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07efeac4",
   "metadata": {},
   "source": [
    "## SVM with and without PCA on the Breast Cancer Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8448a2cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.metrics import accuracy_score, precision_score\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43fb8d7c",
   "metadata": {},
   "source": [
    "# Q1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b9b8048",
   "metadata": {},
   "source": [
    "**a.** Load the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90fb5ca9",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = load_breast_cancer()\n",
    "X = data.data\n",
    "y = data.target\n",
    "\n",
    "df = pd.DataFrame(X, columns=data.feature_names)\n",
    "df['target'] = y\n",
    "\n",
    "print(\"Number of samples =\", X.shape[0])\n",
    "print(\"Number of features =\", X.shape[1])\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e41fb86c",
   "metadata": {},
   "source": [
    "**b.** Split the dataset into training and testing sets, then standardize the features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3942f20c",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, test_size=0.2, random_state=0, stratify=y\n",
    ")\n",
    "\n",
    "scaler = StandardScaler()\n",
    "X_train_scaled = scaler.fit_transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03477458",
   "metadata": {},
   "source": [
    "**c.** Train an SVM on the original (full) feature set as a baseline, and evaluate it using accuracy and precision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d6fb6c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "baseline_svm = SVC(kernel='rbf', C=1.0, random_state=0)\n",
    "baseline_svm.fit(X_train_scaled, y_train)\n",
    "\n",
    "y_pred_baseline = baseline_svm.predict(X_test_scaled)\n",
    "\n",
    "baseline_accuracy = accuracy_score(y_test, y_pred_baseline)\n",
    "baseline_precision = precision_score(y_test, y_pred_baseline)\n",
    "\n",
    "print(\"Baseline SVM (all\", X.shape[1], \"features)\")\n",
    "print(\"Accuracy  =\", baseline_accuracy)\n",
    "print(\"Precision =\", baseline_precision)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2c789e6",
   "metadata": {},
   "source": [
    "**d.** Apply PCA on the dataset. Take the first k principal components, check for k = 2, 3, 4, ..., and choose the best k based on accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1faaf26d",
   "metadata": {},
   "outputs": [],
   "source": [
    "k_values = list(range(2, 16))\n",
    "pca_results = []\n",
    "\n",
    "for k in k_values:\n",
    "    pca = PCA(n_components=k, random_state=0)\n",
    "    X_train_pca = pca.fit_transform(X_train_scaled)\n",
    "    X_test_pca = pca.transform(X_test_scaled)\n",
    "\n",
    "    svm_pca = SVC(kernel='rbf', C=1.0, random_state=0)\n",
    "    svm_pca.fit(X_train_pca, y_train)\n",
    "    y_pred_pca = svm_pca.predict(X_test_pca)\n",
    "\n",
    "    pca_results.append({\n",
    "        \"k\": k,\n",
    "        \"Accuracy\": accuracy_score(y_test, y_pred_pca),\n",
    "        \"Precision\": precision_score(y_test, y_pred_pca),\n",
    "        \"Explained Variance\": pca.explained_variance_ratio_.sum()\n",
    "    })\n",
    "\n",
    "pca_results_df = pd.DataFrame(pca_results)\n",
    "pca_results_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "536af954",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(8, 5))\n",
    "plt.plot(pca_results_df['k'], pca_results_df['Accuracy'], marker='o')\n",
    "plt.xlabel(\"Number of Principal Components (k)\")\n",
    "plt.ylabel(\"Test Accuracy\")\n",
    "plt.title(\"SVM Accuracy vs Number of Principal Components\")\n",
    "plt.grid(True)\n",
    "plt.show()\n",
    "\n",
    "best_row = pca_results_df.loc[pca_results_df['Accuracy'].idxmax()]\n",
    "best_k = int(best_row['k'])\n",
    "\n",
    "print(\"Best k =\", best_k)\n",
    "print(\"Best accuracy at this k =\", best_row['Accuracy'])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3f30489",
   "metadata": {},
   "source": [
    "**e.** Train the final SVM with the best k, and evaluate the model performance using accuracy and precision."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "151b6359",
   "metadata": {},
   "outputs": [],
   "source": [
    "pca_final = PCA(n_components=best_k, random_state=0)\n",
    "X_train_pca_final = pca_final.fit_transform(X_train_scaled)\n",
    "X_test_pca_final = pca_final.transform(X_test_scaled)\n",
    "\n",
    "svm_pca_final = SVC(kernel='rbf', C=1.0, random_state=0)\n",
    "svm_pca_final.fit(X_train_pca_final, y_train)\n",
    "\n",
    "y_pred_pca_final = svm_pca_final.predict(X_test_pca_final)\n",
    "\n",
    "pca_accuracy = accuracy_score(y_test, y_pred_pca_final)\n",
    "pca_precision = precision_score(y_test, y_pred_pca_final)\n",
    "\n",
    "print(\"SVM + PCA (k =\", best_k, \")\")\n",
    "print(\"Accuracy  =\", pca_accuracy)\n",
    "print(\"Precision =\", pca_precision)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56ebb588",
   "metadata": {},
   "source": [
    "**f.** From the same dataset, apply feature selection using correlation scores (heat map)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e308c402",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(14, 12))\n",
    "corr_matrix = df.corr()\n",
    "sns.heatmap(corr_matrix, cmap='coolwarm', annot=False)\n",
    "plt.title(\"Correlation Heatmap of Breast Cancer Features\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a2bab47",
   "metadata": {},
   "outputs": [],
   "source": [
    "target_corr = corr_matrix['target'].drop('target').sort_values(key=abs, ascending=False)\n",
    "print(\"Features ranked by absolute correlation with target:\\n\")\n",
    "print(target_corr)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1c1cdb1",
   "metadata": {},
   "source": [
    "# Q2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed16cf22",
   "metadata": {},
   "source": [
    "**g.** Choose the top 4 to 6 features and apply SVM on them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "082fa6bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "top_n = 6\n",
    "selected_features = target_corr.head(top_n).index.tolist()\n",
    "print(\"Selected features:\", selected_features)\n",
    "\n",
    "feature_indices = [list(data.feature_names).index(f) for f in selected_features]\n",
    "\n",
    "X_train_selected = X_train_scaled[:, feature_indices]\n",
    "X_test_selected = X_test_scaled[:, feature_indices]\n",
    "\n",
    "svm_selected = SVC(kernel='rbf', C=1.0, random_state=0)\n",
    "svm_selected.fit(X_train_selected, y_train)\n",
    "\n",
    "y_pred_selected = svm_selected.predict(X_test_selected)\n",
    "\n",
    "selected_accuracy = accuracy_score(y_test, y_pred_selected)\n",
    "selected_precision = precision_score(y_test, y_pred_selected)\n",
    "\n",
    "print(\"SVM + Feature Selection (top\", top_n, \"features)\")\n",
    "print(\"Accuracy  =\", selected_accuracy)\n",
    "print(\"Precision =\", selected_precision)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37536549",
   "metadata": {},
   "source": [
    "**h.** Compare the results of SVM + Feature Selection with SVM + PCA (and the full feature baseline)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7933b620",
   "metadata": {},
   "outputs": [],
   "source": [
    "comparison_df = pd.DataFrame([\n",
    "    {\"Model\": \"SVM (all \" + str(X.shape[1]) + \" features)\",\n",
    "     \"Accuracy\": baseline_accuracy, \"Precision\": baseline_precision},\n",
    "    {\"Model\": \"SVM + PCA (k = \" + str(best_k) + \")\",\n",
    "     \"Accuracy\": pca_accuracy, \"Precision\": pca_precision},\n",
    "    {\"Model\": \"SVM + Feature Selection (top \" + str(top_n) + \")\",\n",
    "     \"Accuracy\": selected_accuracy, \"Precision\": selected_precision}\n",
    "])\n",
    "\n",
    "comparison_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "665bf621",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(8, 5))\n",
    "x_pos = np.arange(len(comparison_df))\n",
    "width = 0.35\n",
    "\n",
    "plt.bar(x_pos - width / 2, comparison_df['Accuracy'], width, label='Accuracy')\n",
    "plt.bar(x_pos + width / 2, comparison_df['Precision'], width, label='Precision')\n",
    "\n",
    "plt.xticks(x_pos, comparison_df['Model'], rotation=15)\n",
    "plt.ylabel(\"Score\")\n",
    "plt.title(\"Comparison of SVM Variants\")\n",
    "plt.legend()\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "044c6473",
   "metadata": {},
   "source": [
    "PCA compresses all the original features into a small set of uncorrelated components that capture most of the variance, so it can match the baseline accuracy using far fewer dimensions, but the components themselves are combinations of features and are not directly interpretable. Correlation based feature selection keeps the original, interpretable features and can perform close to the baseline when the top features are strongly related to the target, but it ignores interactions between features that are individually weak yet jointly informative, which PCA can capture through its linear combinations."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2edfb762",
   "metadata": {},
   "source": [
    "## Customer Segmentation using DBSCAN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fa9243c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.cluster import DBSCAN\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0781be53",
   "metadata": {},
   "source": [
    "**a.** Generate the synthetic dataset containing at least 50 customer records with two numerical attributes, `Annual_Spending` (in thousands) and `Purchase_Frequency`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12b05dc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(42)\n",
    "\n",
    "cluster1 = np.random.normal(\n",
    "    loc=[20, 10], scale=[3, 2], size=(20, 2)\n",
    ")\n",
    "\n",
    "cluster2 = np.random.normal(\n",
    "    loc=[80, 40], scale=[5, 3], size=(20, 2)\n",
    ")\n",
    "\n",
    "cluster3 = np.random.normal(\n",
    "    loc=[50, 25], scale=[4, 2], size=(15, 2)\n",
    ")\n",
    "\n",
    "noise = np.array([\n",
    "    [120, 70],\n",
    "    [5, 60],\n",
    "    [100, 5]\n",
    "])\n",
    "\n",
    "data = np.vstack([cluster1, cluster2, cluster3, noise])\n",
    "\n",
    "df = pd.DataFrame(\n",
    "    data,\n",
    "    columns=['Annual_Spending', 'Purchase_Frequency']\n",
    ")\n",
    "\n",
    "print(df)\n",
    "print(\"Dataset shape:\", df.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbcf10fe",
   "metadata": {},
   "source": [
    "**b.** Apply feature scaling using `StandardScaler` and explain why scaling is necessary for DBSCAN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdaaa209",
   "metadata": {},
   "outputs": [],
   "source": [
    "scaler = StandardScaler()\n",
    "X_scaled = scaler.fit_transform(df)\n",
    "\n",
    "X_scaled[:5]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c632370",
   "metadata": {},
   "source": [
    "DBSCAN relies on a single distance threshold, `eps`, to decide whether two points are neighbors. If the features are on very different scales, as they are here since `Annual_Spending` ranges across tens of thousands while `Purchase_Frequency` ranges across tens, the feature with the larger numeric range dominates the Euclidean distance calculation and the smaller feature is effectively ignored. Standardizing both features to zero mean and unit variance puts them on a comparable scale, so a single `eps` value treats distance along both dimensions fairly."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1be4840",
   "metadata": {},
   "source": [
    "**c.** Apply the DBSCAN algorithm with suitable values for `eps` and `min_samples`. Display the cluster labels assigned to each customer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc29251f",
   "metadata": {},
   "outputs": [],
   "source": [
    "dbscan = DBSCAN(eps=0.5, min_samples=4)\n",
    "labels = dbscan.fit_predict(X_scaled)\n",
    "\n",
    "df['Cluster'] = labels\n",
    "df\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13f3e96a",
   "metadata": {},
   "source": [
    "**d.** Visualize the clusters using a scatter plot. Represent noise points (outliers) using a different color or marker."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "742ff946",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.figure(figsize=(8, 6))\n",
    "\n",
    "unique_labels = sorted(set(labels))\n",
    "\n",
    "for label in unique_labels:\n",
    "    mask = labels == label\n",
    "    if label == -1:\n",
    "        plt.scatter(df.loc[mask, 'Annual_Spending'], df.loc[mask, 'Purchase_Frequency'],\n",
    "                    c='red', marker='x', s=100, label='Noise')\n",
    "    else:\n",
    "        plt.scatter(df.loc[mask, 'Annual_Spending'], df.loc[mask, 'Purchase_Frequency'],\n",
    "                    label=f'Cluster {label}', s=60, edgecolor='k')\n",
    "\n",
    "plt.xlabel(\"Annual Spending (thousands)\")\n",
    "plt.ylabel(\"Purchase Frequency\")\n",
    "plt.title(\"DBSCAN Clustering (eps=0.5, min_samples=4)\")\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f663302d",
   "metadata": {},
   "source": [
    "**e.** Identify the number of clusters and noise points. Explain the roles of `eps`, `min_samples`, and the label -1 in DBSCAN."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13034434",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_clusters = len(set(labels)) - (1 if -1 in labels else 0)\n",
    "n_noise = list(labels).count(-1)\n",
    "\n",
    "print(\"Number of clusters =\", n_clusters)\n",
    "print(\"Number of noise points =\", n_noise)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a9ea323",
   "metadata": {},
   "source": [
    "`eps` sets the radius within which DBSCAN looks for neighboring points. `min_samples` sets the minimum number of points, including the point itself, that must lie within that radius for a point to be treated as a dense core point. A point that has enough neighbors within `eps` becomes a core point and starts or extends a cluster, a point that lies within `eps` of a core point but does not itself have enough neighbors becomes a border point of that cluster, and a point that is neither a core point nor close enough to one gets the label -1, which marks it as noise, meaning DBSCAN considers it an outlier that does not belong to any cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc1770f1",
   "metadata": {},
   "source": [
    "**f.** Experiment with at least two different combinations of `eps` and `min_samples`, and observe how the clustering results change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c72f748",
   "metadata": {},
   "outputs": [],
   "source": [
    "parameter_combinations = [\n",
    "    (0.3, 3),\n",
    "    (0.5, 4),\n",
    "    (0.8, 5)\n",
    "]\n",
    "\n",
    "fig, axes = plt.subplots(1, len(parameter_combinations), figsize=(18, 5))\n",
    "\n",
    "experiment_summary = []\n",
    "\n",
    "for ax, (eps, min_samples) in zip(axes, parameter_combinations):\n",
    "    model = DBSCAN(eps=eps, min_samples=min_samples)\n",
    "    trial_labels = model.fit_predict(X_scaled)\n",
    "\n",
    "    trial_n_clusters = len(set(trial_labels)) - (1 if -1 in trial_labels else 0)\n",
    "    trial_n_noise = list(trial_labels).count(-1)\n",
    "\n",
    "    experiment_summary.append({\n",
    "        \"eps\": eps,\n",
    "        \"min_samples\": min_samples,\n",
    "        \"Clusters\": trial_n_clusters,\n",
    "        \"Noise Points\": trial_n_noise\n",
    "    })\n",
    "\n",
    "    for label in sorted(set(trial_labels)):\n",
    "        mask = trial_labels == label\n",
    "        if label == -1:\n",
    "            ax.scatter(df['Annual_Spending'][mask], df['Purchase_Frequency'][mask],\n",
    "                       c='red', marker='x', s=80, label='Noise')\n",
    "        else:\n",
    "            ax.scatter(df['Annual_Spending'][mask], df['Purchase_Frequency'][mask],\n",
    "                       s=50, edgecolor='k', label=f'Cluster {label}')\n",
    "\n",
    "    ax.set_title(f\"eps={eps}, min_samples={min_samples}\")\n",
    "    ax.set_xlabel(\"Annual Spending\")\n",
    "    ax.set_ylabel(\"Purchase Frequency\")\n",
    "    ax.legend(fontsize=8)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "pd.DataFrame(experiment_summary)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cae03aee",
   "metadata": {},
   "source": [
    "A smaller `eps` makes the neighborhood radius tighter, so fewer points qualify as neighbors of each other, which tends to break the data into more, smaller clusters and flags more points as noise. A larger `eps` makes it easier for points to be considered neighbors, which tends to merge nearby clusters together and reduces the number of noise points, but if `eps` is too large, distinct groups of customers can be merged into a single cluster. Raising `min_samples` makes it harder for a point to qualify as a dense core point, which also tends to increase the number of points labeled as noise, since sparser regions of the data no longer have enough neighbors to form a cluster."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}