Created using Colaboratory

Author: Animeshcoder

Python_MySQL_P3.ipynb

@@ -0,0 +1,103 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": [],
+      "authorship_tag": "ABX9TyPi/4gV2N2OEUUI1ClLmpNo",
+      "include_colab_link": true
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "view-in-github",
+        "colab_type": "text"
+      },
+      "source": [
+        "<a href=\"https://colab.research.google.com/github/Animeshcoder/MySQL-Python/blob/main/Python_MySQL_P3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Introduction:**\n",
+        "This project demonstrates how to use Pandas, SQLAlchemy, and PyMySQL to extract data from one MySQL database, manipulate it using Pandas, and save it to another MySQL database. The script includes logic for transforming and manipulating data to match the structure of a destination table, as well as checking if data already exists before saving it. In simple way you can understand that if values of column are stored in one database and it is storing that column data to the different columns of another database row wise."
+      ],
+      "metadata": {
+        "id": "ovTxm7DAhAUG"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "x6MBaV9ug-MI"
+      },
+      "outputs": [],
+      "source": [
+        "import pandas as pd\n",
+        "from sqlalchemy import create_engine\n",
+        "import urllib.parse\n",
+        "\n",
+        "password = \"password@123\"\n",
+        "password = urllib.parse.quote(password)\n",
+        "\n",
+        "engine = create_engine(f\"mysql+pymysql://youruser:{password}@yourhost/olddatabase\")\n",
+        "\n",
+        "# write a SQL query to extract the data you need\n",
+        "query = \"SELECT * FROM olddatabase.oldtable where id > '363000';\"\n",
+        "\n",
+        "# use the read_sql_query function from pandas to execute the query and store the result in a DataFrame\n",
+        "df = pd.read_sql_query(query, engine)\n",
+        "\n",
+        "# select the column you want to save\n",
+        "column_data = df[[\"Value\"]]\n",
+        "\n",
+        "# transform the data into multiple columns\n",
+        "column_data = column_data.T\n",
+        "\n",
+        "# rename the columns to match the column names in the newtable\n",
+        "column_data.columns = [\n",
+        "        'Column1',\n",
+        "        'Column2',\n",
+        "        'Column3'  ,\n",
+        "        'Column4'\n",
+        "        ]\n",
+        "\n",
+        "# create a connection to the new database\n",
+        "new_engine = create_engine(f\"mysql+pymysql://youruser:{password}@yourhost/newdatabase\")\n",
+        "\n",
+        "# check if the data already exists in the new database\n",
+        "existing_data = pd.read_sql_table(\"mytable\", new_engine)\n",
+        "if not existing_data.equals(column_data):\n",
+        "    # save the data to the new database\n",
+        "    column_data.to_sql(\"newtable\", new_engine, index=False, if_exists=\"append\")\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "### **Explanation:**\n",
+        "The first part of the script sets up the dependencies and connection details for the MySQL databases. The pandas module is imported as pd, the create_engine function is imported from the sqlalchemy module, and the urllib.parse module is imported. The password for the MySQL server is defined and URL-encoded using the quote function from the urllib.parse module to ensure that it can be safely included in a connection string. An engine object is then created using the create_engine function from SQLAlchemy, which represents a connection to the old MySQL database.\n",
+        "\n",
+        "Next, an SQL query is defined that selects all rows from the oldtable table in the olddatabase database where the value of the id column is greater than '363000'. This query is used to extract the data that will be saved to the new database. The read_sql_query function from Pandas is used to execute this query on the old MySQL database and store the result in a DataFrame.\n",
+        "\n",
+        "The script then selects a specific column (\"Value\") from the DataFrame and stores it in a new variable (column_data). This column data is then transformed into multiple columns by transposing it using the .T attribute. The columns of this transposed data are then renamed to match the column names in the destination table in the new database.\n",
+        "\n",
+        "A new engine object is created that represents a connection to the new MySQL database. The script then checks if the data already exists in this new database by reading data from the mytable table using the read_sql_table function from Pandas and comparing it to the transformed column data using the .equals method. If the data does not already exist in the new database, it is saved to a new table named newtable using the .to_sql method of the DataFrame object."
+      ],
+      "metadata": {
+        "id": "MhudTCIlk9a9"
+      }
+    }
+  ]
+}