One of the challenges I came across recently was to create a connector that would collect specific columns in a csv at AWS and load to a table in Microsoft SQL Server.
To make it easier for someone that might come across the same challenge in the future, I am sharing the code here:
import pyodbc as odbc import pandas as pd df = pd.read_csv(".csv") # Selecting columns to be imported columns = (['col1', 'col2', ...]) df_data = df[columns] records = df_data.values.tolist() # Connection SQL Server DRIVER = '' SERVER_NAME = '' DATABASE_NAME = '' USER = '' PASSWORD = '' server = '' database = '' username = '' password = '' conn = odbc.connect('DRIVER={};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password) cursor = conn.cursor() # Creating Cursor Conectors and inserting data. sql_insert = ''' INSERT INTO dbo.table VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, GETDATE()) ''' try: cursor = conn.cursor() cursor.executemany(sql_insert, records) cursor.commit() except Exception as e: cursor.rollback() finally: print('Task is complete.') cursor.close() conn.close()
I hope this helps!
Thank you for reading