Database Normalization & CSV Import Solution

Project Info -

Developed a comprehensive solution to normalize and structure large datasets using Python and MySQL. The project involved importing CSV data and transforming it into a fully normalized relational database. Key features included: - Data Normalization: Ensured efficient database design by dividing raw data into logical tables while maintaining referential integrity with foreign key relationships. - Chunk-wise Processing: For large datasets, implemented chunk-wise data reading and insertion to optimize performance and memory usage. - Custom Mapping and Renaming: Applied a structured approach to map and rename columns for clarity and usability. - Python Integration: Leveraged Python with Pandas and MySQL connector libraries for seamless data processing and insertion. - Error Handling and Validation: Incorporated robust mechanisms to handle missing or inconsistent data, ensuring data integrity throughout the process. This solution provides a scalable and efficient framework for managing complex data transformations and database structuring.