Have 4 years of experience as Architect, Design lead & SME support in big data technologies Hive, Mapreduce, Spark, Sqoop, Flume, Pig,Python ,Microsoft technologies Azure Data Factory,Analytics,Machine learning,Power BI.Have 8 years of experience in Database technologies Oracle, SQL Server and scripting languages Unix shell script and Pro*c. Have been involved in Architect/Design/analysis/Development/ Testing.
Key Skills Hadoop, Mapreduce, Spark, Azure services, Machine learning, Python, Oracle, SQL Server, Power BI.Horton works Certified Hadoop Developer (HDPCD certification).
• Certified Scrum Master , Certified Oracle SQL and PL/SQL and Specialized on Consultative selling techniques.
Big Data lead & Developer at Cognizant Technology solutions
Jul 2015 - Present
Annual service for British Gas services customers has to happen on a yearly basis depending on their visit dates. Since multiple systems are involved and due to replication issues ASV was not happening as required for customers having data issues. As part of this project Data is extracted and analyzed from different systems in Hadoop and data issues were fixed on source system. Tools used Hadoop – HDFS, Hive, Shell scripting, SAP – CRM & ISU, Qlikview. Role and Responsibilities • Lead a team of Data analysts and coordinating with business for Requirements and solution design. • Implementing the extracts in Hadoop using Hive and shell scripting. • UDF’s were created in java for additional functionality needed. • Coordinate with support team for deployment and support. • Fine-tuned overall system performance for ASV, P0 Extract and resolved performance bottleneck issues effectively. • Accountable for deploying applications into production, from strategic design all the way to development and handling daily operations of Hadoop environments. • Coordinate with SAP team for fixes file requirement and automated fixing.
Big Data Architect & Developer at Cognizant Technology solutions.
Oct 2013 - Present
British Gas customer and utility data is stored in multiple databases and was periodically refreshed into one database namely Microsoft SQL server where reporting and analysis was carried out. Since the deployment of our big data platform (Horton works) all customer and utility data has been moved to HDFS. British Gas took the strategic decision to decommission Microsoft server and build a reporting framework that would identify and monitor incorrect data that would require fixing. Tools used Hadoop – HDFS, Hive, Map reduce, Sqoop, Spark, Shell scripting, Python & R. Role and Responsibilities • As a Technical architect responsible for designing the framework using Big data technologies. • Data model design for the framework using Visio. The snowflake pattern (fact and dimension) was used. • Implementing the data model using hive scripts. All the tables were created in HDFS. • Queries were written in hive to identify incorrect data. • A framework was built to populate the data model in order to monitor the incorrect data in a structured manner so there is a visibility of how many have been fixed since the last run, how many are new instances and how many were not fixed. This was developed using hive. • A scheduler framework was created using python that would read configuration files and output a shell script that would execute the framework. • Visualization (trend graphs and pie charts) were created using R by connecting to hive. • Sqoop was used for transforming data for fixes and exporting to Relational data • Map reduce programming was used to do exception log analysis and for adhoc files received by business. • Enhancement to existing framework is being developed using Spark. • Involved in project plan creation (Governance discussion, Management forums with customer) • Manage and resolve queries, escalations, conflicts and issues across onsite/offshore teams.