Senior Big Data Engineer @ • Lead developer in data warehousing and analytics managing 3 fully-dimensioned clusters with 60+ nodes and over 1 Petabytes(raw) enterprise data warehouse operations in production processing over 1 billion rows daily traffic in a 24x7 environment
• Key technical designer and developer in various external data product for the company in leading to define technical requirements, estimating level of effort and designing schema, optimization and ETL.
• Lead data warehouse developer for internal reporting dealing with day to day internal reporting requirement, bug fix , ad hoc requests and support analytic reports in SQL/Microstrategy for products, marketing, finance, data science, ad auction analysis.
• Built internal reporting data warehouse from ground up by getting data from Hadoop.
Leading role in various cluster migrations such as migrating 50-node sharded MYSQL datamart to 20-node column based Vertica cluster and porting multi-terabytes of data to multiple different clusters for archiving and hardware swap for performance gain and stability
• Improved stability and performance for dimensional data pull from RabbitMQ with 300% speed up
• Improved operational routines by streamlining ETL process and schedule to decrease down time frequency from every two weeks to no down time maintenance
• On-Call response team for quick response to immediate customer issues with hot-fix patches if needed
• Mentored junior developers in ETL development and bug tracing skills in complex , already-developed data warehouse systems. From 2013 to 2014 (1 year) Senior Data Engineer @ • Key developer and responsible for fully-dimensioned multi-terabyte enterprise data warehouse operations in production
processing hundreds millions of rows per day with over 800k app DAU in a 24x7 environment (Single handed)
• Designed and created data marts for consumers in Products, Marketing, Finance and internal data tools
• Improved stability and performance for queries on a 5-Node Vertica platform with projection, segmentation and partitioning
and optimized all regular aggregation queries from worst case 3 hours to under 5 minutes
• Improved operational routines from scheduled maintenance for every two weeks to no down time maintenance.
• Designed and optimized data pipeline from 10+ internal and external data sources to improve reliability, accountability, and
manageability of ETLs
• Achieved high stability with zero down time for at least 3 months since data pipeline improvement (June 2013) From May 2012 to March 2013 (11 months) Teaching Assistant @ Artificial Intelligence and Computer Programming
• Developed assignment and programming ventures in themes such as decision tree, game playing, and logical reasoning
• Trained AI notions in lectures, lab, and review sessions including machine learning, planning, and decision tree
• Helped students with difficulties in various programming projects, for example, A* Search, recognition and puzzle solving From 2009 to May 2012 (3 years) Research Assistant - COMMEX Project @ Created and implemented Bayesian based learning algorithms as framework for detection of signal jamming in cognitive
Contributed to the production of research reports and publications as well as conducted literature and database searches From April 2011 to April 2012 (1 year 1 month) Software Engineer Intern @ • Produced the prototype of the Ad scoring and selection system and incorporated it to a reflected replication
• Researched Internet traffic history and generated ranking models using Decision Tree and Support Vector Machine From May 2010 to August 2010 (4 months) Research Assistant @ LANDroid Project (Joint with ArgonST)
• Devised a completely autonomous robotic network with iRobot Roomba and Gumstix micro controller
• Created an algorithm named “Tentacle” steering robots to establish connections at suitable positions
• Published “Tentacle” algorithm at IROS 2009 robotics conferences in Oct 2009 with 100% convergence in practical experiment From January 2008 to December 2008 (1 year) Reseach Assistant @ Self-Reconfigurable (SuperBot) Robot Project
• Created and implemented software modules under tight available resources on Atmega 128 controlling infrared-sensors, I2C communication and motor control with AVRx
• Designed a architecture for dynamic software module management to resolve resource contentions in Atmega 128
• Achieved the record of 1 kilo-meter run on a 6-module rolling track configuration with only a single charge
• Invented self-recovering modular rolling track with the ability to resume its shape after falling sideways From 2005 to 2007 (2 years)
Looking for a different
Get an email address for anyone on LinkedIn with the ContactOut Chrome extension