Data Engineering Part 1 – How to become a Big Data Engineer
Hey Readers, I am a Data Science Student and recently I have started learning more about Data Engineering. Data Science […]
hive
Query S3 data via Hive on local box
In the last post we discussed about how to generate synthetic data. Here we will talk about how to query […]
Part 2: How to create EMR cluster with Apache Spark and Apache Zeppelin
This is part-2 of the blog series – How to analyze Kaggle data with Apache Spark and Zeppelin. In the […]
Debugging : Hive Dynamic partition Error : [Fatal Error] total number of created files now is 100028, which exceeds 100000. Killing the job.
[Fatal Error] total number of created files now is 900320, which exceeds 900000. Killing the job. tldr; quick fix – […]
Hive – Selected data import/query – Files and folders (mapred.input.dir.recursive)
Data import in Hive by default expects a directory name in its query specified by LOCATION keyword. By default Hive […]