Installed Hadoop Yarn 2.4.1 and Spark 2.0.1 and Zeppelin 0.6.1
- Downloaded Apache Mirroring Site for Hadoop 2.4.1
- Downloaded Spark 2.0.1 with pre-built hadoop 2.4.
- Cloned git repository for Zeppelin and run latest maven version, otherwise installation issue happens (mvn clean intall –DskipTests)
Actually, I tried to install zeppelin-bin file downloaded from Apache Mirror Site, but it was tricky to run statement in Zeppelin Dashboard. It was interpreter issue. Zeppelin seemingly did not support other interpreter.
Just for your information once you set a configuration for some framework like Spark in zeppelin-env.sh, you do not need to run Spark Daemon by yourself. Zeppelin will do when running Spark job in the dashboard.
In zeppelin-env.sh
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
export MASTER=spark://jjoon-PC:7077
export SPARK_HOME=/home/jjoon/big_data/spark-2.0.1
export SPARK_SUBMIT_OPTIONS="--driver-memory 1024M --executor-memory 1G"
'Distributed System Information' 카테고리의 다른 글
| Zeppelin + Python Spark SQL + Hive + Hadoop (0) | 2016.10.15 |
|---|---|
| System Monitoring Tool and a Plugin with Flink and BigBench (0) | 2015.12.16 |
| Haloop? Why using combiner? (1) | 2015.07.12 |
| Big Data Visualization & Value Chain Part.1 (0) | 2015.06.07 |
| Parsing Log by Using Regular Expression (0) | 2015.04.29 |