Difference Between map and flatMap in Java 8

Definition in Java 8 Stream and Optional

Optional

  • map: If a value is present, apply the provided mapping function to it, and if the result is non-null, return an Optional describing the result. Otherwise return an empty Optional.
  • flatMap: If a value is present, apply the provided Optional-bearing mapping function to it, return that result, otherwise return an empty Optional. This Method is similar to map(Function), but the provided mapper is one whose result is already an Optional, and if invoked, flatMap does not wrap it with an additional Optional.
Read More

SSHFS on Centos

Install on machine which needs mount other folders only.

sudo yum install epel-release
sudo yum install fuse
sudo yum install sshfs
sudo modprobe fuse
Read More

Hive vs Pig

Pig Hive
Procedural Data Flow Language Declarative SQLish Language
For Programming For creating reports
Mainly used by Researchers and Programmers Mainly used by Data Analysts
Operates on the client side of a cluster. Operates on the server side of a cluster.
Does not have a dedicated metadata database. Makes use of exact variation of dedicated SQL DDL language by defining tables beforehand.
Pig is SQL like but varies to a great extent. Directly leverages SQL and is easy to learn for database experts.
Pig supports Avro file format. Hive does not support it.
Read More

Hadoop YARN

What is YARN?

YARN stands for Yet Another Resource Negotiator. It is a generic resource platform for managing resources in a cluster. YARN was introduced with Hadoop 2.0, an open source distributed processing framework from Apache.
YARN

Read More

Apache Spark

What is Spark?

Spark is an open-source cluster computing framework originally developed in the AMPLab at UC Berkeley. It is a fast and general engine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing.

Read More