Spark Streaming Example

Spark Streaming Example : 

1. Start spark-shell 
2. Import required libraries :

• import org.apache.spark.SparkConf
• import org.apache.spark.streaming.{Seconds, StreamingContext}
• import org.apache.spark.storage.StorageLevel
• import StorageLevel._
• import org.apache.spark.streaming.StreamingContext.

     3. Create Steaming Context  and define Port

• val ssc = new StreamingContext(sc, Seconds(2))
• val lines = ssc.socketTextStream("192.168.2.156",8585,MEMORY_ONLY)

    4. Split and count  number of words

• val wordsFlatMap = lines.flatMap(_.split(" "))
• val wordsMap =wordsFlatMap.map(w => (w,1))
• val wordCount = wordsMap.reduceByKey( (a,b) => (a+b))
• wordCount.print

  5. ssc.start