Description

Magic Number:143689

Files Created after running

Created new Directory csp554

Copied generated files foodratings143894.txt and foodplaces143894.txt into /user/csp554 as foodratings and foodplaces respectively

Creating an RDD named line
CommandUsed: line=sc.textFile(‘/user/csp554/foodratings.txt’)
Line.take(5)

2) Create a new RDD name ex2RDD and splitting it using “,” Command Used: ex2RDD=line.map(lambda line :line.split(“,”))
ex2RDD.take(5)

3) Create a new RDD name ex3RDDV2 and converting third column into int
Command Used: ex3RDDV2=ex2RDD.map(lambda line1:[line1[0], line1[1], int(line1[2]), line1[3], line1[4], line1[5]])
Ex3RDDV2.take(5)

4) Create a new RDD name ex4RDDV2 and filtering third column such that it is less than 25 Command Used: ex4RDDV2=ex3RDDv2.filter(lambda line1 :line1[2]<25) ex4RDDV2.take(5)

5) Create a new RDD name ex5RDD and making the first column the key Command Used: ex5RDD=ex4RDDV2.map(lambda line: [line[0],line]) ex5RDD.take(5)

6) Create a new RDD name ex6RDD where records are ordered by ascending value of the key