How to install Spark (PySpark) on Windows

Spark supports a number of programming languages including Java, Python, Scala, and R. In this tutorial, we will set up Spark with Python Development Environment by making use of Spark Python API (PySpark) which exposes the Spark programming model to Python. Required Tools and Technologies: Pointers for smooth installation: – As of writing of this […]

A look into ETA Problem using Regression in Python – Machine Learning

ETA Problem using Regression in Python

The term “ETA” usually means “Estimated Time of Arrival” but in the technology realm it refers as “Estimated Completion Time” of a computational process in general. In particular, this problem is specific to estimating completion time of a batch of long scripts running parallel to each other. Problem A number of campaigns run together in […]