Welcome to the Part 2 of building a data pipeline using the GCP Dataflow service. In this part, we will develop & test our pipeline locally. So, lets jump right in. We will focus on below code for testing and developing the Dataflow pipeline:
Local Testing via Jupyter Notebook: https://github.com/amandeepsaluja/gcp-dataflow-pipeline/blob/master/01-flex-template/local_testing/testing.ipynb
Before we start, make sure you have created a new virtual environment and installed the below required packages as mentioned in requirements.txt