Env:
--Spark 3.2
--Synapse-premium
--Intelligent caching enabled.
Intelligent cache enabled
Inital content of CSV file
read successfully
added more content later
the same result was shown even after running df1.take(100) several times instead of displaying the newly records added to it.
The issue was resolved after rerunning the spark.read again
Env:
--Spark 3.2
--Synapse-premium
--Intelligent caching enabled.
Intelligent cache enabled
Inital content of CSV file
read successfully
added more content later
the same result was shown even after running df1.take(100) several times instead of displaying the newly records added to it.
The issue was resolved after rerunning the spark.read again