Spark’s AccessControlException: Permission denied

One day you may face this error:

AccessControlException: Permission denied: user=root, access=WRITE, inode="/user"

Full exception text is:

org.apache.spark.SparkContext  - Error initializing SparkContext. Permission denied: user=root, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x
	at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(

Firstly, you need to check that user from which your Spark working on, has home folder under /user like that: /user/John_Doe.

If you already have that folder, try to grant access to /user dir. If you unable to grant access to that directory or it undesirable in your environment, you can set another directories for Spark to work with:

  • Add this to your’s run command: --conf spark.yarn.stagingDir=hdfs://yourcluster/another/hdfs/path
  • After creation of JavaSparkContext, set: javaSparkContext.setCheckpointDir("/tmp") or path like that.
Telegram channel

If you still have any questions, feel free to ask me in the comments under this article or write me at

If I saved your day, you can support me 🤝

Leave a Reply

Your email address will not be published. Required fields are marked *