{"id":2943,"library":"emr-notebooks-magics","title":"EMR Notebooks Magics","description":"EMR Notebooks Magics is a Python library providing Jupyter magics specifically for Amazon EMR Notebooks. These magics enhance the interactive experience by allowing direct interaction with EMR cluster resources, such as mounting S3 workspaces and executing other notebooks. The library is currently at version 0.2.4 and is actively maintained by AWS Labs, with updates tied to EMR Notebooks and EMR Studio feature development.","status":"active","version":"0.2.4","language":"en","source_language":"en","source_url":"https://github.com/awslabs/emr-notebooks-magics","tags":["aws","emr","jupyter","notebooks","magics","data-science"],"install":[{"cmd":"pip install emr-notebooks-magics","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Required for mounting S3 workspaces via %mount_workspace_dir.","package":"s3fs-fuse","optional":true},{"reason":"Alternative FUSE-based filesystem for S3, can be used with %mount_workspace_dir.","package":"goofys","optional":true}],"imports":[{"note":"This library provides IPython magics, not traditional Python modules for direct import. Magics are used directly in Jupyter cells with a '%' prefix after installation and kernel restart.","wrong":"from emr_notebooks_magics import mount_workspace_dir","symbol":"%mount_workspace_dir","correct":"%mount_workspace_dir ."},{"note":"Magics are invoked directly in Jupyter cells; they are not imported as standard Python functions.","wrong":"import emr_notebooks_magics; emr_notebooks_magics.generate_s3_download_url(...)","symbol":"%generate_s3_download_url","correct":"%generate_s3_download_url s3://my_bucket/path/to/object"}],"quickstart":{"code":"# After installing the package and restarting the kernel in an EMR Notebook.\n\n# Mount the entire Workspace directory to the EMR cluster instance\n%mount_workspace_dir .\n\n# Generate a presigned URL for an S3 object\n# Replace with your actual S3 bucket and object key\n%generate_s3_download_url s3://my-example-bucket/path/to/my-file.csv\n\n# Execute another notebook in the background\n# Make sure 'another_notebook.ipynb' exists in your workspace\n%execute_notebook another_notebook.ipynb","lang":"python","description":"This quickstart demonstrates the core functionalities of EMR Notebooks Magics: mounting a workspace directory, generating a presigned S3 download URL, and executing another notebook. These commands are run directly in a Jupyter cell within an EMR Notebook environment."},"warnings":[{"fix":"Restart the Jupyter kernel (`Kernel -> Restart Kernel`) after installation. Do not attempt to install it via EMR bootstrap actions as the notebook environment is installed later.","message":"After installing `emr-notebooks-magics` (e.g., via `pip install`), you must restart the Jupyter kernel before the magics become available. Installing via bootstrap actions is not supported.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be aware of this limitation. If Spark executors need access to data, ensure data is directly accessible via S3 paths or other cluster-wide storage, rather than relying solely on a Python kernel's mounted workspace.","message":"The `%mount_workspace_dir` magic primarily functions with Python 3 kernels. Spark executors do not have access to the mounted directory when used with a Python 3 kernel.","severity":"breaking","affected_versions":"All versions"},{"fix":"Always enable S3 versioning on your S3 Workspace bucket as a precaution if you enable write access. Strongly consider keeping mounts read-only unless write access is absolutely necessary.","message":"Enabling write access with `%mount_workspace_dir` is irreversible and applies changes directly to your S3 Workspace. By default, mounts are read-only.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Ensure that your EMR cluster's EC2 instance profile (e.g., `EMR_EC2_DefaultRole`) has `AmazonElasticMapReduceforEC2Role` managed policy or equivalent S3 read permissions. For `%execute_notebook`, additional permissions might be needed.","message":"Using magics like `%execute_notebook` requires specific IAM permissions for the EMR-EC2 instance role. Lack of proper S3 read access for the EC2 instance profile will lead to failures.","severity":"gotcha","affected_versions":"All versions"},{"fix":"These magics are only compatible with traditional EMR clusters. For EMR Serverless, manage Spark configurations directly via the `SparkContext` object as a workaround for some functionalities.","message":"EMR Serverless and Amazon EMR on EKS clusters do not support EMR Notebooks magics, including those from `emr-notebooks-magics`.","severity":"breaking","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}