{"id":3510,"library":"hdijupyterutils","title":"HdiJupyterUtils: Utils for Jupyter projects from HDInsight team","description":"HdiJupyterUtils is a Python library providing utilities for Jupyter projects, particularly those from the HDInsight team, often used in conjunction with Sparkmagic for interactive Spark cluster interaction. It is currently at version 0.23.0 and sees active development with regular patch and minor releases as part of the Sparkmagic project.","status":"active","version":"0.23.0","language":"en","source_language":"en","source_url":"https://github.com/jupyter-incubator/sparkmagic","tags":["jupyter","spark","hdinsight","utilities","sparkmagic","livy"],"install":[{"cmd":"pip install hdijupyterutils","lang":"bash","label":"Install latest version"}],"dependencies":[{"reason":"Often used for data manipulation and display; older versions had explicit pinning requirements.","package":"pandas","optional":false},{"reason":"Core for Jupyter interactivity; specific versions required fixes for compatibility.","package":"ipython","optional":false},{"reason":"Core for Jupyter interactivity; specific versions required fixes for compatibility.","package":"ipykernel","optional":false},{"reason":"Common dependency in Jupyter/web contexts, mentioned in dependency updates.","package":"tornado","optional":false},{"reason":"Used for HTTP communication with remote services like Livy in Sparkmagic.","package":"requests","optional":false},{"reason":"Underpins pandas and other numerical operations; noted in installation issues.","package":"numpy","optional":false}],"imports":[{"note":"A primary utility function for displaying HTML content within Jupyter notebooks, which received compatibility fixes for IPython display.","symbol":"display_html","correct":"from hdijupyterutils.ipythondisplay import display_html"}],"quickstart":{"code":"from hdijupyterutils.ipythondisplay import display_html\nimport pandas as pd\n\n# Displaying a simple HTML string\nhtml_message = \"<h2>Hello from HdiJupyterUtils!</h2><p>This utility aids in Jupyter/HDInsight interactions.</p>\"\ndisplay_html(html_message, raw=True)\n\n# Example of displaying a Pandas DataFrame as HTML using the utility\ndata = {\"Name\": [\"Alice\", \"Bob\"], \"Age\": [25, 30]}\ndf = pd.DataFrame(data)\ndisplay_html(df.to_html(), raw=True)\n","lang":"python","description":"This quickstart demonstrates how to use the `display_html` function from `hdijupyterutils.ipythondisplay` to render raw HTML content, including the HTML representation of a Pandas DataFrame, directly in a Jupyter environment. This function is often used internally by Sparkmagic to present results."},"warnings":[{"fix":"Upgrade Python to 3.7 or newer.","message":"Versions prior to 0.20.0 dropped support for Python 3.6. Users on Python 3.6 must upgrade their Python version to use 0.20.0 or newer.","severity":"breaking","affected_versions":"<0.20.0"},{"fix":"Ensure `pandas` version is less than 2.0.0 or upgrade to `hdijupyterutils` 0.23.0+ and verify compatibility.","message":"Version 0.20.5 introduced a pin for `pandas<2.0.0`, indicating potential incompatibilities or breaking changes when using `hdijupyterutils` with Pandas 2.0 or newer.","severity":"breaking","affected_versions":"0.20.5 - 0.22.x"},{"fix":"Review release notes for `ipykernel` and `hdijupyterutils` regarding async execution; report issues to the `sparkmagic` repository.","message":"Version 0.20.0 introduced significant changes for supporting async kernel execution in `ipykernel>=6`. Users upgrading to this version might encounter errors if their setup relies on specific kernel behaviors.","severity":"breaking","affected_versions":"0.20.0+"},{"fix":"Upgrade to `hdijupyterutils` version 0.23.0 or higher for full compatibility with IPython 8 and 9.","message":"Versions between 0.19.2 and 0.23.0 might have experienced compatibility issues with IPython 8 and 9. Version 0.23.0 specifically includes a fix for IPython 8 and 9 support.","severity":"gotcha","affected_versions":"0.19.2 - 0.22.x"},{"fix":"Use `%load_ext autoreload` and `%autoreload 2` at the top of your notebook to automatically reload modules upon changes.","message":"When developing with this or other Python libraries in Jupyter, changes made to imported modules may not be reflected without restarting the kernel or enabling IPython's `autoreload` extension.","severity":"gotcha","affected_versions":"All"}],"env_vars":null,"last_verified":"2026-04-11T00:00:00.000Z","next_check":"2026-07-10T00:00:00.000Z"}