HdiJupyterUtils: Utils for Jupyter projects from HDInsight team

0.23.0 · active · verified Sat Apr 11

HdiJupyterUtils is a Python library providing utilities for Jupyter projects, particularly those from the HDInsight team, often used in conjunction with Sparkmagic for interactive Spark cluster interaction. It is currently at version 0.23.0 and sees active development with regular patch and minor releases as part of the Sparkmagic project.

Warnings

breaking Versions prior to 0.20.0 dropped support for Python 3.6. Users on Python 3.6 must upgrade their Python version to use 0.20.0 or newer.
Fix: Upgrade Python to 3.7 or newer.
breaking Version 0.20.5 introduced a pin for `pandas<2.0.0`, indicating potential incompatibilities or breaking changes when using `hdijupyterutils` with Pandas 2.0 or newer.
Fix: Ensure `pandas` version is less than 2.0.0 or upgrade to `hdijupyterutils` 0.23.0+ and verify compatibility.
breaking Version 0.20.0 introduced significant changes for supporting async kernel execution in `ipykernel>=6`. Users upgrading to this version might encounter errors if their setup relies on specific kernel behaviors.
Fix: Review release notes for `ipykernel` and `hdijupyterutils` regarding async execution; report issues to the `sparkmagic` repository.
gotcha Versions between 0.19.2 and 0.23.0 might have experienced compatibility issues with IPython 8 and 9. Version 0.23.0 specifically includes a fix for IPython 8 and 9 support.
Fix: Upgrade to `hdijupyterutils` version 0.23.0 or higher for full compatibility with IPython 8 and 9.
gotcha When developing with this or other Python libraries in Jupyter, changes made to imported modules may not be reflected without restarting the kernel or enabling IPython's `autoreload` extension.
Fix: Use `%load_ext autoreload` and `%autoreload 2` at the top of your notebook to automatically reload modules upon changes.

Install

pip install hdijupyterutils Install latest version

Imports

display_html
```
from hdijupyterutils.ipythondisplay import display_html
```
A primary utility function for displaying HTML content within Jupyter notebooks, which received compatibility fixes for IPython display.

Quickstart

This quickstart demonstrates how to use the `display_html` function from `hdijupyterutils.ipythondisplay` to render raw HTML content, including the HTML representation of a Pandas DataFrame, directly in a Jupyter environment. This function is often used internally by Sparkmagic to present results.

from hdijupyterutils.ipythondisplay import display_html
import pandas as pd

# Displaying a simple HTML string
html_message = "<h2>Hello from HdiJupyterUtils!</h2><p>This utility aids in Jupyter/HDInsight interactions.</p>"
display_html(html_message, raw=True)

# Example of displaying a Pandas DataFrame as HTML using the utility
data = {"Name": ["Alice", "Bob"], "Age": [25, 30]}
df = pd.DataFrame(data)
display_html(df.to_html(), raw=True)

view raw JSON →