Scrapbook

Warnings

• breaking The `scrapbook` package on PyPI was formerly published under the name `nteract-scrapbook`. With version 0.5.0, the package name changed to `scrapbook`. If you were installing `nteract-scrapbook`, you need to update your dependency to `scrapbook`.
  Fix: Update your `requirements.txt` or `setup.py` to use `scrapbook` instead of `nteract-scrapbook`. For older versions (e.g., 0.2.0), you must explicitly install `nteract-scrapbook==0.2.0`.
• deprecated Scrapbook replaces `papermill`'s direct record functionality. While some backward compatibility exists (e.g., `nb.papermill_dataframe`), it is recommended to transition to Scrapbook's `glue` and `read_notebook` API for recording and retrieving data.
  Fix: Replace `papermill.record()` calls with `scrapbook.glue()`. Utilize `scrapbook.read_notebook()` for accessing recorded data, rather than `papermill`'s older retrieval methods.
• gotcha When using `sb.glue()` to store pandas DataFrames, `scrapbook` leverages `pyarrow` to convert the DataFrame to a base64 encoded Parquet file. This process can fail if the DataFrame contains certain complex nested objects (e.g., columns with dictionaries or sets directly within them), raising an `Arrow` exception.
  Fix: Ensure that pandas DataFrames intended for `sb.glue()` do not contain deeply nested, non-serializable objects like dicts or sets as direct column values. Flatten complex structures or convert them to more basic types if possible before gluing.
• breaking Python 2.7 support was officially dropped after 2020. Versions 0.4.0 and newer are Python 3 (3.5+) only. The documentation states Python 3.6+ is supported.
  Fix: Ensure your project runs on Python 3.6 or newer. Upgrade your Python environment if currently using an older version.
• gotcha Calling `sb.glue()` outside of an active Jupyter/nteract kernel context (e.g., in a plain Python script) may not correctly persist the data or could raise warnings. `scrapbook` relies on the kernel's display machinery to store 'scraps' in the notebook's output.
  Fix: Only use `sb.glue()` within a Jupyter or nteract notebook environment during execution. For external data storage outside a notebook, use standard file I/O or other serialization libraries.

Install

• pip install scrapbook Base Installation
• pip install scrapbook[all] With All Optional I/O Dependencies (e.g., S3, Azure)

Imports

• scrapbook

  import scrapbook as sb

• glue

  sb.glue('my_data', {'key': 'value'})

  Records data into the current notebook's cell output.
• read_notebook

  notebook = sb.read_notebook('path/to/output.ipynb')

  Reads a single notebook file and returns a Notebook object.
• read_notebooks

  scrapbook_collection = sb.read_notebooks('path/to/directory')

  Reads multiple notebooks from a directory and returns a Scrapbook object.
• Scrapbook

  from scrapbook.models import Scrapbook

  Represents a collection of Notebook objects.

Quickstart

This quickstart demonstrates how to 'glue' (record) data into a notebook's output and then 'read' it back. The `sb.glue()` function is used within a notebook cell to store data. Subsequently, `sb.read_notebook()` can be used to load the notebook and access the stored 'scraps' by name. For demonstration purposes outside a live kernel, the example simulates the creation of an output notebook file containing 'scraps'.

import scrapbook as sb
import os

# --- Part 1: Write data to a dummy notebook (simulating execution) ---
# This part would typically run inside a Jupyter/nteract notebook cell.
# For demonstration, we'll create a dummy output file.

# In a real notebook, you'd just call sb.glue directly.
# Here, we simulate it by writing to a temporary file.
notebook_content_template = '''{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/scrapbook+json": {
       "data": {}, 
       "encoder": "json", 
       "name": "my_string", 
       "display": null
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": ["import scrapbook as sb\n", "sb.glue('my_string', 'Hello Scrapbook!')"]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/scrapbook+json": {
       "data": 12345, 
       "encoder": "json", 
       "name": "my_number", 
       "display": null
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": ["sb.glue('my_number', 12345)"]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}'''

# Manually inject the data for the example since we're not running a live kernel
# In a real scenario, these outputs would be generated by `sb.glue` calls
import json
nb_dict = json.loads(notebook_content_template)
# Update the 'my_string' scrap
nb_dict['cells'][0]['outputs'][0]['data']['application/scrapbook+json']['data'] = 'Hello Scrapbook!'
# Update the 'my_number' scrap
nb_dict['cells'][1]['outputs'][0]['data']['application/scrapbook+json']['data'] = 12345

output_notebook_path = 'output_test_notebook.ipynb'
with open(output_notebook_path, 'w') as f:
    json.dump(nb_dict, f, indent=4)

print(f"Created dummy notebook: {output_notebook_path}")

# --- Part 2: Read data from the notebook --- 
# This part can run in a separate script or notebook.

# Read the notebook containing the 'scraps'
nb = sb.read_notebook(output_notebook_path)

# Access a specific scrap by name
my_string_scrap = nb.scraps.my_string
my_number_scrap = nb.scraps.my_number

print(f"\nRetrieved string scrap: {my_string_scrap.data}")
print(f"Retrieved number scrap: {my_number_scrap.data}")

# You can also get all scraps as a dictionary
all_scraps = nb.scraps.to_dict()
print(f"\nAll scraps: {all_scraps}")

# Clean up the dummy file
os.remove(output_notebook_path)
print(f"Cleaned up {output_notebook_path}")