{"id":8069,"library":"datarecorder","title":"DataRecorder","description":"DataRecorder is a Python toolkit designed for efficient and reliable data recording to various file formats. It tackles common issues in data collection like frequent file I/O by caching data and writing in batches, reducing overhead and preventing data loss from unexpected program termination. It supports multithreaded writes and automatically handles file locking. The library provides specialized tools like `Recorder` for sequential data, `Filler` for filling tabular data at specific coordinates, and `ByteRecorder` for binary data. It supports `csv`, `xlsx`, `json`, `txt`, and arbitrary binary file formats. [2]","status":"active","version":"3.6.2","language":"en","source_language":"en","source_url":"https://github.com/g1879/DataRecorder","tags":["data recording","logging","file I/O","csv","xlsx","json","txt","binary","utility"],"install":[{"cmd":"pip install DataRecorder","lang":"bash","label":"Default Install"}],"dependencies":[{"reason":"Required for .xlsx file support.","package":"openpyxl","optional":true},{"reason":"Often used for data manipulation before recording to tabular formats like CSV/XLSX, though not a direct dependency of DataRecorder itself.","package":"pandas","optional":true}],"imports":[{"symbol":"Recorder","correct":"from DataRecorder import Recorder"},{"symbol":"Filler","correct":"from DataRecorder import Filler"},{"symbol":"ByteRecorder","correct":"from DataRecorder import ByteRecorder"}],"quickstart":{"code":"from DataRecorder import Recorder\nimport os\n\n# Example for Recorder\nfile_path = 'my_data.csv'\nr = Recorder(file_path)\n\ndata_row_1 = (1, 2, 3, 4)\ndata_row_2 = (5, 6, 7, 8)\n\nr.add_data(data_row_1) # Record a single row of data\nr.add_data(data_row_2) # Record another single row\nr.add_data('just a string') # Can also record single values\n\n# Simulate collecting more data\nfor i in range(10):\n    r.add_data([f'item_{i}', i * 10, True])\n\nr.record() # Force flush any cached data to file\nr.close() # Close the recorder, ensuring all data is written\n\nprint(f\"Data written to {file_path}\")\n\n# Clean up the created file for re-runnability\nif os.path.exists(file_path):\n    os.remove(file_path)","lang":"python","description":"This quickstart demonstrates how to use the `Recorder` class to append data to a CSV file. It shows adding individual rows and a loop for multiple entries, followed by manually calling `record()` to flush data and `close()` to ensure all buffered data is written."},"warnings":[{"fix":"Update your code to expect and handle dictionary return types for `data` parameters when working with 'db' and 'xlsx' formats. Inspect the structure of the returned dictionary for specific keys/values.","message":"When processing 'db' (database) and 'xlsx' (Excel) formats, the `data` parameter's return value changed to a dictionary format in DataRecorder 3.x. [1]","severity":"breaking","affected_versions":"3.x onwards"},{"fix":"Implement explicit try-except blocks around `record()` calls to catch exceptions and handle unsaved data manually if necessary. The library now prioritizes silent error handling for robustness.","message":"The `record()` method no longer automatically prints data or returns unsaved data upon encountering an exception. This changes the error reporting and recovery mechanism. [1]","severity":"breaking","affected_versions":"3.x onwards"},{"fix":"Ensure `openpyxl` is installed (`pip install openpyxl`) if you plan to use DataRecorder for Excel files. Similarly, other format-specific libraries might be needed for their respective types.","message":"For `.xlsx` file handling, the `openpyxl` library is implicitly required, but not always listed as a hard dependency. If not installed, operations on `.xlsx` files will fail.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Always call `recorder.record()` periodically for explicit flushing, or `recorder.close()` at the end of your script. The safest approach is to use `DataRecorder` objects within a `with` statement, as it ensures `close()` is called automatically: `with Recorder('file.csv') as r: ...`.","message":"While DataRecorder handles caching, forgetting to call `record()` or `close()` (or using it in a `with` statement) may result in data not being flushed to the file, especially in short-lived scripts or on abnormal termination.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-16T00:00:00.000Z","next_check":"2026-07-15T00:00:00.000Z","problems":[{"fix":"Install the library using `pip install DataRecorder`. Ensure the import statement is `from DataRecorder import ...` with correct capitalization.","cause":"The DataRecorder library was not installed, or there's a typo in the import statement.","error":"ModuleNotFoundError: No module named 'DataRecorder'"},{"fix":"Ensure the directory path exists before initializing the Recorder: `import os; os.makedirs(os.path.dirname(file_path), exist_ok=True)`.","cause":"The specified directory for the output file does not exist, and DataRecorder by default might not create intermediate directories (though it aims to create the file itself).","error":"FileNotFoundError: [Errno 2] No such file or directory: 'non_existent_path/my_data.csv'"},{"fix":"Review the documentation or examples for the specific Recorder type and file format. For 3.x and above, ensure that data passed for 'db' and 'xlsx' corresponds to the expected dictionary format. Ensure data types match the target file structure (e.g., don't pass arbitrary strings to an Excel column expecting numbers).","cause":"After version 3.x, if you're working with 'db' or 'xlsx' formats, the `add_data` method might expect a different data structure (e.g., dictionary) than what you are providing, or you are trying to write incompatible data to a structured format.","error":"ValueError: Invalid data format for 'xlsx' or 'db'"}]}