Linear TSV

1.1.0 · active · verified Thu Apr 16

Linear TSV is a Python library providing a simple, line-oriented, and portable tabular data format. Unlike standard CSV, it uses escape codes for newlines and tabs within field data, enabling robust processing with line-oriented shell tools. The format aligns with the TEXT serialization mode of Postgres and MySQL, making it ideal for reliable data exchange. It is currently at version 1.1.0 and maintains an active release cadence.

Common errors

```
ValueError: need more than 1 value to unpack (or similar errors indicating incorrect column count)
```
cause This usually occurs when a line in the TSV is not correctly split into the expected number of columns. This is often due to embedded newlines or tabs in a field that are not properly escaped or are being parsed by a tool that doesn't understand the `linear-tsv` escape conventions.

fix Ensure the input TSV strictly adheres to the `linear-tsv` format's escape rules for newlines, tabs, and backslashes. If the file is supposed to be `linear-tsv` compliant, use `linear-tsv.tsv.un()` to parse it. If it's a standard TSV without embedded delimiters, verify your parser (e.g., `line.strip().split('\t')` or `csv.reader(file, delimiter='\t')`) is correctly configured.
```
Data appears as single-column strings or contains unparsed escape sequences like '\\n' or '\\t'.
```
cause The TSV data contains `linear-tsv` specific escape sequences (`\n`, `\t`, `\r`, `\\`) within fields, but it is being read by a generic text reader or a CSV parser that does not interpret these sequences as delimiters.

fix Utilize the `linear-tsv.tsv.un()` function for parsing. This library is specifically designed to correctly decode these escape sequences into their intended characters, ensuring proper field separation and data integrity for this particular TSV format.

Warnings

gotcha The `linear-tsv` library implements a specific TSV format where newlines (\n), tabs (\t), carriage returns (\r), and backslashes (\\) within field data *must* be escaped. Generic TSV parsers (e.g., `csv.reader` with `delimiter='\t'`) will not correctly interpret these escape sequences, leading to data corruption or incorrect parsing.
Fix: Always use `linear-tsv.tsv.un()` for parsing files adhering to the linear-tsv format. If using other parsers, ensure they are configured to handle these specific escape sequences (e.g., `quoting=csv.QUOTE_NONE` in pandas and custom escape handling).
gotcha The `linear-tsv` format does not natively include or interpret header rows. The `tsv.un()` function treats all lines as data rows.
Fix: If your TSV file includes a header, you must manually separate the first row after parsing to use as column names. For example, `header = parsed_data[0]` and `data_rows = parsed_data[1:]`.
gotcha The format uses `\N` (backslash followed by capital N) to represent `NULL` values, akin to PostgreSQL/MySQL TEXT format. The `tsv.un()` function returns `\N` as a literal string.
Fix: Post-process the parsed data to convert `"\\N"` strings to `None` or your desired representation for null values. Example: `[None if x == '\\N' else x for x in row]`.

Install

pip install linear-tsv Install with pip

Imports

un
wrong:
```
from linear_tsv import un
```
correct:
```
import tsv
lists = tsv.un(data_stream)
```
The PyPI package `linear-tsv` exposes its functionality through the `tsv` top-level module.

Quickstart

This quickstart demonstrates how to parse a `linear-tsv` formatted string using the `tsv.un()` function. It shows how to handle embedded newlines and tabs using their defined escape sequences (`\n`, `\t`) and also the `\N` representation for `NULL` values. The `tsv.un()` function returns a generator of lists of strings.

import tsv
import io

# Example TSV data string with escaped newline and tab
data_string = "col1\tcol2\nval1a\\nwithnewline\tval1b\\twithtab\nval2a\t\\N"

# Use io.StringIO to simulate a file stream
data_stream = io.StringIO(data_string)

# Parse the TSV stream to a generator of lists of strings
parsed_data = list(tsv.un(data_stream))

# Print the parsed data
for row in parsed_data:
    print(row)

# Output:
# ['col1', 'col2']
# ['val1a\nwithnewline', 'val1b\twithtab']
# ['val2a', '\N']

view raw JSON →