{"id":9612,"library":"collate-dbt-artifacts-parser","title":"Collate dbt Artifacts Parser","description":"Collate dbt Artifacts Parser is a Python library that provides a structured way to parse dbt artifacts like `manifest.json` and `run_results.json` into Pydantic models. It leverages `dbt-artifacts-parser` and simplifies access to dbt project metadata. The current version is 0.1.4, and its release cadence is tied to internal dbt-labs development, though it's publicly available.","status":"active","version":"0.1.4","language":"en","source_language":"en","source_url":"https://github.com/dbt-labs-internal/collate-dbt-artifacts-parser","tags":["dbt","data-governance","metadata","parsing","artifacts","data-ops"],"install":[{"cmd":"pip install collate-dbt-artifacts-parser","lang":"bash","label":"Install stable version"}],"dependencies":[{"reason":"Core parsing logic for dbt artifact schemas.","package":"dbt-artifacts-parser","optional":false},{"reason":"Used for data validation and parsing dbt artifact JSON into Python models.","package":"pydantic","optional":false}],"imports":[{"symbol":"DbtArtifactParser","correct":"from collate_dbt_artifacts_parser.parsers import DbtArtifactParser"},{"note":"This is a convenience function, but DbtArtifactParser offers more control over artifact type and version.","symbol":"parse_artifact","correct":"from collate_dbt_artifacts_parser.parser import parse_artifact"}],"quickstart":{"code":"import json\nimport os\nfrom collate_dbt_artifacts_parser.parsers import DbtArtifactParser\n\n# Mock a simple manifest.json structure (truncated for brevity)\nmock_manifest_content = {\n    \"metadata\": {\n        \"dbt_schema_version\": \"https://schemas.getdbt.com/dbt/manifest/v8.json\",\n        \"dbt_version\": \"1.8.0\",\n        \"generated_at\": \"2024-01-01T00:00:00Z\",\n        \"invocation_id\": \"mock_invocation\",\n        \"env\": {},\n        \"adapter_type\": \"postgres\"\n    },\n    \"nodes\": {},\n    \"sources\": {},\n    \"macros\": {}\n}\n\n# Create a dummy file\ndummy_file_path = \"mock_manifest.json\"\nwith open(dummy_file_path, \"w\") as f:\n    json.dump(mock_manifest_content, f)\n\ntry:\n    # Initialize parser for 'manifest' artifact of schema version 'v8'\n    parser = DbtArtifactParser(artifact_type=\"manifest\", version=\"v8\")\n    manifest_model = parser.parse_artifact_file(dummy_file_path)\n\n    print(f\"Successfully parsed manifest of type: {type(manifest_model)}\")\n    print(f\"DBT Version from manifest metadata: {manifest_model.metadata.dbt_version}\")\n\nexcept Exception as e:\n    print(f\"An error occurred during parsing: {e}\")\nfinally:\n    # Clean up the dummy file\n    if os.path.exists(dummy_file_path):\n        os.remove(dummy_file_path)\n","lang":"python","description":"This quickstart demonstrates how to parse a dbt `manifest.json` file using `DbtArtifactParser`. It creates a mock manifest file, initializes the parser with the correct artifact type and schema version, and then parses the file into a Pydantic model. Remember to replace the mock content and path with your actual dbt artifact file."},"warnings":[{"fix":"Always check the `dbt_schema_version` field within your dbt artifact JSON and use the corresponding `vX` in the `DbtArtifactParser` constructor. Consult the `dbt-artifacts-parser` documentation for a mapping of schema URLs to `vX` identifiers.","message":"dbt artifact schema versions (`manifest.json`, `run_results.json`) change frequently with dbt-core releases. Ensure the `version` parameter passed to `DbtArtifactParser` (e.g., 'v8') matches the `dbt_schema_version` within your artifact file (e.g., 'https://schemas.getdbt.com/dbt/manifest/v8.json'). Mismatches will lead to parsing errors.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Be mindful of explicit version pinning for both `collate-dbt-artifacts-parser` and its dependency `dbt-artifacts-parser`. If encountering unexpected parsing issues, verify the compatibility of your installed `dbt-core` version with the `dbt-artifacts-parser` library.","message":"This library relies heavily on `dbt-artifacts-parser` for the core schema definitions and parsing logic. While `collate-dbt-artifacts-parser` specifies `dbt-artifacts-parser==1.0.2`, future changes in `dbt-artifacts-parser` or `dbt-core`'s artifact schemas could introduce incompatibilities.","severity":"gotcha","affected_versions":"All versions"},{"fix":"Exercise caution and thorough testing when upgrading versions, especially in critical pipelines. Rely primarily on the documented API and assume the primary goal is to serve dbt-labs' internal needs first.","message":"The library's GitHub repository (`dbt-labs-internal`) indicates its origin as an internal dbt-labs project. While publicly available, this might imply its primary development focus is internal use cases, potentially leading to less emphasis on backward compatibility for external users or a slower response to external feature requests/bug reports.","severity":"gotcha","affected_versions":"All versions"}],"env_vars":null,"last_verified":"2026-04-17T00:00:00.000Z","next_check":"2026-07-16T00:00:00.000Z","problems":[{"fix":"Verify the path to your dbt artifact file. Use an absolute path or ensure the relative path is correct from where your script is executed.","cause":"The specified artifact file path does not exist or is incorrect.","error":"FileNotFoundError: [Errno 2] No such file or directory: 'path/to/your/manifest.json'"},{"fix":"Double-check that `artifact_type` is one of 'manifest', 'run_results', 'catalog', etc., and `version` correctly corresponds to the dbt schema version (e.g., 'v8' for `dbt/manifest/v8.json`). Ensure consistency between the artifact type, its version, and the actual content of the file.","cause":"The `artifact_type` or `version` parameter provided to `DbtArtifactParser` does not match a known/supported dbt artifact schema.","error":"ValueError: Invalid artifact type or version combination: manifest/vX"},{"fix":"Ensure the `version` parameter in `DbtArtifactParser` accurately reflects the `dbt_schema_version` found within the artifact file itself. If you're unsure, inspect the `dbt_schema_version` field in your artifact JSON.","cause":"The content of the dbt artifact file does not conform to the schema specified by the chosen `artifact_type` and `version` (e.g., trying to parse a dbt v1.0 manifest with a `version='v8'` parser).","error":"pydantic.error_wrappers.ValidationError: 1 validation error for ManifestV8\nmetadata -> dbt_schema_version\n  value is not a valid enumeration member; permitted: ('https://schemas.getdbt.com/dbt/manifest/v8.json', ...)"}]}