{"library":"math-verify","title":"HuggingFace Math-Verify","description":"Math-Verify is a robust Python library from HuggingFace, currently at version 0.9.0, designed for evaluating Large Language Model outputs in mathematical tasks. It provides sophisticated capabilities for parsing and verifying mathematical expressions, including LaTeX and plain numerical formats. The library supports complex features like set theory, equation/inequality comparison, and advanced normalization, aiming to offer higher accuracy in assessing LLM performance on math problems by moving beyond strict format requirements and inflexible comparison logic. It maintains an active development and release cadence.","language":"python","status":"active","last_verified":"Fri May 15","install":{"commands":["pip install math-verify","pip install math-verify[antlr4_13_2]"],"cli":null},"imports":["from math_verify import parse","from math_verify import verify","from math_verify import LatexExtractionConfig","from math_verify import ExprExtractionConfig","from math_verify import StringExtractionConfig"],"auth":{"required":false,"env_vars":[]},"quickstart":{"code":"from math_verify import parse, verify, LatexExtractionConfig, ExprExtractionConfig\n\n# Define extraction configurations\nextraction_configs = [LatexExtractionConfig(), ExprExtractionConfig()]\n\n# Parse the gold standard answer (e.g., from a dataset)\ngold_answer_text = \"${1,3} \\cup {2,4}$\"\ngold_parsed = parse(gold_answer_text, extraction_config=extraction_configs)\n\n# Parse the LLM generated answer\nllm_answer_text = \"${1,2,3,4}$\"\nllm_parsed = parse(llm_answer_text, extraction_config=extraction_configs)\n\n# Verify if the LLM's answer is mathematically equivalent to the gold standard\nis_correct = verify(gold_parsed, llm_parsed)\n\nprint(f\"Gold: {gold_answer_text} -> {gold_parsed}\")\nprint(f\"LLM: {llm_answer_text} -> {llm_parsed}\")\nprint(f\"Are answers equivalent? {is_correct}\")\n\n# Another example with an inequality and asymmetric comparison behavior\ngold_ineq = parse(\"1 < x < 2\")\nllm_interval = parse(\"(1,2)\")\nprint(f\"\\nGold (inequality): {gold_ineq}\")\nprint(f\"LLM (interval): {llm_interval}\")\nprint(f\"Are they equivalent (default)? {verify(gold_ineq, llm_interval)}\")\n\n# To allow symmetric comparison (e.g., if gold is interval and pred is inequality)\ngold_interval = parse(\"(1,2)\")\nllm_ineq = parse(\"1 < x < 2\")\nprint(f\"\\nGold (interval): {gold_interval}\")\nprint(f\"LLM (inequality): {llm_ineq}\")\nprint(f\"Are they equivalent (default)? {verify(gold_interval, llm_ineq)}\")\nprint(f\"Are they equivalent (allow_set_relation_comp=True)? {verify(gold_interval, llm_ineq, allow_set_relation_comp=True)}\")","lang":"python","description":"This quickstart demonstrates how to use `parse` to extract mathematical expressions from strings (both LaTeX and plain expressions) and `verify` to check for mathematical equivalence. It highlights the use of `ExtractionConfig` classes and illustrates the default asymmetric behavior for comparing intervals and inequalities.","tag":null,"tag_description":null,"last_tested":null,"results":[]},"compatibility":{"tag":null,"tag_description":null,"last_tested":"2026-05-15","installed_version":"0.9.0","pypi_latest":"0.9.0","is_stale":false,"summary":{"python_range":"3.10–3.9","success_rate":80,"avg_install_s":6.4,"avg_import_s":1.22,"wheel_type":"wheel"},"results":[{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.21,"mem_mb":34.4,"disk_size":"81.9M"},{"runtime":"python:3.10-alpine","python_version":"3.10","os_libc":"alpine (musl)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.22,"mem_mb":34.4,"disk_size":"81.9M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":6,"import_time_s":0.9,"mem_mb":34.4,"disk_size":"82M"},{"runtime":"python:3.10-slim","python_version":"3.10","os_libc":"slim (glibc)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":5.8,"import_time_s":0.91,"mem_mb":34.4,"disk_size":"82M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.46,"mem_mb":41.4,"disk_size":"109.3M"},{"runtime":"python:3.11-alpine","python_version":"3.11","os_libc":"alpine (musl)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.5,"mem_mb":41.4,"disk_size":"109.3M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":6.1,"import_time_s":1.32,"mem_mb":41.4,"disk_size":"110M"},{"runtime":"python:3.11-slim","python_version":"3.11","os_libc":"slim (glibc)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":6.1,"import_time_s":1.32,"mem_mb":41.4,"disk_size":"110M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.21,"mem_mb":39.3,"disk_size":"95.0M"},{"runtime":"python:3.12-alpine","python_version":"3.12","os_libc":"alpine (musl)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.2,"mem_mb":39.3,"disk_size":"95.0M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":6.6,"import_time_s":1.26,"mem_mb":39.3,"disk_size":"96M"},{"runtime":"python:3.12-slim","python_version":"3.12","os_libc":"slim (glibc)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":6.7,"import_time_s":1.28,"mem_mb":39.3,"disk_size":"96M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.17,"mem_mb":40.2,"disk_size":"94.3M"},{"runtime":"python:3.13-alpine","python_version":"3.13","os_libc":"alpine (musl)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":null,"import_time_s":1.18,"mem_mb":40.2,"disk_size":"94.3M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"math-verify","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":6.8,"import_time_s":1.2,"mem_mb":40.2,"disk_size":"95M"},{"runtime":"python:3.13-slim","python_version":"3.13","os_libc":"slim (glibc)","variant":"antlr4_13_2","exit_code":0,"wheel_type":"wheel","failure_reason":null,"import_side_effects":"clean","install_time_s":6.9,"import_time_s":1.18,"mem_mb":40.2,"disk_size":"95M"},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"math-verify","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-alpine","python_version":"3.9","os_libc":"alpine (musl)","variant":"antlr4_13_2","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":null,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"math-verify","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":1.6,"import_time_s":null,"mem_mb":null,"disk_size":null},{"runtime":"python:3.9-slim","python_version":"3.9","os_libc":"slim (glibc)","variant":"antlr4_13_2","exit_code":1,"wheel_type":null,"failure_reason":"build_error","import_side_effects":null,"install_time_s":1.6,"import_time_s":null,"mem_mb":null,"disk_size":null}]}}