HTMLHeaderTextSplitter Constructor Raises ValueError For Non-Heading Tags That Its Own Engine Supports
The constructor of HTMLHeaderTextSplitter sorts headers_to_split_on using a bare int(tag[1:]) conversion, which fails for any tag outside h1-h6 (e.g., div -> 'iv'). The _generate_documents method already contains a fallback level of 9999 for such tags, causing an internal inconsistency that crashes at initialization.
In langchain_text_splitters/html.py, __init__ sorts headers_to_split_on with key=lambda x: int(x[0][1:]) without handling ValueError, while _generate_documents uses a try/except to fallback to level 9999 for non-numeric tags. This discrepancy rejects valid tags that the splitting engine deliberately supports.
Run the following Python code:
from langchain_text_splitters import HTMLHeaderTextSplitter
HTMLHeaderTextSplitter(headers_to_split_on=[("h1", "Header 1"), ("div", "Div")])
This raises ValueError: invalid literal for int() with base 10: 'iv'.
Fixing Code Block
def _get_header_level(tag: str) -> int:
try:
return int(tag[1:])
except ValueError:
return 9999
# In HTMLHeaderTextSplitter.__init__ replace the sorting line with:
self.headers_to_split_on = sorted(
headers_to_split_on, key=lambda x: _get_header_level(x[0])
)
Introduce a shared helper _get_header_level that encapsulates the same fallback logic already present in _generate_documents. The constructor then uses this helper for sorting, ensuring that non-heading tags receive level 9999 and sort after all numbered headings, matching the engine's behavior.
Edge Case Audit
This change aligns constructor sorting with the existing splitting logic, so no behavioral change occurs for h1-h6 tags. The fallback level 9999 was already used internally. To roll back, restore the original sorting line. Potential edge case: if a tag string is empty or malformed, int('') also raises ValueError and is caught by the same fallback, so no new crash. Ensure unit tests cover non-heading tags and existing h1-h6 order.
