datetime objects in row_filter expressions are not casted and raise an error

[jayceslesar]

Feature Request / Improvement

in the following row_filter in a scan() call,

row_filter=And(
            GreaterThanOrEqual("timestamp_received", start_time),
            LessThan("timestamp_received", end_time),
        ),

the start_time and end_time variables are datetime objects. When running a scan with that row_filter I end up with the error of

TypeError: Invalid literal value: datetime.datetime(2024, 11, 13, 11, 35, 28, 730000, tzinfo=datetime.timezone.utc)

Instead, the expressions should correctly cast the datetime object instead of expecting an isoformat string, which is the current behavior

[kevinjqliu]

Thanks for reporting this issue!
I believe this is the relevant code

iceberg-python/pyiceberg/expressions/literals.py

Lines 122 to 149 in dbcf65b

	def literal(value: L) -> Literal[L]:
	"""
	Construct an Iceberg Literal based on Python primitive data type.
	Args:
	value (Python primitive type): the value to be associated with literal.
	Example:
	from pyiceberg.expressions.literals import literal.
	>>> literal(123)
	LongLiteral(123)
	"""
	if isinstance(value, float):
	return DoubleLiteral(value) # type: ignore
	elif isinstance(value, bool):
	return BooleanLiteral(value)
	elif isinstance(value, int):
	return LongLiteral(value)
	elif isinstance(value, str):
	return StringLiteral(value)
	elif isinstance(value, UUID):
	return UUIDLiteral(value.bytes) # type: ignore
	elif isinstance(value, bytes):
	return BinaryLiteral(value)
	elif isinstance(value, Decimal):
	return DecimalLiteral(value)
	else:
	raise TypeError(f"Invalid literal value: {repr(value)}")

Looks like we don't take into account python's datetime object.

cc @Fokko do you have context around this?

[jayceslesar]

Any thoughts here? there are a few ways to implement this from what I can tell, the easiest would be to just add a clause to the literal function that returns TimestampLiteral(datetime_to_micros(value)) if value is a datetime

[kunnapatt]

@jayceslesar
Thanks I got the same problem, but when I followed your comment, It worked.