[Bug] Doris 3.0.3 [INTERNAL_ERROR]Invalid JSON format while reading the variant data type col

[kumarShailendra1]

Search before asking

• I had searched in the issues and found no similar issues.

Version

I am using doris 3.0.3 version.

What's Wrong?

We are facing an issue while querying the variant column, which holds nested JSON data.
We checked the column data content, and it contains valid JSON data.
Query:
SELECT external_paylaod FROM pipelineTask;

external_paylaod column data type: variant

Sample nested JSON in column external_paylaod data:
{
"workflow_settings": {
"config_params": {
"thread_count": 4,
"data_source": [
{
"client_id": "RANDOM_CLIENT",
"client_name": "generic_client_name",
"hierarchy": {
"Tier1": [
"random_value_1"
],
"Tier2": [
"generic_value_2"
],
"Tier3": [
"random_value_3"
]
},
"project_name": "Multi Client Project",
"ecosystem": "generic_ecosystem",
"timeframes": [
"1D",
"1W",
"1M",
"3M"
],
"sub_ecosystem": "generic_sub_ecosystem",
"category": "random_category"
}
]
}
},
"priority": 2,
"execution_details": {
"task_settings": {
"description": "Generic task execution for multi-ecosystem scenarios.",
"arguments": "--workflow_id abc-1234-5678-xyz",
"task_name": "generic-task-name-98765",
"parameters": {
"engine_config": {
"kubernetes.worker.cpu": 6,
"failure-strategy": "linear-delay",
"failure-strategy.fixed-delay.retries": 3,
"failure-strategy.fixed-delay.duration": "10 s",
"worker.memory.allocated.ratio": 0.3,
"worker.memory.process.size": "12000m",
"worker.network.shuffle.compression.enabled": "true",
"worker.concurrentThreads": 6
},
"engine_version": "2.0-python310",
"credentials": [
"random_s3_bucket",
"random_postgres_url",
"random_analytics_host",
"random_analytics_port",
"random_analytics_user",
"random_analytics_password",
"random_analytics_database",
"random_insights_table",
"random_dev_url",
"random_prod_url",
"random_db_table",
"random_db_pool_size",
"random_db_username",
"random_db_password",
"random_db_host",
"random_db_port",
"random_db_name",
"random_estimation_api_key",
"random_estimation_api_url",
"random_estimation_api_timeout",
"random_estimation_api_retries",
"random_estimation_api_backoff",
"random_ai_host",
"random_ai_key",
"random_api_host",
"random_api_url",
"random_sync_db_host",
"random_sync_db_port",
"random_sync_db_user",
"random_sync_db_password",
"random_sync_db_name",
"random_ai_model",
"random_ai_tokens",
"random_ai_temperature"
]
},
"backup_config": {
"active": 1
},
"category": "python"
},
"runtime_details": {
"task_scale": "large",
"task_count": 24
}
}
}
Note: This error occurs intermittently.
Error:
Query 1 ERROR: errCode = 2, detailMessage = (dp-doriscluster-prod-be-1.dp-doriscluster-prod-be-internal.prod-doris.svc.cluster.local)[INTERNAL_ERROR]Invalid JSON format

What You Expected?

This issue should not be come . I think there might be issue in the serializer or deserializer of the variant data type while reading the nested Json data col.

How to Reproduce?

2. Create a table with a col name external_payload with variant data type
3. insert the above json data into the col external_payload
4. execute the select * from table query multiple time
5. you will get the same issue

Error :

Query 1 ERROR: errCode = 2, detailMessage = (dp-doriscluster-prod-be-1.dp-doriscluster-prod-be-internal.prod-doris.svc.cluster.local)[INTERNAL_ERROR]Invalid JSON format

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct