Skip to content

Commit

Permalink
changed keywords placement
Browse files Browse the repository at this point in the history
  • Loading branch information
@Raghul-M
Raghul-M committed Jan 9, 2025
1 parent ef24191 commit 3986728
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 119 deletions.
60 changes: 57 additions & 3 deletions ods_ci/tests/Resources/CLI/ModelServing/llm.resource
View file
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ Compile Inference Service YAML
[Arguments] ${isvc_name} ${model_storage_uri} ${model_format}=caikit ${serving_runtime}=caikit-tgis-runtime
... ${kserve_mode}=${NONE} ${sa_name}=${DEFAULT_BUCKET_SA_NAME} ${canaryTrafficPercent}=${EMPTY} ${min_replicas}=1
... ${scaleTarget}=1 ${scaleMetric}=concurrency ${auto_scale}=${NONE}
... ${requests_dict}=&{EMPTY} ${limits_dict}=&{EMPTY} ${overlays}=${EMPTY}
... ${requests_dict}=&{EMPTY} ${limits_dict}=&{EMPTY} ${overlays}=${EMPTY} ${version}=${EMPTY}

Check notice

Code scanning / Robocop

There is too many arguments per continuation line ({{ arguments_count }} / {{ max_arguments_count }}) Note test

There is too many arguments per continuation line (4 / 1)
IF '${auto_scale}' == '${NONE}'
${scaleTarget}= Set Variable ${EMPTY}
${scaleMetric}= Set Variable ${EMPTY}
Expand All @@ -154,6 +154,7 @@ Compile Inference Service YAML
Set Test Variable ${scaleMetric}
Set Test Variable ${canaryTrafficPercent}
Set Test Variable ${model_format}
Set Test Variable ${version}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${serving_runtime}
IF len($overlays) > 0
FOR ${index} ${overlay} IN ENUMERATE @{overlays}
Expand Down Expand Up @@ -195,10 +196,9 @@ Compile Inference Service YAML
ELSE
${mode}= Get KServe Default Deployment Mode From DSC
END
Log message=Using defaultDeploymentMode set in the DSC: ${mode}
Log message=Using defaultDeploymentMode set in the DSC: ${mode}S
END


Model Response Should Match The Expectation
[Documentation] Checks that the actual model response matches the expected answer.
... The goals are:
Expand Down Expand Up @@ -415,6 +415,46 @@ Query Model Multiple Times
END
END

Setup Test Variables # robocop: off=too-many-calls-in-keyword
[Documentation] Sets up variables for the Suite
[Arguments] ${model_name} ${kserve_mode}=Serverless ${use_pvc}=${FALSE} ${use_gpu}=${FALSE}
... ${model_path}=${model_name}

Check notice

Code scanning / Robocop

There is too many arguments per continuation line ({{ arguments_count }} / {{ max_arguments_count }}) Note test

There is too many arguments per continuation line (4 / 1)
Set Test Variable ${model_name}
${models_names}= Create List ${model_name}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${models_names}

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create List can be replaced with VAR
Set Test Variable ${model_path}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${test_namespace} ${TEST_NS}-${model_name}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
IF ${use_pvc}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${storage_uri} pvc://${model_name}-claim/${model_path}
ELSE

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${storage_uri} s3://${S3.BUCKET_1.NAME}/${model_path}
END

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
IF ${use_gpu}
${supported_gpu_type}= Convert To Lowercase ${GPU_TYPE}
Set Runtime Image ${supported_gpu_type}
IF "${supported_gpu_type}" == "nvidia"
${limits}= Create Dictionary nvidia.com/gpu=1
ELSE IF "${supported_gpu_type}" == "amd"

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create Dictionary can be replaced with VAR
${limits}= Create Dictionary amd.com/gpu=1
ELSE

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create Dictionary can be replaced with VAR
FAIL msg=Provided GPU type is not yet supported. Only nvidia and amd gpu type are supported
END
Set Test Variable ${limits}
ELSE

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${limits} &{EMPTY}
END

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
IF "${KSERVE_MODE}" == "RawDeployment" # robocop: off=inconsistent-variable-name
Set Test Variable ${use_port_forwarding} ${TRUE}
ELSE

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${use_port_forwarding} ${FALSE}
END

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Log Level NONE
Set Test Variable ${access_key_id} ${S3.AWS_ACCESS_KEY_ID}
Set Test Variable ${access_key} ${S3.AWS_SECRET_ACCESS_KEY}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${endpoint} ${MODELS_BUCKET.ENDPOINT}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Test Variable ${region} ${MODELS_BUCKET.REGION}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
Set Log Level INFO

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase

Compile Deploy And Query LLM model
[Documentation] Group together the test steps for preparing, deploying
... and querying a model
Expand Down Expand Up @@ -911,3 +951,17 @@ Remove Model Mount Path From Runtime
... oc patch servingruntime ${runtime} -n ${namespace} --type='json' -p='[{"op": "remove", "path": "/spec/containers/0/args/1"}]'
Should Be Equal As Integers ${rc} ${0} msg=${out}

Set Runtime Image
[Documentation] Sets up runtime variables for the Suite
[Arguments] ${gpu_type}
IF "${RUNTIME_IMAGE}" == "${EMPTY}"
IF "${gpu_type}" == "nvidia"
Set Test Variable ${runtime_image} quay.io/modh/vllm@sha256:c86ff1e89c86bc9821b75d7f2bbc170b3c13e3ccf538bf543b1110f23e056316
ELSE IF "${gpu_type}" == "amd"
Set Test Variable ${runtime_image} quay.io/modh/vllm@sha256:10f09eeca822ebe77e127aad7eca2571f859a5536a6023a1baffc6764bcadc6e

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (142/120)

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
ELSE
FAIL msg=Provided GPU type is not yet supported. Only nvidia and amd gpu type are supported

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (142/120)

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR

Check warning

Code scanning / Robocop

Don't use test/task variables Warning test

Don't use test/task variables

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
END
ELSE

Check warning

Code scanning / Robocop

{{ bad_indent_msg }} Warning test

Line is over-indented
Log To Console msg= Using the image provided from terminal

Check warning

Code scanning / Robocop

{{ bad_indent_msg }} Warning test

Line is under-indented
END
119 changes: 3 additions & 116 deletions ...rving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve_api.robot
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ ${PROTOBUFF_FILE}= tests/Resources/Files/triton/grpc_predict_v2.proto
*** Test Cases ***
Test Python Model Grpc Inference Via API (Triton on Kserve) # robocop: off=too-long-test-case
[Documentation] Test the deployment of python model in Kserve using Triton

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Test Python Model Grpc Inference Via API (Triton on Kserve)' has too many keywords inside (18/10)
[Tags] Tier2 RHOAIENG-16912
[Tags] Tier2 RHOAIENG-16912 RunThisTest
Setup Test Variables model_name=${PYTHON_MODEL_NAME} use_pvc=${FALSE} use_gpu=${FALSE}
... kserve_mode=${KSERVE_MODE} model_path=triton/model_repository/
Expand Down Expand Up @@ -97,121 +97,8 @@ Suite Teardown
Set Default Storage Class In GCP default=standard-csi
RHOSi Teardown

Setup Test Variables # robocop: off=too-many-calls-in-keyword
[Documentation] Sets up variables for the Suite
[Arguments] ${model_name} ${kserve_mode}=Serverless ${use_pvc}=${FALSE} ${use_gpu}=${FALSE}
... ${model_path}=${model_name}
Set Test Variable ${model_name}
${models_names}= Create List ${model_name}
Set Test Variable ${models_names}
Set Test Variable ${model_path}
Set Test Variable ${test_namespace} ${TEST_NS}-${model_name}
IF ${use_pvc}
Set Test Variable ${storage_uri} pvc://${model_name}-claim/${model_path}
ELSE
Set Test Variable ${storage_uri} s3://${S3.BUCKET_1.NAME}/${model_path}
END
IF ${use_gpu}
${supported_gpu_type}= Convert To Lowercase ${GPU_TYPE}
Set Runtime Image ${supported_gpu_type}
IF "${supported_gpu_type}" == "nvidia"
${limits}= Create Dictionary nvidia.com/gpu=1
ELSE IF "${supported_gpu_type}" == "amd"
${limits}= Create Dictionary amd.com/gpu=1
ELSE
FAIL msg=Provided GPU type is not yet supported. Only nvidia and amd gpu type are supported
END
Set Test Variable ${limits}
ELSE
Set Test Variable ${limits} &{EMPTY}
END
IF "${KSERVE_MODE}" == "RawDeployment" # robocop: off=inconsistent-variable-name
Set Test Variable ${use_port_forwarding} ${TRUE}
ELSE
Set Test Variable ${use_port_forwarding} ${FALSE}
END
Set Log Level NONE
Set Test Variable ${access_key_id} ${S3.AWS_ACCESS_KEY_ID}
Set Test Variable ${access_key} ${S3.AWS_SECRET_ACCESS_KEY}
Set Test Variable ${endpoint} ${MODELS_BUCKET.ENDPOINT}
Set Test Variable ${region} ${MODELS_BUCKET.REGION}
Set Log Level INFO

Set Runtime Image
[Documentation] Sets up runtime variables for the Suite
[Arguments] ${gpu_type}
IF "${RUNTIME_IMAGE}" == "${EMPTY}"
IF "${gpu_type}" == "nvidia"
Set Test Variable ${runtime_image} quay.io/modh/vllm@sha256:c86ff1e89c86bc9821b75d7f2bbc170b3c13e3ccf538bf543b1110f23e056316
ELSE IF "${gpu_type}" == "amd"
Set Test Variable ${runtime_image} quay.io/modh/vllm@sha256:10f09eeca822ebe77e127aad7eca2571f859a5536a6023a1baffc6764bcadc6e
ELSE
FAIL msg=Provided GPU type is not yet supported. Only nvidia and amd gpu type are supported
END
ELSE
Log To Console msg= Using the image provided from terminal
END

Compile Inference Service YAML
[Documentation] Prepare the Inference Service YAML file in order to deploy a model
[Arguments] ${isvc_name} ${model_storage_uri} ${model_format}=caikit ${serving_runtime}=caikit-tgis-runtime
... ${kserve_mode}=${NONE} ${sa_name}=${DEFAULT_BUCKET_SA_NAME} ${canaryTrafficPercent}=${EMPTY} ${min_replicas}=1
... ${scaleTarget}=1 ${scaleMetric}=concurrency ${auto_scale}=${NONE}
... ${requests_dict}=&{EMPTY} ${limits_dict}=&{EMPTY} ${overlays}=${EMPTY} ${version}=${EMPTY}
IF '${auto_scale}' == '${NONE}'
${scaleTarget}= Set Variable ${EMPTY}
${scaleMetric}= Set Variable ${EMPTY}
END
Set Test Variable ${isvc_name}
Set Test Variable ${min_replicas}
Set Test Variable ${sa_name}
Set Test Variable ${model_storage_uri}
Set Test Variable ${scaleTarget}
Set Test Variable ${scaleMetric}
Set Test Variable ${canaryTrafficPercent}
Set Test Variable ${model_format}
Set Test Variable ${version}
Set Test Variable ${serving_runtime}
IF len($overlays) > 0
FOR ${index} ${overlay} IN ENUMERATE @{overlays}
Log ${index}: ${overlay}
${rc} ${out}= Run And Return Rc And Output
... oc kustomize ${LLM_RESOURCES_DIRPATH}/serving_runtimes/overlay/${overlay} > ${INFERENCESERVICE_FILLED_FILEPATH}
Should Be Equal As Integers ${rc} ${0} msg=${out}
END
Create File From Template ${INFERENCESERVICE_FILLED_FILEPATH} ${INFERENCESERVICE_FILLED_FILEPATH}
ELSE
Create File From Template ${INFERENCESERVICE_FILEPATH} ${INFERENCESERVICE_FILLED_FILEPATH}
END
IF ${requests_dict} != &{EMPTY}
Log Adding predictor model requests to ${INFERENCESERVICE_FILLED_FILEPATH}: ${requests_dict} console=True # robocop: disable
FOR ${index} ${resource} IN ENUMERATE @{requests_dict.keys()}
Log ${index}- ${resource}:${requests_dict}[${resource}]
${rc} ${out}= Run And Return Rc And Output
... yq -i '.spec.predictor.model.resources.requests."${resource}" = "${requests_dict}[${resource}]"' ${INFERENCESERVICE_FILLED_FILEPATH} # robocop: disable
Should Be Equal As Integers ${rc} ${0} msg=${out}
END
END
IF ${limits_dict} != &{EMPTY}
Log Adding predictor model limits to ${INFERENCESERVICE_FILLED_FILEPATH}: ${limits_dict} console=True # robocop: disable
FOR ${index} ${resource} IN ENUMERATE @{limits_dict.keys()}
Log ${index}- ${resource}:${limits_dict}[${resource}]
${rc} ${out}= Run And Return Rc And Output
... yq -i '.spec.predictor.model.resources.limits."${resource}" = "${limits_dict}[${resource}]"' ${INFERENCESERVICE_FILLED_FILEPATH} # robocop: disable
Should Be Equal As Integers ${rc} ${0} msg=${out}
END
END
IF $kserve_mode is not None
${rc} ${out}= Run And Return Rc And Output
... yq -i '.metadata.annotations."serving.kserve.io/deploymentMode" = "${kserve_mode}"' ${INFERENCESERVICE_FILLED_FILEPATH} # robocop: disable
Should Be Equal As Integers ${rc} ${0} msg=${out}
ELSE
${exists}= Run Keyword And Return Status Variable Should Exist ${DSC_KSERVE_MODE}
IF ${exists} # done in this way because when use non-admin users they cannot fetch DSC
${mode}= Set Variable ${DSC_KSERVE_MODE}
ELSE
${mode}= Get KServe Default Deployment Mode From DSC
END
Log message=Using defaultDeploymentMode set in the DSC: ${mode}S
END



0 comments on commit 3986728

Please sign in to comment.