What is the most effective way to generate 10 different responses with dp?

[ohwi]

I launched two models with the script below

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    -e CUDA_VISIBLE_DEVICES=0,1 \
    --ipc=host \
    lmsysorg/sglang:v0.4.1.post4-cu121 \
    python3 -m sglang.launch_server --model-path LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct --host 0.0.0.0 --port 30000 --dp-size 2

and ran 10 generations with the script below

import time
import requests


for _ in range(10):
    url = "http://localhost:30000/v1/chat/completions"
    data = {
        "model": "LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct",
        "messages": [{"role": "user", "content": "I want you to write a random code snippet."}],
        "sampling_params": {
            "temperature": 0.8,
            "top_p": 0.95,
        }
    }
    time.sleep(0.5)

    response = requests.post(url, json=data)
    print(response.json()

What I expected was 10 different generations, but the generated result was 5 pairs of 2 same generations like below:

{'id': '947d829bad41463c9ea7421bebece650', 'object': 'chat.completion', 'created': 1736329923, 'model': 'LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': 'Certainly! Here\'s a ra
ndom code snippet in Python that demonstrates a simple functionality for generating random passwords along with some basic validation to ensure they meet certain criteria (like length and inclusion of different character types):\n\n```py
thon\nimport random\nimport string\n\ndef generate_password(length=12):\n    """\n    Generates a random password of specified length with at least one lowercase letter,\n    one uppercase letter, one digit, and one special character.\n
   \n    :param length: Length of the password (default is 12)\n    :return: A randomly generated password string\n    """\n    if length < 4:\n        raise ValueError("Password length must be at least 4 characters to include all requir
ed character types.")\n    \n    # Define character pools\n    lowercase = string.ascii_lowercase\n    uppercase = string.ascii_uppercase\n    digits = string.digits\n    special_chars = string.punctuation\n    \n    # Ensure at least on
e character from each pool is included\n    password = [\n        random.choice(lowercase),\n        random.choice(uppercase),\n        random.choice(digits),\n        random.choice(special_chars)\n    ]\n    \n    # Fill the rest of the
 password length with random choices from all pools combined\n    all_chars = lowercase + uppercase + digits + special_chars\n    password += random.choices(all_chars, k=length-4)\n    \n    # Shuffle to avoid predictable patterns\n    r
andom.shuffle(password)\n    \n    # Convert list to string\n    return \'\'.join(password)\n\n# Example usage\nif __name__ == "__main__":\n    try:\n        password_length = 16  # You can change this value to generate passwords of diff
erent lengths\n        password = generate_password(password_length)\n        print(f"Generated Password: {password}")\n    except ValueError as e:\n        print(e)\n```\n\nThis snippet defines a function `generate_password` that create
s a secure password based on specified criteria and length. It ensures that the password includes at least one lowercase letter, one uppercase letter, one digit, and one special character, enhancing its security. You can adjust the `pass
word_length` variable in the example usage section to generate passwords of different lengths as needed.', 'tool_calls': None}, 'logprobs': None, 'finish_reason': 'stop', 'matched_stop': 361}], 'usage': {'prompt_tokens': 29, 'total_token
s': 554, 'completion_tokens': 525, 'prompt_tokens_details': None}}
{'id': '299ab1afdd484219b29a42919409c1bb', 'object': 'chat.completion', 'created': 1736329931, 'model': 'LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': 'Certainly! Here\'s a ra
ndom code snippet in Python that demonstrates a simple functionality for generating random passwords along with some basic validation to ensure they meet certain criteria (like length and inclusion of different character types):\n\n```py
thon\nimport random\nimport string\n\ndef generate_password(length=12):\n    """\n    Generates a random password of specified length with at least one lowercase letter,\n    one uppercase letter, one digit, and one special character.\n
   \n    :param length: Length of the password (default is 12)\n    :return: A randomly generated password string\n    """\n    if length < 4:\n        raise ValueError("Password length must be at least 4 characters to include all requir
ed character types.")\n    \n    # Define character pools\n    lowercase = string.ascii_lowercase\n    uppercase = string.ascii_uppercase\n    digits = string.digits\n    special_chars = string.punctuation\n    \n    # Ensure at least on
e character from each pool is included\n    password = [\n        random.choice(lowercase),\n        random.choice(uppercase),\n        random.choice(digits),\n        random.choice(special_chars)\n    ]\n    \n    # Fill the rest of the
 password length with random choices from all pools combined\n    all_chars = lowercase + uppercase + digits + special_chars\n    password += random.choices(all_chars, k=length-4)\n    \n    # Shuffle to avoid predictable patterns\n    r
andom.shuffle(password)\n    \n    # Convert list to string\n    return \'\'.join(password)\n\n# Example usage\nif __name__ == "__main__":\n    try:\n        password_length = 16  # You can change this value to generate passwords of diff
erent lengths\n        password = generate_password(password_length)\n        print(f"Generated Password: {password}")\n    except ValueError as e:\n        print(e)\n```\n\nThis snippet defines a function `generate_password` that create
s a secure password based on specified criteria and length. It ensures that the password includes at least one lowercase letter, one uppercase letter, one digit, and one special character, enhancing its security. You can adjust the `pass
word_length` variable in the example usage section to generate passwords of different lengths as needed.', 'tool_calls': None}, 'logprobs': None, 'finish_reason': 'stop', 'matched_stop': 361}], 'usage': {'prompt_tokens': 29, 'total_token
s': 554, 'completion_tokens': 525, 'prompt_tokens_details': None}}

(4 more pairs...)

How can I generate 10 different results with dp?