Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ImportError: cannot import name 'DEFAULT_CHECKSUM_ALGORITHM' from 'botocore.httpchecksum' #4399

Open
1 task
Christhof25 opened this issue Jan 17, 2025 · 3 comments
Open
1 task

ImportError: cannot import name 'DEFAULT_CHECKSUM_ALGORITHM' from 'botocore.httpchecksum' #4399

Christhof25 opened this issue Jan 17, 2025 · 3 comments
Assignees
@RyanFitzSimmonsAK
Labels
bug This issue is a confirmed bug. p2 This is a standard priority issue response-requested Waiting on additional information or feedback. s3

Comments

@Christhof25
Copy link

Describe the bug

Into a script, when I want execute this command :
aws sts get-caller-identity

I have this Error
ImportError: cannot import name 'DEFAULT_CHECKSUM_ALGORITHM' from 'botocore.httpchecksum' (/home/ubuntu/.local/lib/python3.10/site-packages/botocore/httpchecksum.py)

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

12:15:44        2. Update the lastet version of AwsCli, Python and Boto
12:15:46      INVENTORY for BOTO
12:15:47   boto                       2.49.0
12:15:47   boto3                      1.20.34
12:15:47   botocore                   1.35.97
12:15:49   
12:15:49         - Install de Boto3 avec version 1.35.78
12:15:53         - Install de Botocore avec version 1.35.78
12:15:59   
12:15:59      AFTER for BOTO
12:16:00   boto                       2.49.0
12:16:00   boto3                      1.35.78
12:16:00   botocore                   1.35.78
12:16:00   
12:16:01   UPDATE AWSCLI, BOTO, PYTHON => OK
12:16:01   
12:16:01        3. Retrieving AccessKey ID and password from TAM
12:16:02   AccessKey ID and password from TAM => OK
12:16:02   
12:16:02        4. Check connexion AWS
12:16:04   Connexion AWS => OK

Current Behavior

[03:25:30] :	 [Step 2/2] Traceback (most recent call last):
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/bin/aws", line 27, in <module>
[03:25:30] :	 [Step 2/2]     sys.exit(main())
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/bin/aws", line 23, in main
[03:25:30] :	 [Step 2/2]     return awscli.clidriver.main()
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/clidriver.py", line 73, in main
[03:25:30] :	 [Step 2/2]     driver = create_clidriver()
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/clidriver.py", line 82, in create_clidriver
[03:25:30] :	 [Step 2/2]     load_plugins(
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/plugin.py", line 44, in load_plugins
[03:25:30] :	 [Step 2/2]     modules = _import_plugins(plugin_mapping)
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/plugin.py", line 61, in _import_plugins
[03:25:30] :	 [Step 2/2]     module = __import__(path, fromlist=[module])
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/handlers.py", line 27, in <module>
[03:25:30] :	 [Step 2/2]     from awscli.customizations.cloudformation import (
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/customizations/cloudformation/__init__.py", line 13, in <module>
[03:25:30] :	 [Step 2/2]     from awscli.customizations.cloudformation.package import PackageCommand
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/customizations/cloudformation/package.py", line 26, in <module>
[03:25:30] :	 [Step 2/2]     from awscli.customizations.s3uploader import S3Uploader
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/customizations/s3uploader.py", line 22, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.manager import TransferManager
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/manager.py", line 25, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.copies import CopySubmissionTask
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/copies.py", line 16, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.tasks import (
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 16, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.utils import get_callbacks
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/utils.py", line 29, in <module>
[03:25:30] :	 [Step 2/2]     from botocore.httpchecksum import DEFAULT_CHECKSUM_ALGORITHM, AwsChunkedWrapper
[03:25:30] :	 [Step 2/2] ImportError: cannot import name 'DEFAULT_CHECKSUM_ALGORITHM' from 'botocore.httpchecksum' (/home/ubuntu/.local/lib/python3.10/site-packages/botocore/httpchecksum.py)

Reproduction Steps

     export AWS_ACCESS_KEY_ID=*******
     export AWS_SECRET_ACCESS_KEY=*********
    aws sts get-caller-identity

Possible Solution

I install the same version of botocore and boto3

   #Choix d'une version commune
   VERS_CIBLE="1.35.78"

   #J'installe la version boto3
   BOTO3_ERR=$UNXTMP/${UNXJOB}_$$_boto3.err
   if [ ${VERS_CIBLE} != ${VERS_BOTO3} ]
   then
        echo "
      - Install de Boto3 avec version ${VERS_CIBLE}"
        pip3 install boto3==${VERS_CIBLE} >/dev/null 2>$BOTO3_ERR ; CR3=$?
        [ $CR3 -ne 0 ] && cat $BOTO3_ERR
   else
        echo "      - Boto3 already with version ${VERS_CIBLE}"
        CR3=0
   fi


   #J'installe la version Botocore
   if [ ${VERS_CIBLE} != ${VERS_BOTOCORE} ]
   then
        echo "      - Install de Botocore avec version ${VERS_CIBLE}"
        BOTOCORE_ERR=$UNXTMP/${UNXJOB}_$$_botocore.err
        pip3 install botocore==${VERS_CIBLE} >/dev/null 2>$BOTOCORE_ERR ; CR1=$?
        [ $CR1 -ne 0 ] && cat $BOTOCORE_ERR
   else
        echo "      - Botocore already with version ${VERS_CIBLE}"
        CR1=0
   fi
   
   echo "
   AFTER for BOTO"
   pip3 list | grep boto
   echo ""

   VERS_BOTOCORE=$(pip3 list | grep botocore | awk '{print $2}')
   VERS_BOTO3=$(pip3 list | grep boto3 | awk '{print $2}')
   #SI version différente
   if [ ${VERS_BOTO3} != ${VERS_BOTOCORE} ]
   then
        echo "Be careful, problem of version!!!"
   fi

Additional Information/Context

Extract of my log

[03:25:19] :	 [Step 2/2]      2. Update the lastet version of AwsCli, Python and Boto
[03:25:21] :	 [Step 2/2]    INVENTORY for BOTO
[03:25:22] :	 [Step 2/2] boto                  2.49.0
[03:25:22] :	 [Step 2/2] boto3                 1.35.78
[03:25:22] :	 [Step 2/2] botocore              1.36.0
[03:25:23] :	 [Step 2/2]       - Boto3 already with version 1.35.78
[03:25:23] :	 [Step 2/2]       - Install de Botocore avec version 1.35.78
[03:25:27] :	 [Step 2/2] 
[03:25:27] :	 [Step 2/2]    AFTER for BOTO
[03:25:28] :	 [Step 2/2] boto                  2.49.0
[03:25:28] :	 [Step 2/2] boto3                 1.35.78
[03:25:28] :	 [Step 2/2] botocore              1.35.78
[03:25:28] :	 [Step 2/2] 
[03:25:29] :	 [Step 2/2] UPDATE AWSCLI, BOTO, PYTHON => OK
[03:25:29] :	 [Step 2/2] 
[03:25:29] :	 [Step 2/2]      3. Retrieving AccessKey ID and password from TAM
[03:25:30] :	 [Step 2/2] AccessKey ID and password from TAM => OK
[03:25:30] :	 [Step 2/2]      
[03:25:30] :	 [Step 2/2]      4. Check connexion AWS
[03:25:30] :	 [Step 2/2] Traceback (most recent call last):
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/bin/aws", line 27, in <module>
[03:25:30] :	 [Step 2/2]     sys.exit(main())
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/bin/aws", line 23, in main
[03:25:30] :	 [Step 2/2]     return awscli.clidriver.main()
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/clidriver.py", line 73, in main
[03:25:30] :	 [Step 2/2]     driver = create_clidriver()
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/clidriver.py", line 82, in create_clidriver
[03:25:30] :	 [Step 2/2]     load_plugins(
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/plugin.py", line 44, in load_plugins
[03:25:30] :	 [Step 2/2]     modules = _import_plugins(plugin_mapping)
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/plugin.py", line 61, in _import_plugins
[03:25:30] :	 [Step 2/2]     module = __import__(path, fromlist=[module])
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/handlers.py", line 27, in <module>
[03:25:30] :	 [Step 2/2]     from awscli.customizations.cloudformation import (
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/customizations/cloudformation/__init__.py", line 13, in <module>
[03:25:30] :	 [Step 2/2]     from awscli.customizations.cloudformation.package import PackageCommand
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/customizations/cloudformation/package.py", line 26, in <module>
[03:25:30] :	 [Step 2/2]     from awscli.customizations.s3uploader import S3Uploader
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/awscli/customizations/s3uploader.py", line 22, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.manager import TransferManager
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/manager.py", line 25, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.copies import CopySubmissionTask
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/copies.py", line 16, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.tasks import (
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/tasks.py", line 16, in <module>
[03:25:30] :	 [Step 2/2]     from s3transfer.utils import get_callbacks
[03:25:30] :	 [Step 2/2]   File "/home/ubuntu/.local/lib/python3.10/site-packages/s3transfer/utils.py", line 29, in <module>
[03:25:30] :	 [Step 2/2]     from botocore.httpchecksum import DEFAULT_CHECKSUM_ALGORITHM, AwsChunkedWrapper
[03:25:30] :	 [Step 2/2] ImportError: cannot import name 'DEFAULT_CHECKSUM_ALGORITHM' from 'botocore.httpchecksum' (/home/ubuntu/.local/lib/python3.10/site-packages/botocore/httpchecksum.py)
[03:25:30] :	 [Step 2/2] Problem for connexion AWS
[03:25:30] :	 [Step 2/2]      END of processing maj_access_key, bye!!

SDK version used

unknow

Environment details (OS name and version, etc.)

CLOUD_AWS_Ubuntu-22.04_azb (ami-0a71a0754b86976e4) Operating system: Linux, version 6.8.0-1015-aws
@Christhof25 Christhof25 added bug This issue is a confirmed bug. needs-triage This issue or PR still needs to be triaged. labels Jan 17, 2025
@jonathan343
Copy link
Contributor

Hey @Christhof25, thanks for reaching out.

The issue you're seeing is a result of using versions of botocore incompatible with your version of awscli.

The botocore-1.36.0 release introduced new default checksum behavior in the s3 client. With these changes we also introduced a new constant (DEFAULT_CHECKSUM_ALGORITHM) which is used in our s3transfer-0.11.0 dependency, and as a result it's also used in the latest versions of boto3 (v1.36.x) or awscli (v1.37.x). Using any version of botocore < 1.36.0 with the latest versions of awscli and boto3 will result in the import error you're seeing.

I also want to point out that using boto (the boto3 predecessor) is discouraged given the packaged is no longer maintained and last released in 2018. See the following deprecation notice for more info:

This package is no longer maintained and has been replaced by Boto3. Issues and pull requests are not reviewed. If you are having an issue with the Boto3 package or the AWS CLI, please open an issue on their respective repositories.

@Christhof25
Copy link
Author

Christhof25 commented Jan 17, 2025 •
edited
Loading

Thanks @jonathan343 for your analyse.
Can you give me the same number version for boto3 and botocore, please?

@RyanFitzSimmonsAK RyanFitzSimmonsAK self-assigned this Jan 17, 2025
@RyanFitzSimmonsAK RyanFitzSimmonsAK added s3 p2 This is a standard priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Jan 17, 2025
@RyanFitzSimmonsAK
Copy link
Contributor

Thanks @jonathan343 for your analyse. Can you give me the same number version for boto3 and botocore, please?

Both boto3 and botocore are on v1.36.2, and you'll want to use at least v1.36.0 to use the new default checksum behavior.

@RyanFitzSimmonsAK RyanFitzSimmonsAK added the response-requested Waiting on additional information or feedback. label Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@RyanFitzSimmonsAK RyanFitzSimmonsAK

Labels
bug This issue is a confirmed bug. p2 This is a standard priority issue response-requested Waiting on additional information or feedback. s3
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jonathan343 @Christhof25 @RyanFitzSimmonsAK