Live cloud visualization using diagrams and boto3

[AbhiSaridenaTR]

I am working on a project to create automated aws architecutre diagrams using real time data (similar to what lucidscale, cloudcraft, etc.). I'm using the python libraries boto3 and diagrams for this task. boto3 to extract data from the aws console and diagrams to create the architecture diagrams. I'm stuck now and can't figure out a way to obtain the connections between resources. Is there any way using these two libraries(or any other python libraries) that I can obtain the connections/ flow between resources. For example, how can I figure out that an ec2 instance is talking to an s3 bucket. Also is this a viable project or am I being too ambitious??

[tim-finnigan]

Hi @AbhiSaridenaTR thanks for reaching out. There are probably a few different possibilities you could explore here. I think AWS CloudTrail is a good option for tracking logs of API actions. Here is documentation for reference: https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-examples.html.

Also you can refer to this EC2 User Guide documentation for more detail for your specific use case: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/using-cloudtrail.html. And here is the Boto3 documentation for using CloudTrail: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudtrail.html.

You could also look into Workload Discovery described here for visualizing diagrams: https://aws.amazon.com/blogs/mt/visualizing-resources-with-workload-discovery-on-aws/. And here you can find architecture icons and several diagramming tools: https://aws.amazon.com/architecture/icons/.

I hope that helps! Please let us know if you had any follow up questions or more details to share on your use case.