Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Determine how to do resource allocation #21

Open
RaasAhsan opened this issue Oct 31, 2018 · 2 comments
Open

Determine how to do resource allocation #21

RaasAhsan opened this issue Oct 31, 2018 · 2 comments

Comments

@RaasAhsan
Copy link

Hosts have limited resources and therefore they shouldn't be scheduled jobs that would exhaust or leave its resources in high contention.

resources:

  • cpus
  • memory
  • disk space
  • network bandwidth?

Each Gemini node should be able to specify what resources it is offering to the cluster capacity. Every Gemini task/job should specify what resources they require and they should be scheduled on a free Gemini node.

could investigate things like cpu sharing/burst

@JosephMoniz
Copy link
Collaborator

Memory, disk space and bandwidth are fairly intuitive for most people in terms of requesting a % of them or a raw number requirement. How do we represent shares of CPU "units" though? What's the most intuitive way for users?

@RaasAhsan
Copy link
Author

RaasAhsan commented Nov 1, 2018

I was thinking it would just be # of CPUs or # of cores/hyperthreads. I guess we should also consider that hardware isn't homogenous, so we could also develop classes/affinity?

Regarding "shared CPU", I was thinking a node could advertise that it has shared CPUs to offer to the cluster. Any job/task that specifies that its OK to run on shared CPUs could get scheduled on one of those machines (doesn't necessarily have to), and the master could even overschedule those nodes. Basically those jobs/tasks are forced to shared CPUs instead of getting allocated a dedicated one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants