Support more than rust-lang/rust

[steveklabnik]

This should be able to

1. track commits on more projects that just rust-lang/rust. Let's start by showing rust-lang/cargo.
2. have URLS that are namespaced per project
3. combine appropriate things into each release
4. show who contributed to what in the overall list

This is a big job!

[jmatraszek]

I have started working on that, today in the evening will push some code -- probably not everything will work, but I want at least have populate, and maybe also its friends from /bin, working.

[steveklabnik]

amazing! ❤️

[jmatraszek]

OK, after some struggle with piping commands I was able to make populate work with support for multiple projects. It does not break the web part, however the rest from bin may not work now (hence I am not creating a pull request now). Some notes:

1. I have added a new table named projects. It has a name, path to the source code and a GitHub link (not used now, but in the end we will probably want to link to the repository page);
2. I have also added a new table named contributors to get rid of duplicated name/email in commits table, but in the end I am not using it now. I also have some mixed feelings about it right now, so... let's leave the decision for the future.
3. Migrations will fail if there are some releases existing in the db -- the db should be truncated/dropped before running populate once again. Decided to do so, since we are not running "on prod" and the migration was easier to write that way. If that is a problem then I can rewrite it.
4. I removed the hardcoded release numbers and right now I infer them from git tags (grepping for tags starting with digit or with v followed by a digit). This should work for most projects, but may fail if a project use some non standard tagging schema. Right now should suffice, but we should think about configuring this per project or something like that.
5. Tomorrow I will work on the rest of bin and then on the web view.

P.S. I am falling asleep and I have made some mistakes using commit --amend and push --force xD, so right now GitHub displays wrong link to my changes. The proper one is: jmatraszek@9fe00c0

[steveklabnik]

Thanks @jmatraszek !

I have added a new table named projects. It has a name, path to the source code and a GitHub link (not used now, but in the end we will probably want to link to the repository page);

Path to the source code? I think that's probably not needed; we're not going to be doing this on the server; it's to get stuff going locally, really.

I have also added a new table named contributors to get rid of duplicated name/email in commits table, but in the end I am not using it now. I also have some mixed feelings about it right now, so... let's leave the decision for the future.

I think this is going to end up happening in the future, so if you ended up doing it, no big deal.

Migrations will fail if there are some releases existing in the db -- the db should be truncated/dropped before running populate once again. Decided to do so, since we are not running "on prod" and the migration was easier to write that way. If that is a problem then I can rewrite it.

This is the current design, but we'll probably have to tweak it.

I removed the hardcoded release numbers and right now I infer them from git tags (grepping for tags starting with digit or with v followed by a digit). This should work for most projects, but may fail if a project use some non standard tagging schema. Right now should suffice, but we should think about configuring this per project or something like that.

Yup! For sure.

[steveklabnik]

@jmatraszek if you end up dropping off with this, mind mentioning it here? I'd like to make some changes that will build off of your work, and so I'd be happy to take what you have and run with it. I'd like to have some stuff done in the next seven days. I know that sometimes life happens and "ill do this tomorrow" never comes 😄

[jmatraszek]

Yes, of course, will mention you when this is ready. Unfortunately, today I spent more time struggling with Diesel then I thought I would, so didn't manage to do everything I wanted. But to give a status update:

1. populate works, allows to add multiple projects, assigns commits to the proper releases, etc.
2. update-commit-db also works, iterates over all projects and fetches missing commits. I have also thought about adding a project name parameter, so it would fetch only commits from given project. I do not know which is better for you to run. WDYT?
3. new-releases still needs work (scheduled for tommorow;))
4. the-big-red-button -- I am not sure, is this intended to delete everything or just a single project?
5. "frontend" -- still no work here and I may need some help with that (especially HTML stuff, in which I am really bad)
6. there are still some my experiments with name/github link/github name left in the code... and I am not sure what to decide, so maybe I will contact you on IRC to have a talk about this (as it's easier than GIthub's comments), if you do not mind.

You can check the current state of work here: https://github.com/jmatraszek/contributors/commits/support_multiple_projects

[jmatraszek]

@steveklabnik: pull request created! #57

[steveklabnik]

#57 has done the back-end for this; we'll need to do more front-end work.

#49 is likely a blocker here, as it's really hard to get /rust/1.0.0 vs /cargo/1.0.0 without it.

[steveklabnik]

#65 reverts some parts of #57 , which is unfortunate, but necessary. #63 exposes other issues with trying to be generic over projects.

I am tempted to make config files per-project rather than trying to do weird stuff in the code; there's always going to be these kinds of changes per-project, and the arguments to populate are getting pretty large...