Separate vs monolithic repos

[dereklieu]

Opening for discussion, possibly better fitted for ds-how but I'm putting it here in the hopes that it will affect this repo.

If you've worked on Openroads, you know how much of a pain it can be to have a project where the API lives in a different repo from the database which lives in a different repo from the workers that generate data for the database, not to mention two separate repos for the client side code.

There are reasonable reasons for why we did it, including the macrocosm split and iD being a fork. However, it became prohibitively complex** to do something as important as:

1. run a local database
2. run the worker against the database
3. run the api against the database
4. run the client against the api

The root cause was, though we had a build process for each individual piece, we didn't invest the time to write a build process that would string all the pieces together. While not trivial, it would have prevented a lot of "How do I run this" questions that blocked newcomers from gracefully stepping into the project.

I think storing the code in separate repos contributed to the problem I outlined above. It also strongly separated server-side code from client-side code, which is a distinction that I think is not particularly advantageous for us, given the trade-offs and the fact that it's all usually javascript anyway. It also makes it really hard to do integration testing.

If we had the chance to start Openroads from scratch and knowing all the pieces that would be required, I think I would advocate to put all the parts into a single repo, where a single process could build, stage, and deploy the entire application. This would undoubtedly take more work, but we also paid for not doing it, since we were left with a universe of repositories that were poorly integrated with each other. Deploying interdependent changes also required triggering several different builds, better known to @olafveerman as opening tons of PRs against master.

I'm curious to what other people have experienced. Here are two leading questions but feel free to contribute outside of the bounds I've set here:

1. What positive and negative experiences have you found working in projects where different pieces (client-side vs server-side being just one example) have to interconnect but live in separate git repos?
2. Considering we often build the server-side code now in addition to the client-side code, should we modify project-seed to more gracefully allow dumping an API (for example) into the same repo, which shares a build and possibly deployment script with the existing client-side starter code?

**A lot of this has gotten way way better, props to everyone who worked on it @danielfdsilva @kamicut @mileswwatkins @olafveerman