GitHub?

Posted May 17, 2017 18:50 UTC (Wed) by excors (subscriber, #95769)
In reply to: GitHub? by mathstuf
Parent article: A proposal to move GNOME to GitLab

You could have short-lived VMs with persistent storage, so you can still do incremental builds despite starting a fresh VM each time. Or use separate machines for building and for running the tests. EC2's on-demand GPU instances cost ~50%-100% more than reserved ones per hour, so they're cheaper if you're only using it <50% of the time (after rounding up each use to an integer number of hours).

("Cheaper" is relative of course, it looks like EC2's cheapest modern one (with half of a GRID K520) is around $3K/year reserved, and the ones with more compute power cost more, so probably not worthwhile if you could get away with a cheap consumer GPU and don't need any of the other cloud features.)

GitHub?

Posted May 17, 2017 19:18 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (1 responses)

For clean builds, our build/test cycle takes about 20–40 minutes (depending on platform) machines on 10-core CPUs with hyperthreading for one project; another takes at least 45 on the same hardware. Incremental builds can be as small as 5 minutes, 15 for the larger project. The machines cost ~$2500 up front and can run multiple builds (depending on the project). I don't know what the electricity costs. There's also a benefit in being able to sit down with a machine to see what's gone wrong on it (helpful when you're dealing with things like rendering differences).

GitHub?

Posted May 17, 2017 20:00 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Yes, for GPU it might make sense to do local builds.

BTW, one of our clients uses pre-built containers to run stuff on expensive instances (with 1Tb of RAM). The build is handled on cheap instance types and only the final containers are run on the expensive instances that are spun down once the calculation is over.

You can also sometimes get GPU instances on EC2 Spot Instances for very cheap.